How to Validate Parsed Financial Statements

Your parser runs. The statements look right. And then, 3 weeks later, a downstream chart is off by an amount nobody can explain. You dig in and find gross profit that does not equal revenue minus cost of revenue, or total assets that refuse to match liabilities plus equity.

Nothing threw an error, of course. A wrong number is still a number. This is the defining hazard of parsed financial data, and I want to name it properly: the failures are silent, they are entirely plausible, and they surface far downstream from where they were born, which is where they are hardest to trace.

The only real defence is validation — checking the data against the identities that accounting guarantees. Let me show you how to do that well, rather than superstitiously.

Why parsed statements fail to reconcile

Every failure mode from parsing shows up here as a broken identity, which is what makes identities such a good tripwire — you do not have to anticipate each bug, you just have to notice the sums stop working:

• A sign error — a subtracted line added instead — makes a subtotal miss.
• A wrong context — a year-to-date figure taken for a quarter, or an instant value read as a duration — makes quarters not sum to the year.
• An unmapped tag leaves a child out of its subtotal, so the subtotal is short.
• A scale error on a non-XBRL table puts one line off by a factor of 1,000, breaking the balance-sheet identity.
• A dimensional double-count sums a total together with its own segment members, inflating the roll-up.

The sign-convention minefield

Those 5 failure modes are the taxonomy of what goes wrong. But if you ask me which one you probably have right now, the answer is almost always the same.

If I had to bet on which bug you have, I would bet on this one. Sign is the single most common source of a subtotal that misses by a plausible amount, because financial statements are not internally consistent about it and neither are filers. Expenses are sometimes stored as positive numbers that you subtract, and sometimes as negative numbers that you add. Contra accounts — treasury stock, accumulated depreciation, allowance for doubtful accounts — are reductions carried with a sign you have to respect. In XBRL, the calculation linkbase assigns each child a weight of +1 or -1 that tells you whether it adds to or subtracts from its parent; ignore the weight and your roll-up is right in magnitude and wrong in sign for every contra item.

The trap is that a sign error rarely makes a number look absurd. Add an expense you should have subtracted and operating income is too high by twice the expense — still a plausible figure, still passing any “is this a reasonable number” sniff test, and failing only the articulation check. This is exactly why you validate against the identity rather than against your intuition about what looks right.

Scale, context, and the dimensional double-count

3 more structural traps break identities without any sign error at all. Scale: XBRL carries a decimals attribute (a value of -3 means thousands, -6 means millions) and non-XBRL tables state their scale once in a header and never again, so a parser that misses it reads one line in units and the rest in millions, and the balance sheet is off by 6 orders of magnitude on a single row. Context: balance-sheet items are instant facts at a point in time, while income and cash-flow items are duration facts over a period; pull an instant into a duration slot and periods stop summing. Dimensions: the same concept appears once for the consolidated total and again for each segment member, and summing the total with its members double-counts it. Each of these produces a clean-looking statement that quietly fails to add up — and each is caught by the same identities, if you have the structure to state them.

The identities worth checking

You do not need many checks. A handful catches the overwhelming majority of parse defects, and I would rather you ran 3 faithfully than wrote 12 and disabled them. The first 3 below are the core; the last 2 are cheap enough that you may as well:

1. The balance sheet balances. Total assets equal total liabilities plus total equity. This is the single most valuable check, because it exercises the whole balance sheet at once — a scale error, a sign error or a dropped line anywhere in it shows up here.
2. Subtotals articulate. Each subtotal equals the sum of the line items beneath it up to the previous subtotal — gross profit from revenue and cost of revenue, operating income from gross profit and operating expenses, and so on.
3. Periods articulate. For flow statements, the quarters of a year sum to the annual figure, within reason. This one carries a caveat — restatements legitimately break it — covered in handling restatements.
4. Cash flow reconciles. Operating plus investing plus financing cash flows (plus the exchange-rate effect) equal the net change in cash, and that net change equals the change in the cash line on the balance sheet between the 2 period-ends. This ties the cash-flow statement to the balance sheet and catches a whole class of context and sign errors.
5. Retained earnings roll forward. Beginning retained earnings plus net income minus dividends equals ending retained earnings. When it fails, it usually points at a misread net income or a missed dividend line.

The hard part of DIY validation is structure, not arithmetic

So you have identities that catch every failure mode listed above. The obvious next step is to code them. And this is where it stalls, for a reason that is not obvious until you try.

Now, look at the arithmetic in those checks. It is trivial — subtraction and addition. So why is validation hard? Because on a raw parse, a flat list of facts does not tell you which rows are subtotals or which children roll into them. To check that gross profit articulates, you first have to know that gross profit is a subtotal, and that revenue and cost of revenue are its children — structure that lives in the XBRL calculation linkbase you may not have parsed, and does not exist at all in a non-XBRL 6-K or 8-K table. Without that structure you cannot even state the identity, let alone check it. So on a home-grown parse, the “easy” validation step turns into a second parsing project: reconstruct the statement hierarchy before you can test it.

Validate with the structure the response already carries

The Financial Statements API returns that structure inline: every line item has an itemtype of lineitem, subtotal or grandtotal, and an sno_total that fixes the order. So you never have to guess which rows are totals. Write each subtotal as a signed sum of the lines it is built from, check it in every period, and let itemtype list the subtotals you have not written a rule for yet, so none is skipped silently. No linkbase required:

import requests

API_KEY  = "YOUR_API_KEY"
BASE_URL = "https://data.businessquant.com"

def get_statement(ticker, statement, frequency="Annual", period="5y"):
    params = {"ticker": ticker, "statement": statement, "frequency": frequency,
              "period": period, "api_key": API_KEY}
    r = requests.get(f"{BASE_URL}/statements", params=params, timeout=30)
    r.raise_for_status()
    return r.json()

def line_items(nodes):
    """Yield every leaf line item, however deep the tree goes."""
    for node in nodes.values():
        if "sections" in node:
            yield from line_items(node["sections"])
        else:
            yield node

def by_slug(doc):
    """{slug: (sno_total, itemtype, {date: raw})}, with the frequency suffix stripped."""
    suffix = "-" + doc["metadata"]["frequency"].lower()     # revenue-annual -> revenue
    out = {}
    for item in line_items(doc["data"]):
        m = item["metadata"]
        out[m["slug"].removesuffix(suffix)] = (
            m["sno_total"], m.get("itemtype"),
            {v["date"]: v["reportedValue"]["raw"] for v in item["values"]})
    return out

def close(actual, expected, tol=0.01):
    return abs(actual - expected) <= abs(expected) * tol + 1

# Each subtotal as a signed sum. Expenses arrive as positive numbers,
# so the signs live in the rule, not in the data.
RULES = {
    "gross-profit":     {"revenue": +1, "cost-of-revenue": -1},
    "operating-income": {"gross-profit": +1, "operating-expenses": -1},
    "ebt":              {"operating-income": +1, "non-operating-income": +1},
    "profit-after-tax": {"ebt": +1, "tax-provisions": -1},
}

def check_subtotals(doc):
    lines, problems = by_slug(doc), []
    for total, parts in RULES.items():
        if total not in lines or any(p not in lines for p in parts):
            continue                      # not in this template: a bank has no gross profit
        for date, raw in lines[total][2].items():
            if all(date in lines[p][2] for p in parts):
                expected = sum(sign * lines[p][2][date] for p, sign in parts.items())
                if not close(raw, expected):
                    problems.append((total, date, expected, raw))
    # itemtype flags every total, so a subtotal with no rule is reported, never skipped
    unchecked = [slug for slug, (sno, itemtype, vals) in sorted(lines.items(), key=lambda kv: kv[1][0])
                 if itemtype in ("subtotal", "grandtotal") and slug not in RULES]
    return problems, unchecked

problems, unchecked = check_subtotals(get_statement("AAPL", "IS"))
print("subtotal issues:", problems)      # [] when the statement articulates
print("no rule yet:", unchecked)

2 details in that code matter. The signs live in RULES because expenses such as cost-of-revenue arrive as positive numbers, so a blind running sum would add them instead of subtracting them. And the tree is walked recursively because its depth varies: income-statement lines sit directly under their category, while balance-sheet lines sit one level deeper, inside subsections such as Current Assets.

The balance-sheet identity in 3 lines

The subtotal check above validates the internal arithmetic of a single statement. The balance-sheet identity validates the whole sheet at once, and it is even shorter.

Because line items carry stable slugs and the grand totals are flagged, the balance check is a lookup, not a parse. The slugs to look up are assets, total-liabilities and common-equity; on annual data each carries an -annual suffix, which by_slug strips. There is no total-assets or total-equity slug, and a lookup on either would quietly return nothing:

bs = by_slug(get_statement("AAPL", "BS"))
assets, liabilities, equity = (bs[s][2] for s in ("assets", "total-liabilities", "common-equity"))
for date in assets:
    print(date, "balances:", close(liabilities[date] + equity[date], assets[date]))

A cash-flow reconciliation check

The subtotal check and the balance-sheet identity both stay within one statement. The next check ties 2 statements together, which catches a class of errors neither single-statement check can see.

The same slug lookup extends to the cross-statement tie. First, the 3 activity totals plus the exchange-rate effect should add up to change-in-cash. Then that net change should match the change in the balance-sheet cash-and-equivalents line between consecutive period-ends. This catches errors the single-statement checks miss, because it forces 2 statements to agree:

cf = by_slug(get_statement("AAPL", "CF"))
bs = by_slug(get_statement("AAPL", "BS"))

net_change = cf["change-in-cash"][2]
flows = [cf[s][2] for s in ("cash-from-operations", "cash-from-investing-activities",
                            "cash-from-financing-activities", "exchange-rate-effect") if s in cf]
cash  = bs["cash-and-equivalents"][2]
dates = sorted(cash)
for prev, cur in zip(dates, dates[1:]):
    flows_ok = close(sum(f.get(cur, 0) for f in flows), net_change[cur])
    gap = net_change[cur] - (cash[cur] - cash[prev])      # 0 unless restricted cash moved
    print(cur, "flows add up:", flows_ok, "| gap to balance-sheet cash:", f"{gap:,.0f}")

Read the second number as a prompt to look, not as a verdict. Since 2018 the cash flow statement reconciles cash plus restricted cash, while the balance-sheet line holds unrestricted cash only, so a company that moves money in or out of restricted accounts shows a gap here even when every number is right. Compare a gap with the restricted cash disclosed in the filing before you treat it as a parse defect; if the 2 match, the data is fine.

On the API side these identities are checked during ingestion, so a statement that failed to articulate does not ship as if it were fine. Running them again in your own pipeline is still worth it as a guard on anything you compute downstream — validation you can afford precisely because the structure is handed to you rather than reconstructed.

Cross-check against the filer’s own calculation linkbase

Everything above validates against the accounting identities you define. There is one more layer available — the filer’s own declared roll-ups, straight from the filing.

If you want a truly independent oracle — and for anything you trade on, I think you should — compare against the roll-ups the filer itself declared. The XBRL calculation linkbase (EX-101.CAL) states which child concepts sum into which parent, with the sign weights — the company’s own articulation rules, straight from the filing. Fetch it clean by document type through the Documents and Exhibits API and you have a source-of-truth to check a normalized subtotal against. You get the raw linkbase files individually addressable, which is exactly what you want here, because a pre-flattened feed throws away the parent-child weights this check depends on:

def calc_linkbase(ticker):
    f = requests.get(f"{BASE_URL}/secfilings"
                     f"?ticker={ticker}&formtype=10-K&page_size=1&api_key={API_KEY}"
                     ).json()["data"][0]
    accession = f["accession"]
    manifest = requests.get(
        f"{BASE_URL}/secfilings/{accession}/documents?api_key={API_KEY}").json()["data"]
    cal = next(d for d in manifest if d["document_type"] == "EX-101.CAL")
    url = (f"{BASE_URL}/secfilings/{accession}/content"
           f"?filename={cal['filename']}&api_key={API_KEY}")
    return requests.get(url).text        # the filer's own parent/child roll-ups

# Parse the calculation arcs, then confirm each declared parent equals the
# weighted sum of its children -- an oracle sourced from the filing itself.

One caveat on the oracle: the calculation linkbase is authoritative where it exists, but it is not always complete. Some relationships a filer presents on the face of the statement are not declared as calculation arcs, and IFRS filings sometimes carry sparser linkbases. Treat it as a strong independent check on the relationships it does declare, not as a full statement of every roll-up.

Edge cases to keep in mind

The checks above cover the main path. A few practical details keep them from false-alarming on legitimate data.

Use a tolerance. Filings round to thousands or millions, so exact equality will false-alarm. Compare within a small relative tolerance plus a small absolute floor, as above.

Some lines are intentionally non-additive. Per-share figures, ratios and memo lines are not part of a sum. Validate the additive statements — income statement, balance sheet, cash flow — and treat the Ratios and Growth views as derived, not summable.

Legitimate negatives are not errors. Net losses, negative working capital, and negative shareholders’ equity from years of buybacks are all real and correct. A naive “these must be positive” rule will flag healthy companies; validate the identities, not the signs of individual totals.

Restatements break period articulation legitimately. If 4 quarters do not sum to the year, a restatement may be the honest reason rather than a parse bug. Reconcile against the latest reported values, as described in the restatements article.

Start Building

Here is the thought I would leave you with. A statement that does not reconcile is the loudest signal you will ever get that a parse went wrong — and in a domain where every other failure is silent, that is worth a great deal. But you can only check an identity if you know which rows are subtotals, how they roll up, and which sign each one carries.

With itemtype and sno_total arriving in every response, that structure is simply given to you. Which means the balance-sheet identity, the subtotal articulation and the cash tie are a few lines each. The API runs them on ingestion. Run them again in your own guardrails anyway — I would.

• Financial Statements API — statements with itemtype and sno_total on every line.
• Documents and Exhibits API — the raw EX-101.CAL calculation linkbase, for an independent oracle.
• SEC Filings API — the underlying filings, if you want to trace a discrepancy to the source.

Get an API key, run the balance check on a name you already track, and wire it into your load step.