Extract Risk Factors and MD&A From a 10-K in Python

You write a regex that pulls Risk Factors out of a 10-K. You test it on Apple, it works, and you move on. Then a colleague asks why so many of your results are table-of-contents fragments instead of actual risk disclosure.

That is the moment you discover that a 10-K section extractor is not a function — it is a maintained system with 6 independent failure modes, all of which fail silently. I am going to walk you through every one of them, explain why free parsers do not fix the problem, and then ask the question you will eventually land on anyway: what if you stop parsing altogether?

6 reasons regex 10-K section extraction fails

We have all written the function that causes this. Mine looked almost exactly like this:

import re

def get_risk_factors(html_text):
    start = re.search(r"Item\s*1A[\.\s]*Risk\s*Factors", html_text, re.I)
    end   = re.search(r"Item\s*1B[\.\s]*Unresolved", html_text, re.I)
    return html_text[start.end():end.start()]

Notice I did not say the parser “is fragile.” It fails. Each of the 6 below is a distinct failure mode with its own cause, and your parser has to survive all of them simultaneously. I will take them in the order they tend to bite.

1. The table of contents matches first

Every 10-K opens with a TOC containing the literal string “Item 1A. Risk Factors” — hyperlinked, sometimes with a page number, sometimes inside a table cell. Your re.search finds that one, and your extracted section is whatever sits between the TOC entry for 1A and the TOC entry for 1B. Which is nothing.

The standard fix is to take the last match, or the second, or the match past some byte offset. All 3 break on a real filing somewhere in your universe: some filers have no TOC, some have 2, some 10-Ks reference “Item 1A” again inside Item 7, and some incorporate the risk factors by reference from a different document entirely.

2. Filers do not agree on how to write a heading

Your regex assumes headings follow a pattern. They do not. Here are real headings from production filings, every one of them valid:

Item 1A.    Risk Factors
ITEM 1A - RISK FACTORS
Item 1A: Risk Factors
Item 1A.Risk Factors
ITEM 1A.  RISK FACTORS.
Item 1A.  Risk Factors  (continued)

And then the non-breaking spaces. Look at real filing HTML and you will find Item 1A.  Risk Factors. Your \s* does not match \xa0 unless you normalise first — so you normalise — and now you have merged words that were separated by markup boundaries, introducing corruption in parts of the document you were not even trying to touch.

3. The heading is not text, it is a box model

This is the one that ended the argument for me. In modern EDGAR filings a heading is frequently split across nested elements purely for styling:

<div style="padding-left:45pt;text-indent:-45pt;">
  <span style="font-weight:700;">Item&#160;1A.</span>
  <span style="font-weight:700;">&#160;&#160;Risk Factors</span>
</div>

In the raw HTML source, the string Item 1A. Risk Factors does not exist. There is no contiguous run of characters your regex can match. You have to render to text first — which means an HTML parser, which means deciding how to handle tables, block elements and whitespace collapsing. Every one of those decisions shifts your section boundaries by a few characters in a direction you cannot predict, and the shift is different on every filing.

4. Inline XBRL shreds the text layer

Modern filings are iXBRL, which means every numeric and narrative fact in the document is wrapped in ix: elements carrying the XBRL tag, the unit and the scale alongside the display value. Your text extractor has to understand that namespace, or it will mangle the numbers:

<ix:nonFraction name="us-gaap:Revenues" scale="6" unitRef="usd">394,328</ix:nonFraction>

Feed that to a text extractor that does not know the ix: namespace and you get orphaned fragments, doubled values, and garbled numbers. The fact appears once for display and once hidden inside an <ix:hidden> block, and a naive extractor dumps both into the output. At worst, several thousand characters of invisible tagged facts land in the middle of your MD&A with no indication they are there.

5. Not every filing has the section, and not every section is where you think

• Item 1B (Unresolved Staff Comments) is frequently a single line reading “None.” If you are using Item 1B as the end anchor for Item 1A and the filer omitted it, your Risk Factors section now runs to the end of the document.
• Item 6 (Selected Financial Data) has been effectively retired. Boundary logic that assumes items run 1 → 1A → 1B → 2 with no gaps will overshoot.
• Trusts retitle things. Royalty and structured trusts routinely omit MD&A or retitle it “Trustee’s Discussion and Analysis”. Your Item 7. Management's Discussion anchor finds nothing, and you record a null where there was real content.
• Roughly 1 in 1,000 filers scatter sections non-sequentially — Item 1 and Item 7 distributed across pages that do not appear in document order. “Everything between anchor A and anchor B” produces garbage on these, silently.

The common thread is that your parser assumes a structure the spec does not actually guarantee. And the penalty for a wrong assumption is never an error — it is text that looks right.

6. Pre-2002 filings have no standardised structure at all

Anything before Sarbanes-Oxley predates the item conventions you are anchoring on. If your backtest reaches back to 1995, a meaningful chunk of your corpus simply does not have the shape your parser assumes — and your regex returns whatever happens to sit between 2 strings that coincidentally match, with no indication anything went wrong.

Why free 10-K parsers do not solve this either

So your next move is obvious: reach for an open-source 10-K parser instead of writing your own. That was my next move too. It is a reasonable instinct, and for a single company on a recent filing it usually works fine. Here is what happened when I pointed one at an entire universe.

• They are heuristic stacks, and heuristics are tuned to a corpus. An extractor tuned on large-cap US filers meets a royalty trust, an ADR or a 1997 10-K and degrades — not with an exception, but with a plausible-looking string that is the wrong string.
• They fail silently, which is the expensive part. You get text back. It is 180 characters of table of contents, or Item 1A plus all of Item 2 because the end anchor was missing. Nothing raises. Your panel has a column that looks populated and is quietly wrong, and you find out from a portfolio manager rather than from a stack trace.
• They cannot tell “not disclosed” from “not extracted”. Both come back as an empty string or a None. Those are completely different facts — one is a finding about the company, the other is a bug in your pipeline — and collapsing them is how you end up presenting an analysis where a fifth of the sample is missing for reasons nobody can reconstruct 6 months later.
• Form coverage stops at 10-K. Most stop there. 10-Q section keys are different (see below), 8-K uses another convention again, and 20-F, S-1 and F-1 are usually not covered at all — which is a shame, because ADRs and IPOs are exactly where narrative disclosure carries the most information.
• You inherit the maintenance. Every new company you add is a potential new special case. The teams that do this well end up with a few thousand lines of Python, a golden-set regression suite, and a job nobody wants to own.
• Parsing is slow, and you are doing it every time. Extraction at query time means re-parsing multi-megabyte HTML on every run of every backtest.

Each of these looks manageable in isolation. The problem is that they compound. You adopt a parser, it works for a quarter, and then your universe widens or a new filing season arrives with slightly different formatting — and the heuristics miss, silently, in a way you will not discover until someone downstream acts on the data.

Let me put it plainly, because I wish someone had said this to me: a free parser is not free. It is a maintained system with a silent failure mode, and the bill lands on whoever happens to be holding it the day the universe widens.

Extract 10-K sections with 2 API calls

So here is the position you are in. Writing your own parser fails silently. Reaching for a free one just hands you someone else’s heuristic stack to maintain. Both paths end in the same place — a system you own and a backlog you feed. The way out is to stop parsing altogether and let someone else do it at ingestion time, before you ever ask for the section.

That is what the Business Quant Section Extraction API does. When a filing lands on EDGAR, its sections are parsed, cleaned and stored before you ask for them — across 6 form types: 10-K, 10-Q, 8-K, 20-F, S-1 and F-1. Retrieval is a lookup, not a parse, so it comes back in sub-second across the full historical corpus.

Call 1 — what sections does this filing actually have?

curl "https://data.businessquant.com/secfilings/0000320193-25-000079/items?api_key=YOUR_API_KEY"

{
  "metadata": {"ticker": "AAPL", "accession": "0000320193-25-000079", "total_records": 22},
  "data": [
    {"item_sno": 1,  "item_key": "item1"},
    {"item_sno": 2,  "item_key": "item1a"},
    {"item_sno": 4,  "item_key": "item1c"},
    {"item_sno": 10, "item_key": "item7"}
  ]
}

Treat this endpoint as your source of truth. A section not listed here was not extracted and cannot be fetched. I want to be clear that this is not a limitation you have to work around — it is genuinely useful. It is the answer to “did this filer include Item 1C this year?”, a question you would otherwise have to answer by parsing.

Call 2 — give me that section

curl "https://data.businessquant.com/secfilings/0000320193-25-000079/content?item_key=item1a&api_key=YOUR_API_KEY"

The response is raw HTML, not JSON — Content-Type: text/html, containing only the Risk Factors section. No TOC. No exhibit index. No neighbouring items bleeding in. Feed it to BeautifulSoup, to a chunker, to an LLM context window, done.

Working code: pull Item 1A and Item 7 in Python

The curl calls above showed the concept. Here is the same pattern in Python. The get_section function takes an accession and an item key; pass as_text=True and it strips the HTML for you, pass False and you get clean HTML with tables intact — which is what you want if you are feeding the section to an LLM or running your own extraction on the markup.

import requests
from bs4 import BeautifulSoup

API_KEY, BASE = "YOUR_API_KEY", "https://data.businessquant.com"
session = requests.Session()

def latest_filing(ticker, formtype="10-K"):
    r = session.get(f"{BASE}/secfilings", params={
        "ticker": ticker, "formtype": formtype, "page_size": 1, "api_key": API_KEY})
    rows = r.json()["data"]
    return rows[0] if rows else None

def get_section(accession, item_key, as_text=True):
    r = session.get(f"{BASE}/secfilings/{accession}/content", params={
        "item_key": item_key, "api_key": API_KEY}, timeout=60)
    r.raise_for_status()
    if not as_text:
        return r.text                       # clean HTML, tables intact
    return BeautifulSoup(r.text, "html.parser").get_text("\n", strip=True)

filing = latest_filing("AAPL")
risk = get_section(filing["accession"], "item1a")
mdna = get_section(filing["accession"], "item7")

print(len(risk.split()), "words of Risk Factors")

That is it. No regex, no heading heuristics, no silent failures. If the section exists you get clean HTML. If it does not, the /items endpoint already told you — so your pipeline knows the difference before it ever makes the second call.

A shortcut worth knowing

You do not strictly need call 1. Every filing record returned by the SEC Filings API already contains an items[] array listing the sections detected in it. So a single search call gives you both the accessions and the routing table. For bulk jobs, this is the difference between 2N and N+1 requests.

The complete 10-K item key reference

You have the functions. Now you need the right keys. These are the exact strings you pass to the /content endpoint, and I am listing all of them because the naming is not always intuitive — item1c is Cybersecurity, item9a is Controls and Procedures, item11 is Executive Compensation — and passing the wrong one gives you a 404 instead of a helpful suggestion.

Key	Section	Why you would want it
item1	Business	Competitive intel, segment taxonomy
item1a	Risk Factors	Year-over-year risk diffing
item1b	Unresolved Staff Comments	Usually “None” — its presence is signal
item1c	Cybersecurity	Post-2023 mandate; new disclosure surface
item3	Legal Proceedings	Litigation exposure tracking
item5	Market for Common Equity	Buyback activity, issuer purchases
item7	MD&A	Tone/sentiment, guidance language
item7a	Quantitative & Qualitative Market Risk	Rate/FX/commodity exposure
item8	Financial Statements	The statements and the footnotes
item9	Changes in Accountants	Auditor-change red flag
item9a	Controls and Procedures	SOX / material weakness monitoring
item11	Executive Compensation	Narrative comp disclosure
item13	Related Transactions	Related-party red flags

The 10-Q gotcha that costs everyone an afternoon

10-Q keys are prefixed by Part. This is not a stylistic quirk: a 10-Q has an Item 1 in Part I (Financial Statements) and an Item 1 in Part II (Legal Proceedings). They are different sections with the same number.

Key	Section
part1_item1	Financial Statements
part1_item2	MD&A — not item7
part1_item3	Market Risk
part2_item1	Legal Proceedings
part2_item1a	Risk Factors — not item1a
part2_item2	Unregistered Sales of Equity

If you hardcode item1a and run it across a mixed 10-K/10-Q corpus, every 10-Q silently returns nothing. Read the form type off the filing record and route accordingly:

RISK_KEY = {"10-K": "item1a", "10-Q": "part2_item1a", "20-F": "Item3D", "S-1": "RiskFactors"}
MDNA_KEY = {"10-K": "item7",  "10-Q": "part1_item2",  "20-F": "Item5",  "S-1": "MDA"}

risk = get_section(filing["accession"], RISK_KEY[filing["formtype"]])

8-K event codes

8-K keys use a different convention again: Item2_02, Item5_02, Item1_05. Do not lowercase them. The safe universal rule: whatever the /items endpoint hands you, pass back verbatim.

Key	Event	Use case
Item1_05	Material Cybersecurity Incident	Breach monitoring
Item2_01	Completion of Acquisition	M&A close detection
Item2_02	Results of Operations	Earnings, the moment they file
Item2_06	Material Impairments	Writedown detection
Item3_01	Notice of Delisting	Distress signal
Item4_02	Non-Reliance (restatement)	The strongest accounting red flag
Item5_02	Departure/Election of Officers	Executive turnover
Item7_01	Regulation FD Disclosure	Guidance updates

20-F, S-1 and F-1

20-F (foreign private issuers) uses its own numbering: Item3D is Risk Factors, Item5 is the Operating & Financial Review (their MD&A), Item16K is Cybersecurity.

S-1 / F-1 keys are semantic rather than numeric, because prospectuses are not numbered: RiskFactors, UseOfProceeds, Dilution, Capitalization, Underwriting, MDA, Business, ProspectusSummary, ExecutiveCompensation. F-1 adds the cross-border ones: DescriptionOfADS, ExchangeRateInfo, ServiceOfProcess.

Bulk section extraction across a universe

The single-filing case was a warm-up. The reason you actually want section extraction is to run it across 500 or 3,000 companies and get a panel out the other end. One thing I would ask you to keep, even if you rewrite everything else here: the status column. Always distinguish “the filer did not disclose it” from “my pipeline failed”. Collapsing both into None is exactly the silent failure I described above, and you will not catch it later.

import pandas as pd
from concurrent.futures import ThreadPoolExecutor

def extract_one(ticker, formtype="10-K", years=10):
    """One row per filing: ticker, period, section text, and WHY it is missing."""
    r = session.get(f"{BASE}/secfilings", params={
        "ticker": ticker, "formtype": formtype, "page_size": 1000, "api_key": API_KEY})

    rows = []
    for filing in r.json()["data"][:years]:
        key = RISK_KEY[filing["formtype"]]
        row = {"ticker": ticker, "period": filing["reportperiod"], "text": None}

        if key not in filing["items"]:      # items[] ships with the filing record
            rows.append({**row, "status": "not_disclosed"})
            continue
        try:
            rows.append({**row, "text": get_section(filing["accession"], key), "status": "ok"})
        except requests.HTTPError as e:
            rows.append({**row, "status": f"error_{e.response.status_code}"})
    return rows

with ThreadPoolExecutor(max_workers=8) as pool:
    records = [r for rs in pool.map(extract_one, ["AAPL", "MSFT", "NVDA", "JPM"]) for r in rs]

df = pd.DataFrame(records)
df["word_count"] = df["text"].str.split().str.len()
df.to_parquet("risk_factors_panel.parquet")

Exclusions before a universe-scale run

Some entity types do not produce standard narrative sections at all, and including them just fills your failure log with noise. Drop security categories ETF, UNIT and CEF, SIC code 6189 (asset-backed securities) and SIC code 6798 (REITs, whose section structure is often non-standard). The Stocks Profile API carries both fields, so this is a filter, not a research project.

What you do with it: year-over-year risk diffing

Extraction is the boring half, and we are through it. Here is the payoff, and it is my favourite thing to do with this data — detecting what a company started saying and, more interestingly, what it stopped saying. The idea is simple: pull the same section from 2 consecutive annual filings, split both into paragraphs, and use sequence matching to find what appeared and what vanished:

import difflib, re

def paragraphs(text):
    return [p.strip() for p in re.split(r"\n+", text) if len(p.strip()) > 120]

def closest(para, corpus):
    return max((difflib.SequenceMatcher(None, para, p).ratio() for p in corpus), default=0.0)

def risk_delta(prior_text, current_text, threshold=0.75):
    prior, current = paragraphs(prior_text), paragraphs(current_text)
    return {
        "added":   [p for p in current if closest(p, prior)   < threshold],
        "removed": [p for p in prior   if closest(p, current) < threshold],
    }

New risk paragraphs are the interesting object here. When a company adds a paragraph about supply concentration, a regulatory regime or a customer dependency, it is telling you something its lawyers made it tell you. Which is the most reliable kind of disclosure there is. And when a company quietly removes one? That is telling you something too.

The same pattern on item7 (MD&A) surfaces tone shifts. On item1c (Cybersecurity) across a universe it gives you a compliance-posture panel that did not exist before 2023. On part2_item1a in 10-Qs it gives you a quarterly risk-update cadence instead of an annual one — because that is where mid-year risk changes actually get disclosed.

Honest boundaries

A guide that only lists strengths is marketing, not a guide. So here is where I fall short.

Sections come back as HTML, not structured data. item8 returns the statements-and-footnotes section — clean, complete, and full of <table> elements. It is narrative-plus-tables, not a normalised fact set. If what you want is “revenue for FY2025 as a float”, do not parse Item 8 — that is the wrong tool.

Executive compensation is narrative, not a table. item11 gives you the compensation discussion, not a structured comp database with a row per named executive officer.

Exhibits are not items. EX-21 subsidiary lists, EX-10 material contracts and EX-99.1 press releases live in the submission’s file manifest, not the item dictionary. Use the Documents & Exhibits API for those.

Extraction is not 100% on a corpus this messy, and no honest vendor claims it is. Trusts retitle MD&A. Pre-2002 filings lack the structure entirely. Roughly 1 in 1,000 filers scatter sections in ways that defeat boundary detection. That is precisely why the /items endpoint exists as a separate call: it tells you what was extracted, so your pipeline knows the difference between an empty section and a missing one.

The short version

Here is what I would tell you over coffee. The 10-K parser you are about to write has 6 independent failure modes, it fails silently on every one of them, and it needs a new special case each time you widen the universe. It is not a function. It is a maintained system with an owner and a backlog. And reaching for a free parser does not remove that maintenance, it just moves it onto someone else’s shoulders and then back onto yours.

That is the problem Business Quant built the Section Extraction API to solve. It pre-extracts every section from 10-K, 10-Q, 8-K, 20-F, S-1 and F-1 filings at ingestion time and serves them as clean HTML in sub-second lookups, across the full historical corpus. It is free to use.

2 calls. /items to see what is there, /content to go get it. Then delete the regex and take the rest of the afternoon off.