Financial Statement Analysis with Large Language Models
LLMs can out-predict analysts on earnings, but only if you manage the risk
A widely cited University of Chicago Booth study found that GPT4, given nothing but a standardized, anonymized balance sheet and income statement, predicted the direction of a company’s future earnings more accurately than the median human analyst. No footnotes. No industry context. Additionally, no company name. Just numbers and a chain-of-thought prompt.
That result gets repeated constantly in finance and AI circles, usually as proof that large language models are ready to replace a chunk of equity research. What gets repeated far less often is the other half of the evidence: the same model class produces hallucinated financial figures in a meaningful share of unaided cases, and its arithmetic is unreliable enough that serious implementations wrap it in verification layers before letting it near a real filing. Both things are true at once, and any analyst evaluating financial statement analysis with large language models needs both halves of the picture.
What the research actually found
The Kim, Muhn, and Nikolaev study (Chicago Booth, 2024) tested something narrower than “can AI do financial analysis.” Researchers fed GPT4 standardized financial statements stripped of company identity, industry, and narrative disclosures, then used a structured, step-by-step prompt designed to mirror how a trained analyst reasons through a filing.
Under those specific conditions, three findings held up:
- The model predicted the direction of earnings changes more accurately than the median sell-side analyst, and the advantage was largest in situations where human analysts typically struggle.
- Prediction accuracy roughly matched a purpose-built neural network trained specifically for the task, using only general-purpose reasoning rather than domain-specific training.
- Trading strategies built on the model’s predictions produced higher Sharpe ratios than strategies built on other quantitative approaches, meaning better risk-adjusted returns rather than just better raw returns.
The caveat that matters: this is a test of numerical pattern recognition under controlled, structured conditions, not a test of an LLM reading a messy 10-K and forming an independent view. The gap between those two things is where the rest of this article lives.
Where LLMs genuinely help today
Set aside the earnings-prediction headline and look at where financial teams are actually deploying this technology day to day. The value clusters around three jobs.
Extraction and standardization. Financial statements mix structured tables with long, dense narrative sections such as footnotes and MD&A. Reading a decade of 10-Ks by hand to reconcile line-item labels that changed twice is a real cost center. An LLM can pull line items into a standardized schema, harmonize multi-year figures, and flag where a label shifted from one filing to the next, which cuts manual reconciliation time substantially.
Ratio calculation and trend detection. Once data is standardized, models handle the mechanical layer of ratio analysis reasonably well: liquidity measures like the current and quick ratio, profitability metrics such as gross margin, operating margin, and return on equity, and leverage indicators like debt-to-equity and interest coverage. Multi-period trend analysis across three to five years, run this way, separates ordinary seasonality from a genuine structural shift in the business faster than a manual spread.
Narrative summarization and cross-checking. An LLM can read a full annual report and surface the qualitative signal buried in it, tone shifts around supply-chain risk, repeated hedging language, unusual emphasis on a specific segment, in minutes rather than hours. It can also run a basic earnings-quality check, comparing rising net income against falling operating cash flow to flag a mismatch worth a closer look, similar to the kind of consistency check a financial risk modeling process would apply manually.
The reliability problem that gets left out
This is the part most vendor content skips, and it is the reason financial statement analysis with large language models is not yet a plug-and-play replacement for analyst judgment.
Hallucination is not a rare edge case. Independent research and industry benchmarking have found hallucination rates for unassisted LLMs on financial tasks reaching as high as 50% in some testing conditions. In finance specifically, that means fabricated figures that look internally consistent and confident, a fabricated ratio that ties out neatly to a made-up baseline number, which is a worse failure mode than an obviously wrong answer because it does not look wrong.
Arithmetic is a known weak point, not a solved one. LLMs are next-token predictors trained on text, not calculators. They can simulate the format of a calculation without preserving the actual mathematical relationship behind it, which is why serious implementations increasingly separate the model’s reasoning from the arithmetic itself, having the model direct which numbers to pull and which formula to apply while a deterministic system does the actual computation.
Bias and opacity carry real consequences. Because these models are trained on historical data, biased patterns in that data can surface in outputs used for lending decisions or risk scoring. And unlike a transparent ratio calculation where every step is auditable, an LLM’s reasoning path is often difficult to reconstruct after the fact, which is a genuine problem when a regulator or auditor asks how a number was derived.
None of this means the technology is unusable. It means the honest framing is “powerful assistant that requires verification,” not “replacement for judgment.”
Building a workflow that survives contact with real filings
Firms getting durable value out of this tend to follow a similar pattern, regardless of which model or vendor sits underneath it.
- Normalize before you prompt. Harmonize currency, date formats, and segment labels before the data reaches the model. Clean, consistent inputs measurably reduce downstream errors, and several vendors report hallucination rates dropping close to zero once every figure is traceable back to a verified source rather than model-generated.
- Split reasoning from arithmetic. Let the model decide what to calculate and how to frame the finding in plain language, but route the actual computation through a deterministic system rather than trusting the model’s own math.
- Backtest against reality. Compare model output to actual realized earnings and to human forecasts on a rolling basis, and adjust prompts and pipelines based on where the gap shows up.
- Keep a human at the judgment layer. Automate the repetitive, high-volume stages of data collection and screening, and reserve interpretive calls, unusual accounting treatments, and anything feeding a regulatory or investment decision for a person who can be held accountable for the call. This is the same governance discipline that underpins good CFO advisory work more broadly: automation handles scale, a person owns the decision.
Applications at a glance
| Application area | What the LLM does | What still needs a human check |
| Data extraction | Pulls and standardizes line items from tables, footnotes, and multi-year filings | Verify against source for any figure feeding a downstream calculation |
| Ratio analysis | Calculates liquidity, profitability, and leverage ratios; flags trend shifts | Confirm arithmetic through a deterministic layer, not the model’s own math |
| Narrative summarization | Condenses MD&A and risk disclosures; flags tone and emphasis shifts | Confirm any cited fact or figure exists in the source document |
| Earnings prediction | Forecasts direction of future earnings from standardized statements | Best validated as one input among several, not a standalone signal |
| Cross-statement checks | Flags inconsistencies like rising income against falling operating cash flow | Investigate flagged items rather than acting on the flag alone |
The pattern across every row is the same: LLMs compress the time it takes to get to a first-pass answer. They do not remove the need for someone to confirm that answer before it drives a decision, whether that decision touches a financial model, a valuation, or a lending call.
Frequently Asked Questions
How common is hallucination in LLM-based financial analysis, and can it be reduced?
Testing on unassisted models has found hallucination rates reaching as high as 50% on certain financial tasks. The rate drops substantially when the model works from normalized, source-linked data rather than open-ended retrieval, and when arithmetic is handled by a deterministic system rather than the model itself.
What is a Sharpe ratio, and why does it matter here?
The Sharpe ratio measures return relative to risk taken. When a trading strategy built on LLM predictions produces a higher Sharpe ratio than a comparable strategy, it means the model is generating better risk-adjusted returns, not just bigger raw returns, which is the more meaningful test for an investment process.
Are LLMs going to replace financial analysts?
Not based on current evidence. They automate extraction, standardization, and a first-pass calculation layer well. They still need human oversight for interpretive judgment, unusual accounting situations, and anything with regulatory or client-facing consequences, largely because of the hallucination and arithmetic reliability issues above.
How should a small or mid-size company start using this responsibly?
Start with the lowest-risk use case: extraction and standardization of your own historical filings or statements, checked against source. Add ratio calculation next, with a deterministic check on the math. Hold off on letting a model make an unsupervised earnings call or feed a client-facing report until you have a validation step in place, and loop in a fractional CFO if you need that governance layer built out properly.
Conclusion
The research is real: under controlled conditions, large language models can out-predict human analysts at a specific, narrow task. What is equally real, and far less discussed, is that the same models hallucinate financial figures at meaningful rates and cannot be trusted with arithmetic on their own. Financial statement analysis with large language models is a genuine capability upgrade for teams that build in verification, not a shortcut around it.
If you are weighing how to bring this into your own reporting or forecasting process without inheriting the risk side of the equation, talk to our team about building a financial model or CFO advisory process that uses AI where it earns its place and keeps a person accountable everywhere else.
