A large language model can read a financial statement and produce a plausible analysis of it. Whether that analysis is usable in a controlled finance function depends on something the model does not provide: a way to prove each figure it states was derived, not generated.
That distinction is the whole problem, and it is why the research result and the production result diverge so sharply.
What the research actually established
The best-known study on this question is Financial Statement Analysis with Large Language Models by Alex Kim, Maximilian Muhn and Valeri Nikolaev (University of Chicago Booth School of Business, arXiv:2407.17866, July 2024). It is the paper that dominates this search result and it is worth reading before forming a view.
Its setup, in the authors’ own description: the researchers provide standardized and anonymous financial statements to a pre-trained LLM and design chain-of-thought prompts that resemble how a human analyst reasons, then test whether the model can determine the direction of future earnings.
Three properties of that setup matter more than the headline finding:
– Standardized. The statements were normalized to a common format before the model saw them.
– Anonymous. Company identity was removed, which controls for the model recalling the company.
– Directional. The task was predicting the direction of an earnings change, not producing a figure that someone signs.
Why the production result differs?
The gap between the study and a live finance function is not model capability. It is that every property that made the experiment clean is absent in production.
Your statements are not standardized. They come out of a chart of accounts with local conventions, accounts that mean something specific to your business, and mappings that changed in a prior year. A model reading them without that mapping will produce an answer shaped by the general case rather than yours.
Your statements are not anonymous, and that cuts both ways. Identity brings context the model may genuinely use, and also the risk that it reaches for a general recollection of a company or sector instead of the numbers in front of it.
Directional judgment is not what finance asks for. “Earnings will likely rise” is an analyst output. “Accrued liabilities are $4.2m and here is the composition” is a controller output, and it has to be right to the cent, traceable to source, and reproducible next month by someone else.
The close is continuous, not a single judgment. The experiment asked one question of one statement. A close asks thousands of small questions against data that is still moving, under a deadline, with an audit trail obligation attached to each one.
Put AI to Work on Financial Documents
Explore how LLMs can help interpret financial statements, extract key information, and support analysis while keeping validation and human review in the workflow.
The rule that makes this workable
Never let the model produce a number. Let it locate, classify and explain; compute deterministically.
This is the single design decision that separates an LLM finance deployment that survives audit from one that does not. In practice it means the model is allowed to:
– find the accounts that make up a balance and explain what each is
– classify a variance as a timing difference, a reclassification, an accrual or an error
– draft the narrative that accompanies a figure
– flag which of a thousand reconciling items deserves a human look first
and it is not allowed to:
– state a total, a ratio or a variance in its own output
– decide materiality
– close an item without a recorded human or rule-based decision
Every figure in the output comes from a query or a calculation the system ran, which the model annotates rather than produces. When the narrative says “gross margin fell 240 basis points”, that 240 came from arithmetic, and the system can show which rows it came from.
What the audit trail has to capture?
An AI-assisted finance output is auditable when a reviewer who was not present can reconstruct how it was produced. That requires six things recorded per output.
| Recorded | Why it is needed |
|---|---|
| Source data snapshot and as-of timestamp | The ledger moves; the analysis must name the version it read |
| The exact query or calculation behind every figure | This is what makes the number defensible, not the model |
| Model identifier and version | A version change is a change in behavior with no code change |
| The prompt and retrieved context | Without it, the output cannot be reproduced or explained |
| The classification the model proposed | Separates the model’s suggestion from the outcome |
| The human decision, decider and timestamp, where one was required | The control evidence |
Two of these are routinely missed. The as-of timestamp is missed because the system reads a live table and nobody records which version it read, so the analysis cannot be reproduced a week later. The model version is missed because the vendor endpoint updates underneath the deployment. Pin the version and record it, or accept that last quarter’s output cannot be explained.
Where this genuinely pays?
The value is not in replacing analysis. It is in triage and explanation at a volume humans cannot reach.
The reconciliation case is the clearest. A large intercompany or bank reconciliation generates far more exceptions than the team can investigate before the deadline, so items are worked in whatever order they appear. A model that reads each exception together with its surrounding transactions, proposes a classification, and ranks them by how likely they are to be a genuine error changes the order the team works in. The figures stay deterministic. What changes is which exception gets looked at first, and how much context the person has when they open it.
The second case is narrative. Variance commentary is written under time pressure by people who already know the answer, and it is where disclosure inconsistencies creep in. A drafted narrative constrained to computed figures, reviewed rather than authored, is faster and more consistent.
Assess LLMs for Your Financial Workflows
Identify suitable use cases, data requirements, accuracy checks, and safeguards before integrating LLMs into financial analysis processes.
What we would not do yet?
Three things, stated plainly because the honest boundary is the useful part of this post.
We would not let a model determine materiality. Materiality is a judgment with a standard behind it and a signature attached to it.
We would not deploy against an unmapped chart of accounts. The mapping work is unglamorous and it is most of the project. Skipping it produces a system that is confidently wrong in a way that takes a quarter to detect.
We would not run it without the recomputation layer. A deployment where the model states figures directly will pass a demo and fail an audit, and the failure surfaces at the worst possible moment.
Where this connects to the wider finance automation picture is in AI-powered account reconciliation, and for Oracle ledger environments specifically, Oracle AI consulting covers the integration constraints that shape the design.
FAQs
Can an LLM calculate financial ratios accurately?
It can produce arithmetic that is usually right, which is not the same as accurate, and "usually" is not a standard any finance function operates to. Compute ratios deterministically and use the model to explain them.
Does fine-tuning on our financial data solve the accuracy problem?
No. Fine-tuning changes style and format adherence more than arithmetic reliability, and it adds a data boundary question. The recomputation layer is the control, regardless of how the model was trained.
Is this different from the AI features already in our ERP?
Usually in scope rather than kind. Embedded features work inside the vendor's data model; a custom build is what you need when the workflow crosses systems the vendor does not see.
Make Financial Data More Actionable
Talk to an AI Expert
