Key takeaways
- AI output is unverified until a named person signs against a checklist. AI assistance changes speed, not accountability.
- Use three risk tiers (low, medium, high) so verification effort matches the consequence of an error.
- Verify code with evidence: independently defined tests, static analysis, dependency and license checks, secret scanning, and review of what a change removes.
- For migrations, prove behavior is preserved by running old and new systems on the same real data and investigating every difference.
- Keep a short evidence pack for medium and high tiers, and ask vendors the same five questions when you buy AI-assisted work.
In late October, Western Australia’s government was reported to be asking Deloitte for a fast refund on a A$166,000 report about merging three universities. Critics found references to sources that do not exist and arithmetic errors. Deloitte said it had used generative AI to help with research, summarizing and proofreading (TNW, citing Guardian Australia). The government said it was told the errors do not change the report’s modeling or findings, and Deloitte is to correct and reissue it. These details come from press reporting, and we have not independently checked them.
Whatever the final outcome, the case shows how an unverified AI-assisted deliverable turns into a commercial dispute. Software teams face the same exposure with generated code, migrations and documentation. This article sets out a verification standard that buyers can ask for and delivery teams can adopt.
How to verify AI-generated work: AI output is unverified until a named person signs against a checklist
AI assistance changes how fast a draft appears. It does not change who is accountable for the result. A usable standard has three parts: a risk tier that decides how much checking an item needs, a named reviewer who is not the person who prompted the tool, and a record of what was checked.
Step 1: Tier AI-assisted work by risk and consequence
| Tier | Examples | Minimum verification |
|---|---|---|
| Low | Internal scripts, draft documentation, test data | Author review; automated checks where they exist |
| Medium | Production features, integrations, reports used for internal decisions | Independent reviewer; automated tests and scans; source check for any claim or reference |
| High | Payment, identity or safety-relevant code; regulatory filings; client-facing analysis; legacy migrations of core systems | Independent reviewer plus a second sign-off; behavior-equivalence testing; documented evidence pack |
Step 2: Verify AI-generated code with tests and evidence
Generated code often looks correct. These checks give a reviewer something firmer than a read-through:
- Tests that the AI did not write alone. Generated tests can mirror generated code, including its mistakes. Have a human define the cases that matter and review coverage of edge and failure paths.
- Static analysis and security scanning on every change, with the same gates as human-written code.
- Dependency and license checks. Confirm that every package the tool introduced exists, is maintained and fits your license policy.
- Secret and configuration scanning before merge.
- Review of what the change does not do. Look for removed validation, loosened permissions and silently swallowed errors.
For a closer look at what to trust and what to review in AI-refactored code, see our guide to AI code refactoring for legacy applications. If you want an independent review of AI-generated code, see our AI code audit and validation service.
Step 3: Prove behavior is preserved in AI-assisted modernization
Legacy migration is where unverified output costs the most, because the original system’s rules are often undocumented. For AI-assisted refactoring or translation of legacy code, the central question is whether the new system behaves like the old one where it must, and differently only where you intended. Our testing and cutover checklist, Proving a Modernized System Behaves Like the Old One, covers the cutover side in detail. For system-specific guidance, see our posts on mainframe modernization and database modernization.
-
- Capture current behavior with characterization tests built from real inputs and outputs before changing anything.
- Run old and new side by side on the same data and compare results, investigating every difference.
- Migrate in slices that can be rolled back, and keep a record of the decisions people made about differences.
- Treat anything AI summarized about legacy logic as a hypothesis until a person who knows the system confirms it.
Step 4: Verify AI-generated documents and analysis
For reports, proposals and research, the equivalent of a test suite is a source check. Every reference should be opened and matched to the claim it supports. Every figure should be recalculated from its inputs. Quotes should be traced to the original. A reviewer should record which of these were done.
Step 5: Keep a verification evidence pack for every deliverable
For medium and high tiers, keep a short record per deliverable: what AI tools were used and for what, who reviewed it, which checks ran and their results, and what was changed after review. This is what lets a team answer a client or auditor with facts.
Make AI-Assisted Software Delivery More Accountable
Establish clear ownership, review checkpoints, evidence requirements, and human approval processes across AI-supported engineering workflows.
What to ask a vendor about AI-assisted software delivery
When you commission AI-assisted work, these questions are reasonable to include in a statement of work or review meeting:
- Which parts of the deliverable used AI assistance, and for what tasks?
- Who independently reviewed it, and against what checklist?
- What automated tests, scans and source checks were run, and can we see results?
- For migrations, how was behavior equivalence demonstrated?
- How are errors found after delivery corrected and communicated?
Verify AI-Generated Code Before Production
Combine automated testing, code review, security checks, and engineering validation to identify defects and prevent unreliable changes from entering enterprise systems.
Conclusion
AI assistance can shorten delivery, but speed only counts when the output has been checked. A workable standard is simple: tier work by consequence, require tests and evidence for code, prove behavior is preserved in modernization, verify documents against sources, and keep an evidence pack a client can inspect. Buyers should ask for that standard in writing, and delivery teams should adopt it before a client asks. For a review of your own AI-generated code, see our AI code audit, or learn how we run AI-driven development with verification built in.
Verifying AI-assisted work: frequently asked questions
Does this slow teams down?
Verification adds work, but tiering keeps it proportionate. Low-risk items move quickly, and effort concentrates where a mistake is expensive.
Should AI-assisted work be disclosed to clients?
That is a contract and policy decision. A clear statement of how AI was used and verified generally builds trust more than silence.
Is AI-assisted delivery unreliable?
The case above is about missing verification, not proof that AI assistance cannot be reliable. With the right checks, it can shorten delivery and still meet the same bar.
Build Confidence in AI-Augmented Engineering Teams
Book an Engineering Assessment