Proving a Modernized System Behaves Like the Old One: A Testing and Cutover Checklist

A cutover plan is the written sequence and decision rule for moving from a legacy system to its replacement: what must be proven first, who decides, what triggers a rollback, and when the old system is switched off. The evidence comes before the date.

Key takeaways

• A modernized system is accepted on evidence about its behavior, not on a demonstration, a vendor report or a green test run.
• Build the test plan from what the old system actually does, not from requirements. Legacy systems carry rules nobody wrote down, and requirements documents describe the system someone intended.
• Replay real historical inputs, and include the periods that stress the system: month end, year end and known peaks. A replay set drawn from a quiet quarter proves only the easy cases.
• Every check needs a stop condition written before testing starts. A check that cannot fail is not a check.
• One of the seven checks is not engineering’s signature to give. Book that person’s time on day one, because it is the one that gets assumed.

What does a modernization test plan have to prove?

That the new system matches the old one on results, failures, timing and data.

Functional tests written from requirements will miss the undocumented rules a legacy system has accumulated over decades: the condition added in a 1997 patch, the rounding rule finance agreed verbally, the exception path that exists because one customer is invoiced differently. None of that is in the specification, and all of it is in production.

So the plan has to be built from observed behavior. Each check below produces a specific piece of evidence and names the condition that stops the cutover. The general framework this sits inside is covered in the legacy application modernization roadmap.

What are the seven checks?

Seven, in this order, and three of them are the ones that get cut when the schedule tightens.

The three marked yes are cut for the same reason: they cost calendar time rather than effort, so they look compressible on a plan and are not. Budget them separately from the build, at the start, when they are a design choice.

How do you build the replay set?

From the periods that stress the system, not the periods that are easy to extract.

This is the check most often done badly, because a clean recent quarter is the simplest data to get hold of and it proves almost nothing. A replay set earns its place when it contains:

• A full business cycle. Whatever periodicity the system has, monthly, quarterly or annual, the replay has to cross it at least once.
• Year end. Different rules, different volumes, different reporting.
• A correction batch. The month somebody reversed and reposted a large volume of transactions.
• A structural event. An entity acquired mid-period, a chart of accounts change, a tax rate change.
• The leap day, and any date the system treats specially.
• A known incident. If the legacy system once failed in a particular way, the replacement should fail the same way or demonstrably better.

Identifying which historical periods actually contain the oddities is a sampling problem, and it is one of the few places in this checklist where AI changes the work rather than the pace of it. A model profiling years of transaction history will surface the correction batches, the structural events and the volume anomalies far faster than a person scanning month ends, and it will find periods nobody remembered. A person still decides which of them belong in the replay set.

Two constraints shape this before any engineering starts. You need a data handling decision, because equivalence testing needs real records and that is a privacy and governance question, not a technical one. And you need retention long enough to reach back to those periods, which is not a given.

Where does AI help here, and where does it not?

It compresses the analysis and leaves the evidence untouched.

Where it earns its place. Building the comparison harness. Reading the legacy source to propose candidate edge cases and boundary values that a human test designer would have to find by hand. Profiling historical data to identify which periods actually contain the oddities, which is a genuinely hard sampling problem. And triaging the output: a field-level comparison across a real estate produces thousands of differences, and sorting them into likely harmless and likely real so a person reviews the second group is a large, measurable saving.

Where it does nothing. It does not shorten the replay, because both systems still have to run. It does not shorten the parallel run, because that is calendar time by definition. It cannot set a tolerance, because a tolerance is a statement about what the business will accept. It cannot judge materiality, because a one-cent rounding difference on an interest calculation is either a defect or an agreed change and nothing in the code says which. And it cannot sign the reconciliation.

The same split governs AI-generated code more broadly, and GrowExx sets out its position in AI code review versus a human expert and the true cost of shipping unreviewed AI-generated code.

Who decides to cut over, and against what rule?

One named person decides, using a rule written before testing started, with the rollback already rehearsed.

The rule itself is three sentences and it has to exist on paper before the first test runs:

• Which checks must pass outright. Name them. Not all seven carry equal weight on every system.
• What level of unexplained difference is acceptable, if any. For a general ledger the honest answer is usually none. For a reporting system it may not be. Decide which you are.
• What triggers a rollback, expressed as an observable condition rather than a judgment call at 3am.

How do you rehearse the cutover and the rollback?

Run the whole sequence in a production-like environment, time it, and run the rollback as well.

A rollback that has never executed is an assumption with a plan attached. Rehearsing it answers three questions you otherwise discover during an incident: how long it takes, what data would be lost or need reprocessing, and at what point it stops being possible.

Agree the point of no return in advance and write it into the rule. Keep the legacy system available, read only, for a stated period after cutover, and set the date it is switched off. A migration with no date for switching the legacy path off becomes two systems permanently, which is the most expensive outcome available and the one nobody decides to choose.

What goes wrong most often?

Tests are built from requirements instead of behavior, and nobody owns the decision.

• Testing only the periods that were easy to collect.
• Comparing totals and missing offsetting field level differences underneath them.
• Ignoring error paths, so failures behave differently in production than anyone expected.
• Skipping the rollback rehearsal, so the first rollback happens during an incident.
• Treating a vendor test report as evidence about your own data.
• Discovering in the final week that the reconciliation needs a business owner who was never asked.
• Accepting an AI triage of field differences without sampling the harmless group. The triage is a prioritization, not a verdict. If nobody ever reads a sample of what the model set aside, the saving has been taken out of the evidence rather than out of the analysis.

The decision

Build the test plan from the old system’s real behavior rather than its documentation. Run all seven checks, write the stop condition for each before testing starts, and budget the three that cost calendar time separately from the build.

Then rehearse the cutover and the rollback, write the decision rule in advance, and name the one person who will use it. Book the business owner’s reconciliation time on day one, because it is the only signature on the list that engineering cannot provide and the one most often discovered to be missing in the final week.

GrowExx provides this verification as part of application modernization services, with an independent AI code audit of code produced with AI assistance.

FAQs