Ask anyone who has run a system replacement what went wrong and you will hear about the data. Not the features, not the interface — the fifteen years of records that had to move, and the fortnight everyone lost to them.
It is the most predictable overrun in software, which makes it the most avoidable.
Why it is always worse than expected
The old data does not match its own rules
The legacy system has a required field that is empty in 4,000 rows, because a validation was added in 2019 and nothing backfilled. It has dates in three formats, because the input changed twice. It has duplicate customers with slightly different spellings.
None of this is visible until something tries to import it strictly.
Nobody knows what the fields mean
There is a column called status2 containing single letters. The person who knew what "R" meant left in 2021. It is used by a report the finance team depends on.
The mapping is not one-to-one
The old system had one address per customer; the new one has billing and shipping. Which does the existing address become? That is a business decision, and it needs someone with authority to make it — multiplied by every ambiguous field.
Volume changes everything
A migration that runs in four minutes on a test extract takes eleven hours on the full set. If your cutover window is one weekend, that is a scheduling problem you want to find in a rehearsal.
How to run one that works
1. Get a real extract in week one
Not a sample. Not a description. The actual data, or a full copy of it. Every week you wait is a week of decisions made on assumptions. This is the single highest-value thing you can do, and it costs nothing.
2. Profile before you plan
Count nulls in required fields, distinct values in coded columns, records failing the new rules, likely duplicates, and outlying dates. A day of profiling reshapes the estimate — usually upward, which is uncomfortable and much better than discovering it later.
3. Decide what does not come
Not everything should. Records for customers inactive for a decade, attachments nobody has opened, superseded revisions. Archive them somewhere queryable and migrate less. This is the cheapest way to shrink the job, and it is usually available.
4. Make it a repeatable script, never a manual process
You will run it many times. It must be automated, versioned, and re-runnable from scratch, because you will find a mapping error at rehearsal three and need to start again cleanly.
5. Rehearse on production data, at least twice
Full volume, timed. The first rehearsal finds the data problems. The second confirms the fixes and gives you a reliable duration for the cutover plan. A migration that has never been rehearsed at full scale is a plan, not a procedure.
6. Reconcile with numbers, not impressions
Record counts by type, financial totals, and spot checks on records the business cares about. Agree these before cutover and have the business sign them off after. "It looks right" is not acceptance.
7. Keep the old system readable
Do not decommission on day one. Leave it available read-only for a few months. The first time someone asks "what did this look like before," you will be glad it is there.
Budget it as its own project
For a legacy replacement, migration is frequently 20–40% of total effort. If a proposal has it as a line item at 5%, either the data is unusually clean or nobody has looked at it — and the second is far more likely.
Ask any vendor how they will profile the data and how many rehearsals are included. The answer tells you whether they have done this before. More questions in that vein are in our guide to evaluating a development firm, and the wider cost picture is in what custom software costs.
We scope migration as its own phase within application development engagements, for exactly this reason.
Facing a migration? Send us a description of the source system and we will tell you what to look for first.