Data Platform Modernization
We move a legacy warehouse onto a cloud platform without breaking the reporting the business runs on. The work covers schema conversion, workload migration and parallel validation. The cutover plan is written so that it can be reversed.
The problem
We can't scale the warehouse any further and we cannot staff it. Finance closes on it every month, so we cannot simply switch it off.
Legacy platforms are usually replaced for commercial and staffing reasons before technical ones. A license renewal comes up, or the last engineer who understood the stored procedures leaves. The difficult part is downstream of the platform itself. Several hundred objects, a month-end close and a set of regulatory reports depend on behavior that was never written down.
Our approach
Most of the effort in a migration goes into reconciliation. Code conversion is reasonably predictable and tools do a good deal of it. The schedule is set by how long it takes to prove that the new platform returns the same answers as the old one, for every report in scope.
- 01
Inventory and dependency mapping
We catalog every table, view, procedure, job and report, then trace what depends on what. Usage telemetry from the source system separates the objects that are still read from the ones nothing has touched in years. Estates of this age carry a lot of the second kind. We retire those instead of converting them.
- 02
Target design
We set warehouse sizing, the database and schema layout, the role hierarchy and access model, and the storage and clustering decisions that determine what the platform costs to run. The design follows the workloads found in the inventory rather than a reference architecture.
- 03
Conversion
We convert schemas and code, automating where the patterns repeat and writing by hand where they do not. Legacy procedural logic is rebuilt as SQL and transformation models. A line-by-line transliteration carries the old platform's constraints into the new one, so we redesign the patterns that existed only to work around the old engine.
- 04
Parallel run and reconciliation
Both platforms run the same workloads over the same period. We reconcile row counts, aggregates and full column-level comparisons for the reports in scope, and we trace every difference to a documented cause. Differences are common. Some of them turn out to be defects in the legacy platform that had gone unnoticed.
- 05
Cutover and decommissioning
We write a dated runbook with a rollback path, a freeze window and a defined set of go or no-go checks. Decommissioning follows an agreed period with the old platform left idle. That period leaves time to find anything the checks missed.
How it fits together
What you get
- Object-level inventory with usage telemetry and a migrate, rebuild or retire decision recorded against every item
- Target platform design document covering warehouses, schemas, roles and access model
- Converted schemas, transformation code and orchestration, in your repository
- Reconciliation harness that compares source and target results, plus the signed-off reconciliation report
- Cutover runbook with rollback procedure and go or no-go criteria
- Handover documentation and working sessions with the team who will own the platform
Technology
Target platforms
- Snowflake
- Databricks
- Microsoft Fabric
- BigQuery
Typical sources
- Teradata
- Netezza
- Oracle Exadata
- SQL Server
- SAP BW
- Informatica
- Talend
Conversion and validation
- dbt
- SQLGlot
- Python
- custom reconciliation harnesses
Engagement
- Duration
- Twelve to thirty weeks, depending on estate size and the number of reports in reconciliation scope
- Team
- A lead architect, two to four data engineers, and a validation engineer dedicated to reconciliation
- Starts with
- Most engagements open with an assessment of two to three weeks. It produces the inventory and a costed migration plan. If you decide to run the migration with your own team from there, the plan is written to support that.