Section 5: Designing Data Migration Solutions. Lesson 24.
A migration strategy decides how data will move from the old system (source) to the new one (target). Before anyone writes scripts or runs jobs, the team must decide whether everything moves at once, in stages, or through a hybrid method. The choice directly affects downtime, cost, risk, testing, resource planning and the user experience.
AWS frames migration planning as one part of a wider lifecycle: prepare, plan, migrate and optimise. Strategy design happens in the planning stage, informed by everything from Sections 2 to 4: requirements, profiling, volume, complexity and platform compatibility.
Big bang migration
The entire system or dataset moves in one major operation, usually within a planned maintenance window:
- Stop, or freeze, the old system.
- Extract, transform and load all data into the new system.
- Validate.
- Switch all users over to the new system.
It is usually the fastest route to completion, but carries more risk because everything moves at the same time. If something fails, the whole cutover fails.
Best when:
- The system is relatively small
- Data volume is low
- Business downtime is acceptable
Example: a small company moves its customer records from an Excel spreadsheet or an old MySQL database into a new CRM over a single weekend. Nothing important breaks while the system is offline, and on Monday everyone is on the new CRM.
Phased (staged) migration
Data or functionality moves in controlled, separate stages: a sequence of smaller, manageable migrations. A team might migrate one department, region, application or module at a time, or start with a specific group of tables. Microsoft lists phased or incremental cutover as one of the most common approaches in cloud migration planning.
Best when:
- The system is large, complex or business-critical
- Data volume is high
- Downtime is not acceptable
Example, a banking system:
- Customer profile data
- Account data
- Transaction history
- Reporting systems
Each phase gives the team time to test, fix unforeseen problems, train users and improve the process before the next one. The order follows dependencies: accounts need customers, and transactions need accounts (lesson 29). Google Cloud stresses the same prerequisites: catalogue the workloads, map dependencies, then plan the order.
Speed versus risk
The core trade-off is time against risk. In the lecture's illustration, the same small CRM might take about two days as a big bang, but seven or eight days phased. The phased version takes longer because every stage has its own load, test and sign-off.
| Factor | Big bang | Phased |
|---|---|---|
| Duration | Short, one window | Longer, many stages |
| Downtime | Planned outage | Minimal per stage |
| Risk | Concentrated in one event | Spread across stages |
| Rollback | All or nothing | Per stage |
| Running two systems | No | Yes, during the transition |
| Best for | Small, simple, downtime-tolerant | Large, complex, always-on |
Hybrid approaches
Real projects often mix the two:
- Trickle migration with change data capture (CDC): bulk-load the history, then continuously sync changes from old to new until a short final cutover. This gets close to zero downtime.
- Parallel run: old and new run side by side, and outputs such as reports and balances are compared before the old system is switched off. Common in finance.
- Pilot then big bang: migrate one small group first as a rehearsal, then everything else in one window.
- Big bang per module: each module is moved all at once, but modules go in phases.
Choosing a strategy
| Question | If yes, lean towards | Because |
|---|---|---|
| Can the business stop for a weekend? | Big bang | Simplest and fastest |
| Is it 24/7 or customer-facing? | Phased or trickle with CDC | Keeps downtime near zero |
| Is it a large volume or many dependencies? | Phased | Smaller, testable pieces |
| Is the data financial or regulated? | Parallel run | Proves results match before switch-off |
| Is the team new to the target? | Pilot first | Learn on low-risk data |
From old to new: a worked mapping
Whatever the strategy, each table still has to be transformed from the old structure to the new one. The lecture's customer table:
| Old column | New column | Transformation |
|---|---|---|
| customer_id | id | Rename |
| full_name | first_name + last_name | Split |
| email_address | Rename | |
| phone | phone_number | Rename (standardise format) |
| city | city_name | Rename |
INSERT INTO new_db.customers (id, first_name, last_name, email_address, phone_number, city_name)
SELECT customer_id,
split_part(full_name, ' ', 1),
nullif(substring(full_name from position(' ' in full_name) + 1), ''),
lower(trim(email)),
phone,
city
FROM old_db.customers;Most fields map directly with a new name; only full_name needs a real transformation. That is the core idea of data transformation between two structurally different schemas, covered in depth in lessons 25 and 26.
What a strategy document contains
- The approach (big bang, phased or hybrid) and why
- Scope and waves: what moves in each phase, in dependency order
- The downtime window, or how downtime is avoided
- Cutover plan and go/no-go criteria
- Rollback plan for each phase
- Validation approach and success criteria (lesson 10)
- Communication, training and support plan
What comes next
With the strategy chosen, the rest of Section 5 covers the technical execution: ETL concepts, data mapping techniques, pipeline design, tool selection and handling dependencies. Together these are the core of designing a migration solution, and what makes a move from legacy software to modern cloud infrastructure smooth, secure and efficient.
Key takeaways
- Strategy decides how data moves: all at once, in stages, or a hybrid.
- Big bang is fast but high-risk; use it for small, low-volume, downtime-tolerant systems.
- Phased is slower but safer; use it for large, complex, always-on systems, ordered by dependencies.
- Hybrids such as trickle with CDC, parallel runs and pilots combine the strengths of both.
- Every strategy still needs table-level mapping and transformation.
Next: 25. ETL (Extract, Transform, Load) Concepts.