Database migration is a process of transferring data between different versions or formats of database management systems and computer platforms. This procedure is usually performed programmatically to achieve smooth and efficient migration with minimal human resources.
Since the data is one of the core digital assets, database migration has been established as priority activities since we started using computers. Despite of that fact, procedure of migration often is neglected on IT departments plans and schedules, and it leads to low quality of data in the target system. Related studies proved that about 84% of all database migration projects failed due to poor management. Major reasons of that failures are listed below:
- Breakdown of the target systems
- Poor or low data quality in target environment
- Loss of opportunity for business
- Cost overruns
This paper explores necessary steps to successful database migration suitable for any scale projects.
Step 1: Investment in Profiling Source Data
The first key to success in database migration is valid source data. Understanding the characteristics of the source data is vital to identify undocumented features, quality and capacity of the data as well as anomalies. This step is called data profiling, it is designed to help you understand the strengths and weaknesses of data sets. When working with complex large data warehouses, it is worth investing in industrial strength data profiling tool and software.
Step 2: Model of Database Migration
Database migration is ranged from single step procedure with just one source and one target to complicated multi-steps data processing that deal with multiple source and multiple targets that can be partially merged during the procedure. Before walking into database migration, it is extremely important to design model showing every step of the process. It will serve as important roadmap of the data flow as well as agreement between the stakeholders. The model of migration process must be supplied with interim steps to validate the volume and the quality of data that is being transferred. Having all the checkpoints in place, data analysts can check that all parameters are not exceed the accepted limits and there are no hidden surprises.
Step 3: Define Roles & Responsibilities Upfront
Database migration may be a complex and long duration project which involves many IT specialists, so the formal agreement for all parties is necessary part of overall procedure. At project planning phase the related specialists must identify all required roles and assign responsibilities to those roles. Later the project manager must assign the defined roles to every team member.
Step 4: Decomposition is Everything
Like any other large and complicated project, database migration should be split into milestones based on functionality of each phase. It is recommended to start with the smallest data set, test the selected approach to database migration and then continue with larger data sets. Following this way, migration specialists will be able to get proof of concept and fine-tune the migration process quickly if it is necessary. Every migrated data set must be evaluated according to the specification prepared on the step 2 and refined for the next release when it is necessary.
Step 5: Run Performance Test
Most of database migration projects have a short timeframe for transferring to avoid business and/or operational interruption. To meet those strict requirements always conduct a full test of the code components for performance before passing it to production. The team members involved in database migration should continue improving the migration parameters until the desired quality of the target system is achieved. Recurrent performance tests will also allow to get full control over the database migration process.
