Making Migration Work

By David Davies

“Migration is the set of activities, processes and tools that enable the transfer of data and other digital assets from one collection of systems (Source) to another (Target).”

That may sound simple, until you start to consider that each element of that sentence adds more and more complexity and difficulty in moving from one thing to another.

Moreover, a migration can have a variety of different parts, such as Customer Migration, Network Migration, Product or Service Migration, System Migration, Platform Migration or Data Migration (this is a far from exhaustive list) and each have different approaches with different strengths and weaknesses.

Migration is Part of the Transformation

The very earliest decisions on a transformation program, or any program that will require a migration, will have profound implications for the entire project. Decisions around technology may result in incompatibility, for example, a new Billing System doesn’t support all of the product mix of the legacy system, and when it comes to data, the architecture must be examined carefully to determine if any translation must be made.  

Too often, we find ourselves engaged at the latter stages of a program without thinking of those things, and the migration workstream must fit to what’s been built, despite all the costs and limitations that this may introduce. It’s much better to keep the migration at the forefront of the project, helping architect the approach, and head off possible issues such as:

  • Incompatible structures such as product taxonomy or customer persona.
  • Legal or regulatory implications of the migration; can we ‘fix’ data we deem incorrect?
  • Is there a requirement to recontract if moving legal entities and how that is to be managed.
  • What customer engagement is required and if there is a marketing plan to minimize churn.
  • Are the data structures built to allow easy transition or do they need complex transformation?
  • How is historic data handled, archived, and/or anonymized?

Planning ahead and including a migration workstream in the initial stages of the program can markedly help with these issues, as well as ensuring that the long tail items such as data quality are started early.

Assuming you’ve built your program for success (or even if you haven’t) there are then many aspects of the migration that need to be planned, agreed, built, and executed to get to the required end-state. Among those are:

  • Agreed Outcome – Regardless of the type of migration being conducted, it is essential to understand the program’s ambitions; should everything remain identical, or can new features be used? Should we keep existing products or move to new (whether visible to the customer or not)? While some of these decisions will be ratified later, the ideal outcome must be agreed upon as early as possible to be tested and challenged to get the best results.
  • Data Discovery – Capture and document the current data structures and understand the alignment with the future state. Look for complex and incompatible relationships and unsupported data types that must be translated as part of the migration. Especially, look for free text fields that (may) contain vital data and explore consistency and how they might be handled (AI/ML is a subject for a later Blog). Build data dictionaries and ERDs as the data is explored.
  • Knowledge Retention – You’re likely replacing a platform, and there are operations teams and engineers who have a long history maintaining it. Keeping the knowledge to allow translation, and operation until it is decommissioned is vital. People may retire and good people will leave without a suitable incentive plan in place.
  • Data Quality Remediation (DQR) – We’ve yet to find a legacy platform with great, or even good data quality. Over the years, even the most controlled platforms accumulate issues, so it’s essential for important data be identified and corrected before moving over to the new platform. This can then be corrected in situ or in flight as part of the migration, especially for repeatable and consistent errors such as malformed emails that can be predicted and resolved. A warning though, be aware of the legal framework when ‘correcting’ data in case there is customer approval needed even for obvious errors. One specific aspect to be mindful of is that data is linked and cascades to other systems (either integrally or in a derived form), so data correction must follow a carefully designed process to ensure consistency throughout the ecosystem.
  • Extract/Transform/Load – You’ll usually use a tool to move data of any kind of volume, with its job being to extract, translate, correct, and load data from the old to the new systems. Our ETL tools have been of great benefit for our clients, and can adapt quickly to the prevailing technology estate.
  • Integration, Testing, and Trials – Once you’ve built the processes and systems to enable the migration, a test or stub environment is crucial. While friendly or test customers can be used to confirm and verify that things are working as expected, they cannot substitute for testing to find the inevitable, and unpredictable problems in any migration.
  • Execution – Big bang, phased, by cohort or by system, the actual execution can be performed in many ways. The key is maintaining the momentum if there are repeated anti-social hours for migration and understanding the migration and ops teams’ commitment (or lack) over time is vital for the first and last migrations to maintain the same quality.
  • Reporting – During the migration a reporting cycle to the wider program is maintained to track progress and highlight improvements that can be made. Regular engagement helps demonstrate the momentum of this stage of the program and alerts to reoccurring issues.
  • Mop Up – Any strategy must recognize that 100% migration is highly unlikely, and an approach for the leftovers that cannot be migrated need to be agreed upon. Approaches might be unilaterally closing the service, allowing legacy to remain until it churns out, manual cross or up sell, and so on, but however this is approached, it needs to be sympathetic to and informed by the previously agreed outcomes.
  • Decommissioning – With the migration complete and outcomes met, it is unusual not to have decommissioning of the legacy platform as part of the plan to save operational and licensing costs and simplify an estate. Whether this is owned by the delivery program or the migration workstream can be debated, but ownership must be clear and the migration workstream should be well positioned to assist using the acquired knowledge of structure and operation. Depending on how the migration is executed, decommissioning can also be achieved in various ways: retiring the legacy platform at the end of the migration, in phases, wherein, obsolete platforms can be decommissioned earlier; or dual-run the new and old. It is therefore crucial when defining a migration strategy to include decommissioning at the heart of that thought processes.

A major challenge for migrating in a cost-effective way is the myriad of skills that are required and the risk that each is provided by an individual rather than using multi-skilled teams. A typical migration requires management, data architecture, customer engagement, requirements capture, systems and process architecture, and technical development at a minimum.

Finally, make sure to use the right approach and the correct tool for the job. Migrations might be performed with large teams costing hundreds of thousands of dollars when the data could have been manually moved over the course of a few months for a fraction of that if time was not a major factor. Whilst seemingly minimizing the risk of a manual migration, you should always explore whether the complexity of the migration matches the need to keep things cost effective and deliver the agreed outcomes.

How We Help

At Cartesian, we have decades of experience successfully delivering all aspects of migrations. Our multi-disciplinary teams and flexibility mean that we can quickly accomplish your required outcomes in a cost- and time-efficient manner. Contact us to help the start journey and use our expertise to help make sure you start out, and end, in the right way.

Examples of Cartesian’s Work

  • Cartesian led the customer migration and decommissioning of switches for a Tier 1 operator, enabling higher customer penetration and revenue.
  • Cartesian used their expertise to enable a global telecoms provider to migrate users from an operationally critical legacy email service to an optimized new service for verticals with no tolerance to downtime.
  • Cartesian worked with a Tier 1 operator to migrate critical business data from existing legacy source systems to the new workflow systems, migrating 4500+ network build projects from legacy systems to the new workflow system as part of automatic migration.
  • Cartesian managed the migration readiness of data, process, and customer migration for a CRM platform migration for a major European triple play operator. This included high levels of data quality management for the more than four million customers under a complex regulatory framework.