Data migration is a process that involves cleaning and transforming data, extracting that data from the current system, and then moving the data to a new solution. At the simplest level, this is a relatively straight-forward process.
The problem is, it’s rarely that simple.
Organizations accumulate massive quantities of data over time, which makes organizing and mapping that data more complicated. In fact, data mapping is perhaps one of the most challenging aspects of the migration project, making it one of the riskiest. In addition to data mapping, running a migration project has the potential to affect “business-as-usual”, which can slow down customer interactions if you’re not careful. Risk mitigation is therefore an important piece to the migration puzzle. By coming into the migration process prepared, you lessen the likelihood that your migration could interrupt business-critical operations and the customer experience. Here are seven ways to build risk mitigation into your migration project.
1. Set a Comprehensive Project Scope
Understanding where you’re migrating from and the type of data moving to the new location is only the base layer of project scope. For risk mitigation, it’s crucial to also define how you’ll filter that data, how the project will be managed and timed out, and whether any unique specifications must be considered.
Should data filtering assume you’re moving all historical data, or data solely from active users? Should you be migrating only the data that’s been produced within a specified time frame? Who will be responsible for monitoring and reporting on the migration? Who will oversee keeping the project on track? Could any business processes affect this schedule?
What if you operate in multiple geographic locations? Should the migration only run at certain times of the day to prevent disruption to the customer experience? Plan for every scenario. Anticipate every obstacle.
2. Start at the Source
Understanding the strengths and weaknesses of your data source is important for mitigating risk during your migration project. How has this source system affected your data integrity? Are there any issues with the system scalability that could slow down the project?
A migration puts the source system under pressure. If the system has trouble retrieving items, it loses data, or has network latency, it could affect the usability of the system for your migration. Define who owns the source resources and who within your organization has enough experience with it to help if trouble arises. Then, outline a backup plan in case you need to escalate a roadblock even further. Where will you turn if your “expert” cannot provide the answer? If you’re working with a migration partner, you have that resource baked in. If not, you’ll have to look for help directly from the source’s support resources.
3. Understand Your Target
Just as your source must be assessed for its scalability and configuration, it’s important to do the same with the target. Risk mitigation in migration is an act of thoroughness and planning. Before you begin the move to the target source, you should be able to answer questions like:
- Does this target have enough storage for the quantity of data I’ll be moving? Will it be able to scale appropriately to accommodate an expanded use of the system?
- How will we store data in this new system to meet regulations like GDPR? How can we use geographic distribution of our data to maximize the benefits of our migration in this respect?
- Do any other resources currently depend on the target system? Do we have a process in place to resolve issues associated to those resources if necessary?
- Do we have internal experts or outside consultants available in case we uncover an issue that requires escalation?
- Do we know of any issues today that could impact our end-users? If so, what’s our plan to communicate these issues?
4. Pinpoint Environment Challenges (On-Prem Only)
If your software is on-premise, you must also ask what other environments should be considered for risk mitigation? The source system is made up of core resources like RAM, CPU, and Disk, as well as its software and operating system. But, what about source dependencies? Do source SQL or Exchange servers come into play? Are they up to par with the rest of the ecosystem?
These same principles of core and dependent resources apply to your target. It’s crucial to understand how many dependencies you’re working with, where they’re located, and whether you have the power to control them during the migration. Some solutions are managed by the provider. Understanding which dependencies are within your control vs. which are not will help you plan your project and mitigate the risk.
With more and more solutions Cloud-hosted, your visibility into these types of limitations is minimized. If it’s a concern for your organization, ask questions of your Cloud hosting company about things like bandwidth, load testing, etc.
5. Define Constraints and Plan the Solutions
Once you’ve combed over your source and target with a fine-toothed comb, it’s time to look at the bigger picture of your project constraints. Knowing what you know now, which constraints might you be able to work around? How will the ones you can’t work around affect your timelines and budget?
For example, if issues with latency or lost data are found in your source system, how can you expect those to impact your migration? Do you have enough hardware, software, servers, computers, etc. to get the job done in the timeline you proposed in your project scope? If not, what’s the plan to remedy that? Has the team agreed upon a process to deal with case escalation and special considerations if they arise?
It’s also important to plan solutions that avoid interruptions in the user experience. For more complex migration projects, sandbox environments should be used to eliminate interference with users and prevent any negative repercussions that could occur from a failed test or data load.
Lay out the major project constraints and their ideal solutions. Who needs to be aware of this information? Your stakeholders should of course be informed, but issues affecting your end-users must be communicated, too. Understanding the constraints and whether they’re “fixable” or not provides backstory for any new budget considerations or altered timelines that may result. Now you have something concrete to back up your plan and mitigate the risk on your migration project.
6. Holistic Monitoring
Many of our customers hire us or our migration partners to help monitor their migration project. The reason this is so common is because the hours it takes to regularly pull metrics, update the team on status and performance, and monitor the project can easily become a full-time job. If you can’t dedicate an internal staff member to stay on top of the project, you must find someone who can.
Holistic monitoring is foundational to the project’s risk mitigation. It requires a clearly defined process for tracking the migration, with prescribed cadences for when those metrics should be analyzed and who on the team should be updated.
The person monitoring the migration must know what to watch for and understand the potential constraints so they can keep a closer eye on those aspects of the project. The fully scoped plan and the defined constraints should help avoid anomalous behavior, but if something does happen, does that person know how to identify it?
7. Follow Best Practices for Data Mapping
As mentioned in the opening paragraphs, data mapping is one of the most challenging aspects of data migration. Therefore, following best practices for data mapping is a big part of risk mitigation. Here are a few pointers:
- Document EVERYTHING. GDPR requires a record of data processing activities and full documentation of why the data was processed, where the data was transferred, etc.
- Use automated tools to classify data, enforce data governance, update the data map when new information is added/removed/altered, and handle existing data changes.
- If you are using manual processes as well as automated processes, make sure you compare the results of each to resolve discrepancies.
- Establish a standardized process for developing your maps with clear rules for how the map should be used. This will reduce redundant work.
- Follow a testing and approval process for validating the functionality of the data map. Use sample data sets first to make sure the data mapping is working the way you envision.
- Keep systems updated and perform system maintenance on a regular basis, so when the time comes to modify a map you can be confident everything will flow together.
- Use sandbox environments to test data maps in a safe space without affecting your users or customer experience.