Keeping your Data Migration on Time and Within Budget

The Starfish team has seen a lot of successful data migrations, but we’ve also seen how easy it is to get off track and over budget. In this blog post, we’ll address the most common reasons for reworks and budget overages, and the steps you can take to prevent them from happening you.

1. Access credentials weren’t tested and don’t work.

To get started with your migration, your partner will need access credentials for your system(s). A kick-off meeting can easily become a waste of time for both you and your partner if they go to log-in and the credentials don’t work. Testing prior to the first meeting helps avoid starting your migration with an unnecessary slowdown. The best way to test credentials is to try logging in from outside of your network.

2. Field maps are incorrect.

Clearly defined field maps also are a crucial aspect of your preliminary migration work.  Field maps help your migration partner understand where your data needs to go and how it needs to be labeled. To establish the correct field map and ensure there is no missing information or wrong destination fields, you’ll want to double check field names for both source and destination systems for each field mapping, especially for custom fields that are not in the base maps.

3. There are repurposed source fields that weren’t mentioned.

If repurposed fields weren’t mentioned beforehand, they will often be discovered after the first test migration. A repurposed field is one that is housing information that really is not meant for the field. For example, you may find information in the Department field that has nothing to do with the department and should be moved elsewhere. When a client expects his data in a different destination field or different field format but does not mention a repurposed field ahead of time, the migration team and client must take the time to resolve it after the fact. To avoid this issue, be sure to point out all repurposed fields to your migration team prior to the start of the migration. Also, as a general best practice, try to avoid repurposing fields going forward.

4. Field truncation is occurring because the source field is longer than destination field.

Field truncation can occur  when the source field length does not match the destination field length. For example, your source field may be 250 characters long, but the destination is only 200 characters long, so some of that data will get lost. The migration team will have to build map reroutes to fix this, and the time it takes will end up delaying the project. Review field properties before the test load is run to double check for field truncation issues.

5. Business rules in the destination system aren’t accounted for.

Sometimes, there are conflicts with destination system business rules. Business rules in a destination system may enforce certain conditions or mappings that teams are not aware of. One example of these types of rules would be a field validation rule where one field has a dependency on the value of another. In this scenario, the system may ask for A but also require that you provide A-2. If A-2 doesn’t exist in your source system, the business rule can’t function correctly, so the destination results will be incorrect. These incorrect results cause reworks, which slow down the migration and inevitably lead to budget overages. Business rules in the destination system must be shared with the team building the destination system to speed test validation and eliminate reworks for everyone.

6. Field type properties don’t match on mapped fields.

Earlier in this post we talked about field truncation and how it affects the data. Mismatched field types also can cause unexpected errors. Field types of mapped data must be reviewed to ensure similarity and correctness. Reviewing them will also help migration teams identify and resolve differences. For example, if you map numeric fields from the source system to text fields in the destination system, you lose the ability to use that data in calculations. Similarly, text mapping from the source system to formatted email or URL fields in the destination may be rejected by the destination system due to improper email formatting. Phone numbers can be particularly tricky in these situations because they are formatted differently from one CRM to another. Extensions may or may not be included in the actual phone field, and user entered data is error-prone when an entry style hasn’t been enforced. Avoid the double work and budget strain caused by this issue by instituting data transformations for these field types.

7. Data hasn’t been properly cleaned so the results are unexpected.

Garbage data can respond unexpectedly to the migration map, which will create questionable results. When testers find bad data, they must spend time understanding why they have those results or ask your migration team to research it, thus incurring additional costs and throwing off the project timeline. The team cannot move forward until they realize what is causing the unexpected results: Is it an error in mapping? Is there missing data in the filter criteria? Is it simply bad data? Pre-migration data analysis and cleanup sidesteps this issue to avoid wasted efforts. To do this, pull listings of the following fields and look for data inconsistencies or invalid data values:

• Phone numbers
• Email addresses
• Web addresses/URLs
• Any pick list that allows freeform user entry
• Fields used for filtering purposes

8. No one took the time to validate the data before go-live.

Our StarfishETL migration team has sometimes encountered clients who give approval for a final migration without actually reviewing the data from the test run. When the client’s users go into the new system, they are shocked to find incorrect data, and the migration team must scramble to correct it while the system is already in use. The task is extremely time consuming and it erodes the buy-in of the client’s users because they are viewing the initial launch as a frustration and failure. The solution to this migration issue is to simply make sure you validate your data from the test migration before you give approval for the final go-live.

9. All users were not accounted for in the new system.

Make sure the user list for your new system accounts for any late additions before you do your final migration. It is nearly always the case that a few users are added to the source system after the test migration and before the final run. Be sure to double check your user list and cross-reference between the source and destination systems before the final migration. If all users are not established in the destination system, their data will map to a default user, which means it must be corrected by creating and running an updated job.