Data migration: what it is and the best practices in performing any

data migration guide

As organizations adopt best-of-breed technology, they’re likely also moving away from SaaS applications they no longer need.

This transition can be ridden with pressure. 

Once your subscription with a SaaS app ends, all of its data becomes unavailable; this forces you to adopt its replacement quickly in the hopes of avoiding any loss of data. 

To execute this process smoothly, you’ll need to perform data migration. 

How can you go about performing one? We’ll review a solution worth trying, but before we do, let’s align on some of the basics: We’ll start by defining what it is and then move on to some general best practices for performing it.

Related: Everything you need to know about data synchronization

What is data migration?

Data migration is the process of moving data from one system to another. Depending on the situation, this can involve an ETL, ELT, or an extract and load approach.

Data migrations can be used in scenarios beyond replacing SaaS applications. To help you keep track of the forms they can take, let’s break down three we often see:

A visual breakdown of 3 approaches to data migration
  • Centralizing data: where data is gathered across data warehouses and loaded into a centralized data warehouse or data lake 
  • Replacing a SaaS application: where data gets taken out of a SaaS application, loaded into a data warehouse, and then has its records transformed to match the format that the destination system uses. Finally, the data is loaded into the destination system
  • Retiring a SaaS application: where data is extracted from a SaaS application and loaded into a data warehouse, typically for auditing purposes (e.g. old employee data)

Before we move beyond its definition, it’s worth noting the distinction between data migration and data integration—which involves collecting data from various sources, transforming it, and then loading it into a data warehouse. In the case of the latter (data integration), the data isn’t necessarily moved out of the source systems, and the purpose is often to help analysts perform business intelligence (BI). 

Related: How to solve for any data integration challenge 

Best practices to keep in mind

Regardless of the solution you end up using (we’ll cover them in the following section), your data migration process needs to include the following in order to operate smoothly.

An icon of a map and a destination pin sitting on it

1. Consider your data migration strategy carefully

Your team will need to decide between a “big bang” migration, where data is moved from the source system to the target system over a short period of time, or a “trickle” migration, where the process is performed in phases. 

One approach isn’t necessarily better than the other as each has its own pros and cons. 

The big bang migration allows you to perform the migration fairly quickly, but it may leave you with inadequate time for testing. Moreover, your source system will have to be down during the migration process. Meanwhile, the trickle migration allows you to avoid any downtime, as your source and target systems run in parallel. The process of implementing this approach, however, can be fairly complex.

In short: The approach you and your fellow stakeholders choose comes down to the nature of the migration, your available resources, and you and your colleagues’ personal preferences.

An icon of data

2. Perform data cleansing in the source system 

Left unresolved, data quality issues can cause all sorts of problems when put through your transformation logic. And even if the data is transformed and loaded into the target system successfully, your team is still left with unreliable information. 

An icon of a magnifying glass on top of a line graph

3. Conduct audit checks shortly after the migration is complete

Audit checks ensure that there isn’t any data loss and that the information between the source system and target system looks identical. 

These checks can take different forms, depending on the context of the migration. It might involve validating that records appear consistent across systems, that the summation of a certain type of record (e.g. the dollar value of invoices) is the same, etc.  

Related: How to use a managed file transfer solution

An icon of two gears in different colors

4. Test the migration in a staging environment

A poorly run migration leads to all kinds of adverse outcomes that can’t be undone. 

Given the stakes, you’ll need to test the migration in a staging environment rigorously, and only once everything is working as planned, you can move it to production.

An icon of a calendar

5. Stick to the plan

Issues can crop up at any point along your data migration project—and when they do, it’s important to remember that that’s par for the course. Occasional disappointments and stress can be demotivating, but the outcomes from the initiative make it well worth any headache.   

Related: Examples of workflow automations that benefit clients, prospects, and employees

How to implement a data migration

You have a couple of options to choose from:

1. Custom code: Your team executes the migration using scripts

Pros:

  • It allows you to keep the migration in-house, which removes any risk associated with using vendors
  • Assuming you have the right technical resources on-hand, the approach can be executed effectively

Cons:

  • Your organization may not have developers who can handle this. And even if you do, you’d be forcing them to tackle this in place of other business-critical tasks that they’re uniquely qualified to perform
  • If any of the develops who are involved in the migration leave your company, they may be taking valuable information with them—thereby putting your data migration plans at risk

2. Enterprise Automation Platform: Your team uses a 3rd-party vendor to execute the project 

Pros:

  • It provides a low-code UX, enabling business teams to review the data mappings, transformations, and even participate in the implementation process themselves
  • It offers pre-built connectors, allowing you to streamline connectivity between your source and target systems

Cons:

  • It introduces risks associated with using vendors
  • It may be unnecessary to engage a vendor, assuming the data migration project is relatively straightforward and the appropriate internal resources are made easily available

Related: A look at the two approaches you can take to stream data

Use Workato to perform data migrations—and a whole lot more

Workato, the leader in Enterprise Automation, offers all of the benefits outlined above, as well as the following:

  • Hundreds of thousands of automation templates (“recipes”), allowing your team to implement workflow automations quickly and with minimal customization
  • Customizable chatbots that enable employees to access actions and data in their apps without leaving their business communications platform, whether that’s Slack or Microsoft Teams
  • An API platform that lets you manage the entire lifecycle of APIs for internal teams and external partners
You can learn more about Workato by scheduling a demo with one of our automation experts.

About the author
Jon Gitlin Content Strategist @ Workato
Jon Gitlin is the Managing Editor of The Connector, where you can get the latest news on Workato and uncover tips and frameworks for implementing powerful integrations and automations. In his free time, he loves to run outside, watch soccer (er...football) matches, and explore local restaurants.