What is data migration?

There are many reasons your enterprise might need to undertake a data migration project. For example, you might be replacing servers or storage devices or consolidating or decommissioning data center.

Whether you’re moving to a public cloud, private cloud, hybrid cloud, or multicloud environment, you’ll need to find a secure, cost-effective, and efficient method of migrating your data to its new storage location.

Today, businesses generate ever-growing amounts of data and face increasingly urgent pressure to maximize the value they extract from it. In this climate, success depends increasingly on choosing optimal environments for your workloads and making sure your data is stored efficiently and accessibly.

Many enterprises are choosing to move workloads to the cloud in hopes of hosting their applications in the most cost-effective and best-performing IT environment available. Selecting the right data migration solution is a key component of the cloud migration planning process and should be considered even in its earliest stages.

You can choose among several options for transferring data from a local data center to the cloud, but broadly speaking, they fall into two categories:

• Online migration, in which data moves across the Internet or a private or dedicated WAN connection.
  
  
• Offline migration, in which data is transferred via a storage appliance that’s physically shipped between its data center of origin and the target cloud storage location.

The best option for your specific data migration project depends upon how much data you need to move, how quickly the migration must be accomplished, the types of workloads involved, and your security requirements.

Database migration is an example of specialized workload migration. Many public and private cloud providers offer tools that can facilitate or automate parts of the database migration process to ensure that your database remains secure throughout the transfer and that no data loss or corruption occurs. Additionally, most cloud providers offer migration services that can verify your data’s integrity after the transfer.

Typically, the first step in the database migration process involves converting the source database’s schema (if necessary) so that it’s compatible with the target database. A database’s schema is like a blueprint for how it is organized, controlling its logical architecture and structure. If the target database management system uses a data definition language (DDL) that is not compatible with the source’s, the schema will need to be converted.

The next steps are to migrate the data and set up ongoing incremental data warehouse updates. You can also consolidate multiple different databases into one during this process, if necessary. To learn more about how data is organized when stored in the cloud, take a look at “Cloud Databases Explained”.

Migrating an entire data center environment to the cloud or another location is a large-scale, comprehensive process. Completing such a migration project successfully—with minimal downtime or disruption to operations—requires careful planning and coordination.

When contemplating a data center migration or any other large-scale data migration project, it’s important to consider timelines early in the planning stages, since petabyte-scale transfers can take multiple weeks to complete, even with relatively high-speed network connections.

The more carefully your enterprise plans its data migration, the less likely you are to encounter surprise costs or unplanned downtime and the less likely it is that your end users will be frustrated or inconvenienced during and after the migration. You’ll want to establish goals, set a timeline, and anticipate any challenges that you may encounter.

There are three primary factors you should consider when determining how you’ll approach the project:

Type of workload. Specialized workloads—such as virtual machines (VMs), backups, or databases—can usually be moved with software vendor-provided tools that are specific to the type of data being migrated. If you don’t have access to these tools, you’ll want to carefully plan for potential downtime. You can transfer data for mission-critical workloads in stages, testing at intervals throughout the process and keeping the source and target systems running in parallel. Alternatively, you can plan a large-scale transfer outside of production hours (if you can accomplish it the available window).

Volume of data. When you’re migrating fewer than 10 terabytes (TB) of data, shipping the data to its new storage location on a client-provided storage device is often the simplest and most cost-effective method. For transfers involving larger amounts of data—say, up to multiple petabytes (PB)—a specialized data migration device supplied by your cloud provider can be the most convenient and affordable option. While, in theory, you could use online migration for any amount of data, time constraints limit its feasibility for large amounts of data.

Speed to completion. For online migrations, the amount of data being transferred and the speed of your network connection will determine how long data migration takes. For offline migrations, shipping time must be taken into account. If start-to-finish migration speed is your primary concern—and if you have adequate available bandwidth to dedicate to the migration—online transfer could be the best option. But if your migration deadline is flexible and/or you have bandwidth or other networking constraints, offline migration might be the right choice.

To ensure that your project goes smoothly, adhere to the following best practices:

Understand the data and what it’s used for. Who uses the data now, who will use it in the future, and how will it be used? Data that’s leveraged for analytics, for example, may have very different storage and formatting requirements than data being retained for regulatory compliance. Be sure to gather information from all relevant stakeholders and business units throughout the migration process.

Assess the source and target environments carefully. Will the same operating system be running in both environments? Will database schemas or other formatting need to change? Are there any problems (like redundancy issues or an excess of “dirty” data) that need to be addressed before the migration?

Verify business requirements and potential impact early in the process. What kind of migration timeline is necessary? If a data center is being decommissioned, when will its lease expire? What types of data security must you maintain throughout the migration process? Is any data loss or corruption tolerable, and if so, how much? How would delays or unexpected stumbling blocks affect the business?

Though the benefits of modernizing IT systems outweigh the risks associated with data migration—especially over the long term—data migration can be stressful and risky. Here are some of the risks to account for:

• Security: Make sure all data is securely encrypted before migration. For offline migrations that involve shipping data storage devices, verify security of the shipper’s freight and logistics services.

• Long transfer times: It can be challenging to predict online transfer times with complete accuracy. Network bottlenecks may restrict connection speeds, or system hardware limitations might throttle the amount of data that can be read from or written to them.
  
  
• Higher-than-expected costs: Unanticipated costs often result from improper planning. For example, online transfers that take longer than expected will incur additional charges. Retaining a vendor-provided storage appliance for longer than you’d initially agreed to keep it (which can result from transferring the data to the appliance or shipping delays) can incur extra charges as well.

Today, there are plenty of tools to facilitate enterprise data migrations. These include vendor-specific solutions offered by cloud providers to support their customers’ move into their public or private cloud environment, as well as licensed and open source tools. Your data migration strategy will determine which tools work best for your project.

Some popular choices include the following:

• Veeam offers a Quick Migration tool for VMware vSphere to speed and simplify the process of migrating VM-based workloads between storage environments and hosts.
  
  
• Zerto offers a converged disaster recovery, backup, and workload mobility platform that supports migrations ranging in size from single applications to entire data centers.
  
  
• Rclone is an open source command line utility that can be used to migrate data to and from cloud object storage. It can automatically segment large objects and upload their parts in parallel.
  
  
• Cyberduck is an open source FTP and SFTP client that can be used to transfer individual files or file volumes between hosts or into a cloud environment.

A data migration service can supplement your in-house capabilities or manage the entire migration process from strategy through completion, testing, and documentation. The latter type of service—often referred to as “white glove data migration service”—is more expensive, as you’d expect, but may be worthwhile when your in-house data migration expertise is limited and the applications you’re migrating are business-critical. A database migration consultant can help you plan a cost-effective migration process that minimizes or eliminates downtime.