Storage data migration 101: Online versus offline migration

By | 3 minute read | August 16, 2018

Data Migration, Enterprise AI, Storage for AI, OpenStack Clouds, Linux

Storage data migration involves transferring data between storage subsystems or server systems, and it’s a common project for organizations across various industries. There can be several reasons to do a storage data migration:

  • Doing a storage hardware refresh
  • Resolving storage performance or capacity issues
  • Moving to storage subsystems with new features
  • Relocating your data center

In this 101 series, I’ll explain some basic concepts of storage data migration and the typical storage data migration process.

To get started, let’s talk about the two general kinds of data migration based on the impact to users and applications: online and offline migration.

Online data migration

With online migration, data will be migrated from one storage subsystem to another non-disruptively, which means that your applications are still up and running during the data migration, and users are transparent to the data migration, and there is little or no performance impact. Online data migration is always our highest target, but it can be hard to achieve.

The following are typical online data migration scenarios:

  • Migrating data virtualized by a storage virtualization appliance (such as IBM SAN Volume Controller)
  • Migrating data using a virtual machine hypervisor (for example, VMware ESXi)
  • Migrating data using a volume manager on an operating system (like LVM on IBM AIX)
  • Migrating data from the application level (such as IBM Spectrum Scale failure group, Oracle ASM)

However, downtime may be still required in the above scenarios under some circumstances. If you’re considering an online data migration, you’ll need to get a pre-check against the environment by experienced data migration consultants to determine whether it’s possible.

Offline data migration

Offline migration means the data migration process will be disruptive to users and applications. Depending on the migration method and data amount to be migrated, the downtime could vary from several hours to days.

The following are typical offline data migration scenarios, with an indication of the downtime required:

  • Migrating data using a storage virtualization appliance such as IBM SAN Volume Controller (two downtimes are required for moving in and moving out the appliance, but the applications can keep running while the background data migration is in progress)
  • Migrating data using an embedded data migration utility, for example, IBM FlashSystem A9000 (only one short downtime is required to connect the host with the new storage subsystem and define migration relationships)
  • Migrating data using storage synchronous/asynchronous replication, such as IBM DS8000 Metro Mirror/Global Mirror (one short downtime is required to switch to the new storage subsystem)
  • Migrating data using synchronizing tools such as rsync tool (these tools will check the timestamp and size of files and only copy files changed since the last successful copy; usually requires one downtime for final cutover)
  • Migrating data using backup/restore (usually requires long downtime and might therefore be the last choice)

Once you determine whether you’ll be doing an online or offline migration, there’s still a bit more you’ll need to know to have the basics covered. In the second part of this series, I’ll cover how applications access storage and what that means for the migration, as well as the storage data migration process, so please stay tuned.

Meanwhile, remember that there are storage experts available to help your organization with any challenges you might face on your storage migration journey.

Where you can find support

Storage data migration can be tricky. Here are the common challenges organizations may face:

  • Choosing an optimized migration method: Determining the best migration method for each application that minimizes downtime and reduces effort
  • Risk management: Foreseeing possible risks in the migration procedure and developing a relevant mitigation plan — for example, will the migration method cause extended or unexpected application downtime? Will the migration method cause data corruption or data loss?
  • Downtime estimation: Predicting accurate application downtime required for data migration, so as to reduce the impact to user
  • Schedule control: Arranging the data migration sequence of applications to reduce overall impact to user
  • Emergency decision and rollback plan: Being prepared in the event of an emergency situation during data migration to make decisions about continuing the migration or rolling back to the original storage, and determining if the rollback plan is verified and operable

IBM Systems Lab Services has a team of consultants with proven expertise on all aspects of designing and building storage and software defined infrastructure systems. This includes a wealth of experience with storage data migration, and we’ve already helped many organizations worldwide to migrate data to new storage platforms. Contact IBM Systems Lab Services today if you’ve got questions about migrating your data to a new storage platform.