Learn advanced automation concepts with FlashCopy Manager for VMware

Learn how to leverage the FlashCopy Manager for VMware command-line interface for automating your data protection strategy.

This article uncovers flexible automation possibilities of FlashCopy® Manager for VMware by using its command-line interface in combination with scripting and the VMware vSphere SDK for Perl. You can create advanced data protection scenarios for highly dynamic vSphere environments, that are not possible with the Data Protection for VMware vCenter GUI plug-in, by using these methods.

Share:

Frank Lautenbach (lautenb@de.ibm.com), IT Architect, IBM

Frank Lautenbach is an IT Architect at IBM with 15 years of experience in systems and software engineering in multiple domains. Formerly engaged in the areas of industrial image processing, embedded systems, and telematics, today, Frank is a member of the IBM Tivoli Storage FlashCopy Manager development team. Frank was the technical release lead for FlashCopy Manager releases 2.2 and 3.1.



Thorsten Gosert (gosert@de.ibm.com), IT Specialist, IBM

Thorsten Gosert has worked as an IT Specialist at IBM since 2001. Formerly engaged in the areas of AIX and DB2 administration, Thorsten also has several years of software development experience in the areas of supply chain management, telematics, and Tivoli Storage Solutions. Since 2009, Thorsten has been a member of the IBM Tivoli Storage FlashCopy Manager development team.



26 November 2012

Introduction

IBM® Tivoli® Storage FlashCopy Manager software provides fast application-aware backups and restores by making use of advanced snapshot technologies in IBM storage systems. Support for VMware vSphere environments was added in version 3.1. You can back up complete datastores in just minutes while keeping virtual machines running without interruption.

IBM Tivoli Storage FlashCopy Manager for VMware provides a vCenter GUI plug-in which is a graphical user interface (GUI) that integrates with the VMware vSphere client. It allows easy access and control of backup and restore functionality.

Less known is the command-line interface (vmcli) that IBM Tivoli Storage FlashCopy Manager for VMware also provides. The vmcli was designed as a dual purpose interface. The vCenter GUI plug-in interacts with FlashCopy Manager core components using the vmcli as a machine-to-machine interface, which is what the vmcli is optimized for. However, you can also use the vmcli to issue commands directly.

This article describes how you can implement advanced, automated, and more flexible backup and restore operations by directly using the vmcli in combination with shell and Perl scripting. For highly dynamic vSphere environments (making extensive use of vMotion), this approach, in particular, provides more flexibility when compared to the GUI and it can also provide capabilities that are not possible using the GUI.


Prerequisites

This article presumes an operational VMware vSphere or ESX(i) environment, version 4.1, or later. IBM Tivoli Storage FlashCopy Manager for VMware V3.1, or later must already be installed and configured properly. Before using the vmcli to automate backup and restore operations, ensure that you can successfully back up and restore from the vCenter GUI plug-in. See Resources for links to information about how to install and configure IBM Tivoli Storage FlashCopy Manager for VMware.

Parts of this article use the vSphere SDK for Perl provided by VMware. To use vmcli commands from within the Perl scripts, you must install the Perl SDK on the vStorage Backup server where IBM Tivoli Storage FlashCopy Manager for VMware is installed. See Resources for vSphere SDK for Perl links.


Backup and restore granularity

IBM Tivoli Storage FlashCopy Manager for VMware supports the data protection of virtual machines that are stored on VMFS datastores comprised of one or more LUNs from SAN-attached storage. IBM Tivoli Storage FlashCopy Manager is based on hardware snapshots of LUNs, so, technically, backups are always performed at the datastore level -- performing a consistent hardware snapshot of all LUNs that make up the datastore. Nevertheless, you can select distinct virtual machines for backup (and exclude virtual machines from the backup). Only selected virtual machines can be:

  • Restored
  • Backed up in a file system consistent state (because an optional VMware snapshot or suspend operation is performed before the actual hardware snapshot)
  • Offloaded to Tivoli Storage Manager (when IBM Tivoli Storage Manager for Virtual Environments is installed)

Restore and attach operations are always performed at the level of virtual machines or even single disks.

Understanding this background information, can help you provide the correct information that is required by the vmcli to back up, restore, and attach virtual machines.


Collecting information about the vSphere environment

Direct use of certain vmcli commands requires that you use an input file (infile) to specify the entities to process. The infiles must use the correct syntax. The formats for the backup and restore infiles are described in Detailed description of infiles.

For example, the backup infile (backupObjectListFile) expects datastores to be specified either by URL or by name. An explicit include or exclude statement for a virtual machine must identify the virtual machine by its instance UUID.

In the VMware vSphere Client, a datastore URL can be determined from the summary page of the selected datastore. However, the summary page does not show the instance UUID of virtual machines.

To determine these low-level details, the following VMware tools can be used:

Because PowerCLI is specific to the Windows platform, but FlashCopy Manager for VMware runs on Linux systems, this article focuses on the Managed Object Browser and Perl SDK tools.

Managed Object Browser

The Managed Object Browser is a web-based server application that runs on all ESX(i) and vCenter Server systems. The MOB is ideal for exploring the data and object model of a vSphere environment in detail. It can be accessed by a common web browser by entering this URL:

https://VirtualCenter_or_ESX(i)_server-name/mob

Your web browser prompts you for a user name and password. When connecting to the host, a warning about SSL certificate authority might be displayed. In this case, import the certificate to proceed.

After you log on, a page similar to Figure 1 is displayed.

Figure 1. Entry page of the Managed Object Browser
Managed Object Browser home page showing properties and methods

Navigating through the model presumes a basic knowledge of how the vSphere object model is structured. At the same time, clicking through the model helps to increase your understanding of it. For our purpose, it is important to understand the inventory hierarchy and the relationships between its objects (datacenters, datastores, virtual machines, hosts, and so on). This structure represents the underlying object model of the inventory views in the VMware vSphere Client. The inventory hierarchy structure is basically the same for both a vCenter Server and an ESX(i) host. However, the structure for a vCenter Server can be more complex.

Figure 2 shows the hierarchy for a vCenter Server.

Figure 2. VMware inventory model
Diagram shows the hierarchy of a vCenter Server

The root object of the hierarchy is the ServiceInstance singleton. Its properties and methods are shown when you look at the MOB with your web-browser (see Figure 1). The key property of this singleton object is the content property. The ServiceContent data object has a long list of properties. Its rootFolder property is the key to exploring detailed information about the inventory hierarchy that is depicted in Figure 2. The root folder recursively holds datacenters and subfolders, if applicable. In common environments, datacenters are not placed into folders so you can see all of your datacenters enlisted by the childEntity[] property of the rootFolder managed object (Figure 3).

Figure 3. Root folder properties
Diagram shows root folder properties

Clicking a datacenter reference link leads to the datacenter managed object which in turn displays a list of its datastores (datastore property). Each datacenter managed object has four folder objects that contain: virtual machines (VM) and virtual applications, datastores, hosts (compute resources), and networks (the network folder is only sketched in Figure 2). Clicking these folder object references leads you to a recursive hierarchy of folders and entities. When you click through the VM folder hierarchy, you can see that it corresponds to the hierarchy of the VMs and Templates inventory view in the VMware vSphere Client. The recursive hierarchy of folders, compute resources, hosts, and resource pools under the Host Folder corresponds to the Hosts and Clusters view. Similarly, the Datastore Folder maps the Datastores view, and the Network Folder maps the Networking view.

To find out more about this topic, VMware provides a number of valuable resources. Examples are the vSphere API Reference Guide and the vSphere Web Services SDK Programmer's Guide. Both guides are available from the VMware vSphere Web Services SDK Documentation Center. There is also a valuable series of articles about this topic on the VMware Blog. Part 3 of the series Introduction to the vSphere API covers the inventory hierarchy in detail. See Resources for links to these sources.

For our purpose, we are interested in using datastore URLs and instance UUIDs of virtual machines to identify entities in FlashCopy Manager for VMware infiles.

Determining the instance UUID of a virtual machine

To determine the instance UUID of a virtual machine, go to the VirtualMachine managed object (content > rootFolder > childEntity[] (Datacenter) > vmFolder > ...). Click the config property to retrieve the VirtualMachineConfigInfo data object. Its instanceUuid property has the required information.

Determining the URL of a datastore

Although the datastore URL is shown in the VMware VSphere Client, you can also find it using the MOB. If you follow this method, you can copy and paste the URL to your infiles.

Go to the Datastore managed object (content > rootFolder > childEntity[] (Datacenter) > datastoreFolder > ...).

Tip: In addition to the datastoreFolder property, the DataCenter managed object also has a datastore property that represents a flat list of all datastores owned by the datacenter. Rather than traversing the folder structure down to your datastore of interest, consider using this shortcut to the datastores.

vSphere SDK for Perl

Using the MOB lets you learn more about the object model behind vSphere and you can also use it to retrieve the necessary information that FlashCopy Manager for VMware infiles require. However, using the MOB can be cumbersome and it is not a process that you can automate. As an alternative, you can use VMware's Perl SDK to retrieve all kinds of information from a vSphere environment.

The installation of the SDK, including the necessary prerequisites, is described in the vSphere SDK for Perl Installation Guide. After you install the SDK, review the vSphere SDK for Perl Programming Guide. A link to both guides can be found in Resources.

Start with the "Hello Host" example script (connect.pl) to ensure that your SDK is installed properly and a connection to your server can be established. You can find the script together with a group of other example scripts in the /usr/lib/vmware-vcli/apps directory.

At a minimum, each script requires the –url or –server command-line argument specifying the vCenter or ESX(i) server to connect to. Optionally, you can specify user name (-user) and password (-password). If you omit these arguments, they are queried interactively when you start the script.

To illustrate, the script in Listing 1 retrieves the instance UUID of a virtual machine that's identified by its name. It also obtains the URL of a datastore identified by its name, together with a few other attributes.

Listing 1. Perl script retrieves basic information from vCenter server
#!/usr/bin/perl -w

use strict;
use warnings;
use VMware::VIRuntime;

Opts::parse();
Opts::validate();

# Connect to server; implicitly evaluates environment variables and CLI arguments:
Util::connect();

displayVMInfo("VM_xyz");
displayDatastoreInfo("Datastore_xyz");

Util::disconnect();

sub displayVMInfo {

  my ($name) = @_;

  my $vmref = Vim::find_entity_view(view_type => 'VirtualMachine',
                                                      filter => {'name' => $name});

  print "VM name: " . $vmref->name . "\n";
  print "VM moref: " . $vmref->{'mo_ref'}->value . "\n";
  print "VM Instance UUID: " . $vmref->config->instanceUuid . "\n";
  print "VM UUID: " . $vmref->config->uuid . "\n\n";
}

sub displayDatastoreInfo {

  my ($name) = @_;

  my $dsref = Vim::find_entity_view(view_type => 'Datastore',
                                                      filter => {'name' => $name});

  print "Datastore name: " . $dsref->name . "\n";
  print "Datastore moref: " . $dsref->{'mo_ref'}->value . "\n";
  print "Datastore URL: " . $dsref->info->url . "\n";
  my $vmsrefs = $dsref->vm;
  print "Number of VMs: " . @$vmsrefs . "\n";

  foreach my $vmmoref (@$vmsrefs) {
     my $vm = Vim::get_view(mo_ref => $vmmoref);
     print "VM name: " . $vm->name . "\n";
  }
  print "\n";
}

The script starts with use statements and parses command-line arguments. For simplicity, the names of the virtual machine and datastore are hardcoded so the script parses only built-in options. After that, the first VMware specific function is called to connect to the server defined by the –url or –server command-line argument.

The script then calls two functions that retrieve and display the details for the specified VM and the specified datastore. The key function that retrieves properties of a single entity (a virtual machine, host, datastore, and so on) is Vim::find_entity_view(). Its counterpart, Vim::find_entity_views(), can be used to retrieve data from multiple entities of the same type, simultaneously. You can use this function to collect VMs meeting a certain criteria. You can also use this function for other purposes, such as collecting certain sets of datastores. Review the vSphere SDK for Perl Programming Guide and the sample scripts packaged with the vSphere SDK for Perl (see Resources) for details about several usage scenarios.


Identifying inventory objects in a vSphere environment

There are multiple ways to uniquely identify entities in a VMware vSphere environment. The most common one is the Managed Object Reference (MoRef). The MoRef is used to uniquely identify each server-side managed object of the internal object model. This method of identification includes, but is not limited to, the managed entities that make up the inventory (virtual machines, datastores, hosts, and so on). The following examples show what a typical MoRef looks like:

  • vm-124 identifying the server-side object representation of a virtual machine
  • datastore-6756 identifying the server-side object representation of a datastore

Knowing the MoRef of a managed object allows you to query the object through the MOB using this URL:
https://VC_SERVER/mob/?moid=moref

The MoRef is only unique within the scope of a particular target server. For example, a virtual machine, can have different MoRefs on the ESX/ESXi host where it currently runs and the vCenter server that manages it. So, its purpose is more for server-internal management rather than external usage. For that reason FlashCopy Manager uses alternative ways for the unique identification of virtual machines and datastores.

Each virtual machine has two properties, the UUID (Universally Unique Identifier) and the instance UUID. The UUID is automatically assigned to a virtual machine and stored in its SMBIOS system information descriptor.

The UUID is a 16-byte Integer that is based on the identifier of the physical computer and the path to the virtual machine's configuration file. However, this UUID may not actually be as unique or persistent as you might expect. If you move or copy the virtual machine, the first power-on of the moved or copied virtual machine gives you a choice of creating a new UUID or keeping the original UUID. Depending on how you answer this question it can lead to duplicate UUIDs.

The instance UUID is also a 16-byte Integer, but, in contrast to the UUID, it unambiguously identifies an instance. It is unique, fixed, and universal for each individual instance of a virtual machine, within the vSphere environment of the vCenter server that manages it. This characteristic enables it to be used to identify virtual machines in FlashCopy Manager infiles.

A datastore is identified by its URL which has characteristics comparable to the instance UUID of a virtual machine. In fact, the URL of VMFS datastores includes a 16-byte UUID prefixed by storage and file system type identifiers. For example:
sanfs://vmfs_uuid:4d9f541a-624d0fe0-4f94-001517868541/

For NFS datastores, the URL looks different:
netfs://9.152.148.170//vol/VWNSANbackesxfcm2/

Note: Instance UUIDs and datastore URLs are only unique within the scope of a single vCenter environment managed by a single vCenter server. If there are multiple vCenter servers that are controlling multiple environments, the UUIDs and URLs are only unique in combination with the UUID of the vCenter.


Overview of vmcli commands

You can use the vmcli to perform backup and restore operations. You can also use it to perform other operations, such as deleting backups or backup schedules and viewing configuration information.

Definition of vmcli terms: tasks and runs

The Data Protection for VMware command-line interface (vmcli) introduces a concept of tasks and runs. A task describes a function that can be called, for example, a backup, restore, or delete function. Only backup tasks can be scheduled. The execution of a task, is defined as a run.

Task definitions are stored in the vmcli database. The call of the function defined in a task, results in a run. The console output that results from a run is captured and stored in the vmcli database. This information can be recalled later by the inquire command set.

In general, the output on the console has the schema shown in Output 1.

Output 1. Typical console output of a vmcli command
#TASK 1 backup 20120626072850631
#PARAM <task param>
#PARAM <task param>
.
.
.
#RUN 1 20120626072850631
#LANG en_US
# <run params, info messages, parent/child... >,
.
.
#END RUN 1  20120626073050631
#END TASK 1

To allow the automated parsing of the console output, the vmcli provides several tags. Table 1 lists the important tag types.

Table 1. vmcli tag types
TAGDescriptionExample
#PARAM paramname=valueDefines the parameter name and its associated value. This tag is used for parameters which belong to a task or a run. The parameter name is separated from its value by an equal sign (=).#PARAM BACKUP_TYPE=FCM
#PARAM OBJECT=datastoreurl:ds:///vmfs/volumes/4fa8df2e-4cce5dfe-b31b-001a64d354b0/
#PARAM OBJECT=vminstanceuuid:50382b03-ed32-b96f-6407-58da7ed56192
#INFO MSG_IDInfo message
#WARNING MSG_IDWarning message
These tags are used to print out informational or warning messages. These tags are not interpreted, but provide additional information to the user.#INFO FMM0005I Start of program at: Mon 18 Jun 2012 04:53:53 PM CEST.
#WARNING FMM1554W The agent 'acsgen V. 3.2.0.0' terminated with exit code 1.
#CHILD paramname:value
#PARENT paramname:value
The CHILD and PARENT tags are used to group parameters within a run, for example, to list the parameters of a single VM.
The PARENT tag identifies the parent object, in our examples the UUID of the VM. The CHILD tag provides the parameters for the VM.
#CHILD datacentername:FCM_VC5
#PARENT vmuuid:4238bc7d-579f-1bf4-2bac-3643b42582ac
#CHILD vminstanceuuid:5038a7eb-20f6-e4f0-a176-f84b380c65eb
#PARENT vmuuid:4238bc7d-579f-1bf4-2bac-3643b42582ac
#CHILD vmname:RINV3
#PARENT vmuuid:4238bc7d-579f-1bf4-2bac-3643b42582ac

Inquire and inquire_detail commands

Backup status and details can be obtained from the vmcli database, the Tivoli Storage FlashCopy Manager for VMware repository and the Tivoli Storage FlashCopy Manager for Virtual Environments repository through the vmcli inquire commands. You can use the vmcli -f inquire and vmcli -f inquire_detail commands.

The vmcli is the common interface for Tivoli Storage FlashCopy Manager for VMware and Tivoli Storage FlashCopy Manager for Virtual Environments. In general, the vmcli calls command-line tools of the underlying installed product to initiate operations like backup and restore. While the vmcli -f inquire command gathers information from the vmcli database, the vmcli -f inquire_detail -t backupType command retrieves additional information from the repository of the product specified by the -t parameter. The following are valid backupType values: ALL, FCM, and TSM. If a backup type is not specified, all available product repositories are queried.

The inquire command

The inquire command retrieves information from the vmcli database. There are three main types of inquire commands:

The following examples show you how to use the three types of inquire commands.

  1. Inquire for task types

    vmcli -f inquire

    This command prints out all tasks and associated runs that are stored in the vmcli database. Tasks could be of the types backup, restore, attach, detach, and delete.

    The output for this type of inquire command is structured in Output 2:

    Output 2. Inquire output
    #TASK 1 backup 20120626072850631
    #PARAM <task param>
    #PARAM <task param>
    .
    .
    .
    #RUN 1 20120626072850631
    #LANG en_US
    # <run params, info messages... >
    .
    .
    .
    #PARAM STATUS=success
    #END RUN 1 2012062607331451
    .
    .
    .
    #RUN n 20120626072850631
    #LANG en_US
    # <run params, info messages... >
    .
    .
    .
    #PARAM STATUS=success
    #END RUN n 2012062607331451
    #END TASK 1
    .
    .
    .
    #TASK m backup 20120626072850631
    #PARAM <task param>
    #PARAM <task param>
    .
    .
    .
    #RUN 1 20120726072850631
    #LANG en_US
    # <run params, info messages... >
    .
    .
    .
    #PARAM STATUS=success
    #RUN 1 20120726072850631
    .
    .
    .
    #RUN n 20120626072850631
    #LANG en_US
    # <run params, info messages... >
    .
    .
    .
    #PARAM STATUS=success
    #END RUN n 2012062607331451
    #END TASK m

    You can also specify other options on the command such as --type taskType, -t backupType, and -B backupID to filter for the objects of interest. You can find more information about inquire options in the IBM Tivoli Storage FlashCopy Manager for VMware Installation and User's Guide (see Resources).

  2. Inquire for tasks

    vmcli -f inquire -T taskID

    This command displays the task with the specified task ID and all its corresponding runs. The output for this type of inquire command resembles the output in Output 2 except it only contains information for the specified task. As with inquire for task types, you can also specify other options on the command to filter for the objects of interest.

  3. Inquire for virtual machines

    This type of inquire command provides a more virtual machine centric view of the backup metadata. It shows virtual machine backup information rather than task information, therefore, filter options like --type and -t do not apply.


    The following command queries all virtual machine backups from the datacenter that is specified in an infile:

    vmcli -f inquire -I infile

    The infile file uses the following syntax:

    datacenter:datacentername


    The following command queries all backups for a VM name on a specified datacenter:

    vmcli -f inquire -t FCM -q vmsingle -I infile

    The infile file uses the following syntax:

    datacenter:datacentername
    vmname:vmname


    The output contains a dummy task with a dummy run to follow the common structure for inquire output, as illustrated in Output 3.

    Output 3. Inquire output for virtual machines
    #TASK 0 inquire 20120731091714637
    #PARAM BACKEND=FCM
    #PARAM DATACENTER_NAME=FCM_DC1
    #RUN 0 20120731091714639
    #LANG en_US
    #PARAM BACKEND=FCM
    #CHILD backupid:A0H558IZZY
    #PARENT vmuuid:42045154-a9db-c04e-f0e7-617eaf72bf0d
    #CHILD backupdate:20120727141555331
    #PARENT vmuuid:42045154-a9db-c04e-f0e7-617eaf72bf0d
    #CHILD vminstanceuuid:5004267e-6816-bf3d-2fd0-852af30f161e
    #PARENT vmuuid:42045154-a9db-c04e-f0e7-617eaf72bf0d
    #CHILD vmname:TGTest1
    .
    .
    .
    #CHILD attachedstate:ATTACHABLE
    #PARENT vmuuid:422db8ea-7328-3bff-29a1-fed7342aae5a
    #PARAM STATUS=success
    #END RUN 0 20120731091714962
    #END TASK 0
    #INFO FMM16014I The return code is 0.
    #END

    This special type of inquire does not print out the run information that is stored in the vmcli database. It retrieves the backup information for the VMs from tables that are populated during Tivoli Storage FlashCopy Manager backup operations and contains information about the backup of a single VM.

The inquire_detail command

Querying Tivoli Storage FlashCopy Manager for VMware by using the inquire_detail command with the -t backupType parameter, provides you with the backup information from the product repository specified by backupType.

Output 4 shows an example.

Output 4. Inquire_detail output
./vmcli -f inquire_detail -t FCM
#TASK 1280 inquire_detail 2012062708512363
#RUN 1308 2012062708512363
#LANG en_US
#PARAM BACKEND=FCM
#PARAM Version: 3.1.0.0 Build: 548
#INFO FMM0005I Start of program at: Wed 27 Jun 2012 08:51:23 AM CEST.
#BACKUP
#PARAM BACKUP_ID=A0H3WJG4DF
#PARAM STATUS=SUCCESSFUL
#PARAM USABILITY_STATES=REMOTELY_MOUNTABLE,REPETITIVELY_RESTORABLE
#Backup
#PARAM BACKUP_ID=A0H3LNZX25
#PARAM STATUS=SUCCESSFUL
#PARAM USABILITY_STATES=REMOTELY_MOUNTABLE,REPETITIVELY_RESTORABLE
#INFO FMM0020I End of program at: Wed 27 Jun 2012 08:51:23 AM CEST.
#INFO FMM0021I Elapsed time: 00 sec.
#INFO FMM0024I Return code is: 0.
#PARAM STATUS=success
#END RUN 1308 20120627085123897
#END TASK 1280
#INFO FMM16014I The return code is 0.
#END

To retrieve information about a single backup, this output can be filtered by specifying the -B backupID parameter as shown in Output 5.

Output 5. Inquire_detail output filtered by backup ID
./vmcli -f inquire_detail -t FCM -B A0H3WJG4DF
#TASK 1280 inquire_detail 2012062708535563
#RUN 1308 2012062708535563
#LANG en_US
#PARAM BACKEND=FCM
#PARAM Version: 3.1.0.0 Build: 548
#INFO FMM0005I Start of program at: Wed 27 Jun 2012 08:53:55 AM CEST.
#BACKUP
#PARAM BACKUP_ID=A0H3WJG4DF
#PARAM STATUS=SUCCESSFUL
#PARAM USABILITY_STATES=REMOTELY_MOUNTABLE,REPETITIVELY_RESTORABLE
#INFO FMM0020I End of program at: Wed 27 Jun 2012 08:53:55 AM CEST.
#INFO FMM0021I Elapsed time: 00 sec.
#INFO FMM0024I Return code is: 0.
#PARAM STATUS=success
#END RUN 1308 20120627085355897
#END TASK 1280
#INFO FMM16014I The return code is 0.
#END

Backup command

With the backup command you can define a backup task, rerun a defined backup task, and change a defined backup task.

A backup task can be a scheduled task that runs at a specific time or interval. Alternatively, the task can be run immediately. The backup command requires an infile (backupObjectListFile) that contains the VMware objects to be included or excluded in the backup. A detailed description of the backup infile is provided in Detailed Description of infiles.

The backup command uses the following syntax:

vmcli -f backup [-T taskID] [-t backupType[,backupType]] -I backupObjectListFile 
[-B backupID] [-D deviceClass] [-H esxHost]
[--vmbackupmode (SNAPSHOT_INCL_MEM|SNAPSHOT_EXCL_MEM|SUSPEND|ASIS)]
[--start scheduleStartDateTime [--interval intervalSeconds [--end scheduleEndDateTime]]
[--priority 1-5]] [--name taskName] [--description descriptionInFile.txt] [--runnow]

For a detailed description of the backup parameters, review Tivoli Storage FlashCopy Manager for VMware Installation and User's Guide (see Resources).

Backup command examples:

  • Run a Tivoli Storage FlashCopy Manager for VMware backup:
    vmcli -f backup -t FCM -I backupInfile -D STANDARD --vmbackupmode SNAPSHOT_INCL_MEM
    --name backup_test01 --description desc.txt –-runnow
  • Run a Tivoli Storage FlashCopy Manager for VMware backup at given start time:
    vmcli -f backup -t FCM -I backupInfile -D STANDARD --vmbackupmode SNAPSHOT_INCL_MEM
    --name backup_test02 --description desc.txt –-start 201110012055000
  • Run a Tivoli Storage FlashCopy Manager for VMware backup at a scheduled time:
    vmcli -f backup -t FCM -I backupInfile -D STANDARD --vmbackupmode SUSPEND 
    --name "scheduled_backup" --start 20111001210000 --interval 3600 --end 20111001220000
  • Rerun an existing backup task:
    vmcli -f backup -T 1234 –-runnow
  • Update an existing backup task:
    vmcli -f backup -T 1234 -t FCM -I backupInfile -D STANDARD
    --vmbackupmode SNAPSHOT_EXCL_MEM --name fcm_backup
    --description desc.txt

Restore command

The restore command can be used to restore a VM or virtual disk of a VM either to the original location or to an alternative location.

The restore command uses the following syntax:

vmcli -f restore [-t backupType] -I restoreObjectListFile [-H esxHost]

For details about restore parameters, refer to the Tivoli Storage FlashCopy Manager for VMware Installation and User's Guide (see Resources).

Information about which back up and which VMs or virtual disks from this backup are restored and to which location is provided in the restoreObjectListFile infile. A detailed description of the restore infile is provided in Detailed Description of infiles.

Important: A restore operation is limited to a single backup. Therefore, you cannot restore virtual machines from different backups using the same restore command.

Restore command examples:

  • Run a restore command using the value for the AUXILIARY_ESX_HOST parameter in the FlashCopy Manager for VMware configuration profile to attach the datastore:
    vmcli -f restore -t FCM -I restoreObjectListFile
  • Run a restore command using the specified ESX host:
    vmcli -f restore -t FCM -I restoreObjectListFile -H esxHost

Detailed description of infiles

Input files (infile) are used to specify which entities a vmcli command should process. The following types of infiles are described in this section:

In any infile, lines starting with a # character and blank lines are ignored.

Backup infile

For the backup infile, at least one datastore statement is required. Datastore statements contain the datastoreurl or datastorename identifier. Wildcards are allowed in datastore name and VM name values.

The supported wildcards are:

  • ? - represents any single character
  • * - represents any character sequence (also empty)

For a datastore, include and exclude(!) statements are allowed for datastoreurl and datastorename identifiers. For virtual machines, include and exclude(!) statements are allowed for the vminstanceuuid identifier, but only exclude(!) statements are allowed for the vmname identifier.

The following rules apply to the backup infile:

  1. Specifying datastores by datastore url or by datastore name without explicitly including or excluding virtual machines will back up all VMs on these datastores.
  2. It is possible to have both datastore url and datastore name in the same infile.
  3. A datastore statement can be turned into an exclude statement by prefixing it with the ! character.
  4. VM instance UUID statements (vminstanceuuid) are allowed as include and exclude statements; wildcards cannot be used.
  5. VM name statements (vmname) are only allowed as exclude statements; wildcards can be used.

    Examples:

    • !vmname:* excludes all VMs from backup
    • !vmname:TG* excludes all VMs starting with TG
    • !vmname:??Test? excludes all VMs with a name that starts with two characters followed by Test, followed by another single character, for example, TGTest1, FLTest2, ...
  6. You must not mix VM include statements with VM exclude statements. If you do, the include statements overrule all exclude statements, that is, if there is a single include statement all exclude statements are ignored.
Table 2. Backup infile statements
Statement Description
datastoreurl:dsUrlInclude datastore with this URL in the backup.
datastorename:dsNameInclude datastore with this name in the backup; wildcards are allowed.
!datastoreurl:dsUrlExclude datastore with this URL from the backup.
!datastorename:dsNameExclude datastores that match this name from the backup; wildcards are allowed.
vminstanceuuid:vmInstanceUuidInclude VM with this instance UUID in the backup. Fails if it is not found within the included datastores which additionally need to be specified in the infile when this statement is used.
!vminstanceuuid:vmInstanceUuidExclude VM with this instance UUID from the backup.
!vmname:vmNameExclude VM with this name from the backup (needs to be unique); wildcards are allowed.

Example backup infile:

#backup all VMs in datastore DS_TG_fcmcli2_XIV_1
datastoreurl:sanfs://vmfs_uuid:4ebb8d9a-e3b70869-1202-e41f132d6c1c/
# exclude only vm TGTest3
!vmname:TGTest3

Restore infile

The restore infile must have one backup ID and one VM identifier per line. Optionally, you can also specify a different target VM, an alternative datastore, an ESX host, and virtual disks to be restored.

You cannot have different backup IDs (backupid) in the same restore infile. The restore process supports the restore of a single backup only. Having different backup IDs in the restore infile results in an error.

Table 3. Restore infile statements
StatementDescription
backupid:backupid vminstanceuuid:vmInstanceUuidRestore of a VM identified by its instance UUID.
backupid:backupid vminstanceuuid:vmInstanceUuid::vmname:newvmnameRestore of a VM to original location but with a new name.
backupid:backupid vminstanceuuid:vmInstanceUuid datastoreurl:dsUrl::datastoreurl:dsnewUrlRedirected restore of a VM to another datastore.
backupid:backupid vminstanceuuid:vmInstanceUuid::vmname:newvmname datastoreurl:dsUrl::datastoreurl:dsnewUrlRedirected restore of a VM to a new name and another datastore.
backupid:backupid vminstanceuuid:vmInstanceUuid esxhostname:esxhostnameRestore of a VM and registration of this VM with the ESX host.
backupid:backupid vminstanceuuid:vmInstanceUuid diskkey:diskkey1,diskkey2...Restore the disks with the specified keys. Original disk files are overwritten.
backupid:backupid vminstanceuuid:sourcevmInstanceUuid::
vminstanceuuid:targetvmInstanceUuid diskkey:diskkey1,diskkey2...
Restore the disks with the specified keys to the specified target VM as new disks.

Example restore infile

# restore VM to vmname VM_new
backupid:A0H3YHFNH1 vminstanceuuid:502d38ff-580c-9427-1014-95dc794301b6::vmname:VM_new

Use Cases

This section gives examples that describe how to automate typical scenarios related to data protection in a dynamic vSphere environment.

Note that these script examples are not meant for use in a production environment as-is, but serve the purpose of explaining the basic concepts.

Pure datastore backups

Problem

Currently it is not possible on the vCenter GUI plug-in to select only a datastore in order to back up all VMs in this datastore, including those eventually added in the future.

Solution

Create a backup infile manually and put only the datastore url or datastore name in this infile. The infile might look as follows:

datastorename:TG_fcmcli2_XIV_2

Then simply run the backup command.

vmcli -f backup -I backupObjectList -D STANDARD --vmbackupmode SNAPSHOT_INCL_MEM
--name backup_test01 --description desc.txt --runnow

This command backs up all VMs that are on the datastore TG_fcmcli2_XIV_2.

Back up a set of virtual machines independent of their location

Problem

You want to regularly back up a fixed set of virtual machines regardless of their datastore location at the time of the backup.

Using wildcards in manually created backup infiles gives you flexibility to manage the dynamics in your VMware vSphere environment. However, this approach still makes assumptions about datastores and the virtual machines that they contain. Additionally, if the names of virtual machines and datastores do not follow a well-defined naming schema, it can be difficult to find appropriate wildcard expressions. You also can only exclude virtual machines by name, not include them.

Solution

The Perl script in Listing 2 creates a backup infile that is based on a set of virtual machines specified by name in an input file. The script automatically obtains the instance UUIDs of these virtual machines and determines the URLs of the datastores that hold their corresponding files. This information is then written to a file, formatted according to the backup infile specification.

Listing 2. Create a backup infile for a set of virtual machines identified by name
#!/usr/bin/perl -w

use strict;
use warnings;
use VMware::VIRuntime;

Opts::parse();
Opts::validate();

# Connect to vCenter server and implicitly evaluate environment variables
# and CLI arguments
Util::connect();

createBackupInfileFromVMnames("infile.txt", "backupinfile.txt");

Util::disconnect();

sub createBackupInfileFromVMnames {

   my ($filename) = $_[0];
   my ($filename_out) = $_[1];
   my @linebuffer;
   my %DatastoreMap;

   open INFILE, $filename or die $!;
   open BACKUP_INFILE, ">", $filename_out or die $!;
   
   while(my $line = <INFILE>) {
      chomp($line);
      if($line ne "") {
         
         my $vmref = Vim::find_entity_view(view_type => 'VirtualMachine',
                                                             filter => {'name' => $line});
         if($vmref) {
            print "Adding virtual machine with name: $line to backup infile\n";
            push(@linebuffer, "# vmname: " . $vmref->name . "\n");
            push(@linebuffer, "vminstanceuuid:" . $vmref->config->instanceUuid . "\n");

            my $dsrefs = $vmref->datastore;
            foreach my $dsmoref (@$dsrefs) {
                  my $ds = Vim::get_view(mo_ref => $dsmoref);
               $DatastoreMap{$ds->info->url} = $ds->name;
            }     
         }
         else {
            print "Error: Virtual machine with name <$line> not found!\n";
         }  
      }
   }

   # print datastores to file
   foreach my $key (keys %DatastoreMap) {
      print BACKUP_INFILE "# datastorename: $DatastoreMap{$key}\n";
      print BACKUP_INFILE "datastoreurl:$key\n";
   }

   print BACKUP_INFILE "\n";

   # print virtual machines to file
   while( @linebuffer) {
       print BACKUP_INFILE (shift(@linebuffer));
   }

   close(INFILE);
   close(BACKUP_INFILE);
}

The script reads the names of the virtual machines from the infile.txt file, line by line. For each virtual machine, it uses the vSphere SDK for Perl to query the virtual machine's instance UUID and writes the corresponding specifications to an internal buffer. It also queries the URLs of the datastores that hold the files of the virtual machine and stores them into an associative array for later use. After all the virtual machines are processed, the URLs of all collected datastores are written to the output file. Then, the collected virtual machine specifications are written from the internal buffer to the file. The resulting backupinfile.txt file can then be used as an infile on a backup command, as in,

vmcli -f backup -I backupinfile.txt

Example infile.txt:

VM_1
VM_2
VM_3

Resulting backupinfile.txt:

# datastorename: nextra1
datastoreurl:sanfs://vmfs_uuid:4d36a1e3-b46cf8b7-daab-e41f136af7c8/
# datastorename: nextra2
datastoreurl:sanfs://vmfs_uuid:4d36a282-3ea7d4c3-746c-e41f136af7c8/
# vmname: VM_1
vminstanceuuid:5004306d-f75e-7e5c-86d2-18672f4f62ea
# vmname: VM_2
vminstanceuuid:5004b54f-2590-33a0-df3e-527ddf4e6172
# vmname: VM_3
vminstanceuuid:5004c387-5e24-b0cd-39e8-5f6fe02a5e1d

Disaster recovery

Problem

In the unfortunate case of a disaster, you need to restore your virtual machines and their data as quickly as possible. Assuming a backup strategy that partitions the environment into different tiers and levels of protection, it usually means that you must restore a number of backups. The backups are restored sequentially until you get your complete environment recovered and running again. This can turn into a cumbersome procedure if you first need to figure out which backup contains the most recent backup for a given virtual machine. The assumption here is that the backup strategy covers multiple backup tasks that run on different schedules or are manually triggered.

Solution

A small Perl script can automatically determine the last available backup for each virtual machine in your datacenter or datacenters and create the necessary restore infiles. The example in Listing 3 shows a script that creates restore infiles based on datacenters provided as command-line arguments.

Listing 3. Automatic creation of restore infiles for disaster recovery
#!/usr/bin/perl -w

use strict;
use warnings;

foreach (@ARGV) {
   createRestoreInfileFromDataCenterName($_, "restoreinfile");
}

sub createRestoreInfileFromDataCenterName {

   my ($datacenter) = $_[0];
   my ($filename_out_prefix) = $_[1];
   my $inqdcinfile="/tmp/tmpinqdcinfile.txt";

   my %VMBackupMap;
   my %VMuuid2nameMap;

   chomp($datacenter);
   # write name of datacenter to temporary infile:
   open TMP_INQDCINFILE, ">", $inqdcinfile or die $!;       
   print TMP_INQDCINFILE "datacenter:$datacenter\n";
   close (TMP_INQDCINFILE);

   print "Inquiring available backups for datacenter: $datacenter... \n";

   # inquire available backups for given datacenter
   my $inquireoutput=`/opt/tivoli/tsm/tdpvmware/common/scripts/vmcli                     
                                                              -f inquire -I $inqdcinfile`;
   
   my $lastBID;
   my $lastBackupDate;
   my $lastVMinstanceUUID;
         
   # parse output of inquire command, a separate file is written for each Backup ID:
   my @inquireoutputlines = split /\n/, $inquireoutput;

   foreach my $inquireline (@inquireoutputlines) {
      if ($inquireline =~ /^#CHILD backupid/) {
         $lastBID = substr $inquireline, index($inquireline, ':')+1;
      }
      if ($inquireline =~ /^#CHILD backupdate/) {
         $lastBackupDate = substr $inquireline, index($inquireline, ':')+1;
      }
      if ($inquireline =~ /^#CHILD vminstanceuuid/) {
         $lastVMinstanceUUID = substr $inquireline, index($inquireline, ':')+1;
      }
      if ($inquireline =~ /^#CHILD vmname/) {
         # The vmname is last after BID, backup date, instance UUID so now it is complete
         my $vmname = substr $inquireline, index($inquireline, ':')+1;
         print "Found backup for VM=$vmname with BID=$lastBID from $lastBackupDate\n";
         # for each virtual machine, store the BID of the most recent backup available:
         if($VMBackupMap{$lastVMinstanceUUID}) {
            my $currentValue = $VMBackupMap{$lastVMinstanceUUID};
            my $currentDate = substr $currentValue, index($currentValue, ':')+1 ;
            if($currentDate < $lastBackupDate) {
               $VMBackupMap{$lastVMinstanceUUID} = $lastBID . ":" . $lastBackupDate;
               print "Found newer backup for VM '$vmname' with
                                                    BID=$lastBID from $lastBackupDate\n";
            }
         }
         else {                     
            print "Found backup for VM '$vmname', BID=$lastBID from $lastBackupDate\n";
            $VMBackupMap{$lastVMinstanceUUID} = $lastBID . ":" . $lastBackupDate;
            $VMuuid2nameMap{$lastVMinstanceUUID} = $vmname;
         }
      }
   }

   if(keys( %VMBackupMap ))
   {
      print "\n--------------------------------- \n\n";

      # Finally, create one infile per BID:
      foreach my $key (keys %VMBackupMap) {
         my $BID = substr $VMBackupMap{$key}, 0, index( $VMBackupMap{$key}, ':');
         my $filename_out = $filename_out_prefix . "_" . "$BID.txt";
         print "Adding VM '$VMuuid2nameMap{$key}' from datacenter '$datacenter'
                                                              to file '$filename_out'.\n";
         open RESTORE_INFILE, ">>", $filename_out or die $!;
         print RESTORE_INFILE "# vmname: $VMuuid2nameMap{$key} datacenter: $datacenter\n";
         print RESTORE_INFILE "backupid:$BID vminstanceuuid:$key\n";
         close(RESTORE_INFILE);
      }
   }
   else
   {
      print "No backups found for datacenter $datacenter.\n\n";
   }
}

The script takes the names of the VMware vSphere datacenters that you want to restore as command-line arguments (space separated). For each datacenter, the script performs a vmcli inquire command to retrieve all the available backups for the datacenter. Because this vmcli inquire command requires an infile, the script first writes the name of the current datacenter to a temporary file that is then used as the infile. The console output of the inquire command is parsed, collecting backup ID (BID), backup date, name, and instance UUID for each virtual machine. There can be multiple occurrences for the same virtual machine because it can be contained in multiple backups. So the script determines the most recent backup for each virtual machine, storing its BID and date in the VMBackupMap associative array. After the console output is parsed, the associative array holds a single entry for each virtual machine including the corresponding BID of its most recent backup. The content of the map is finally written to the restore infiles. Because the vmcli does not support restoring multiple backups in a single run, multiple files are written, one for each BID. Note that the lines are appended to the files which means that running the script multiple times results in redundant content. To get clean results, delete the files before you start the script.

Each restore infile created by the script contains one or more sets of lines that have the following format:

# vmname: vmname datacenter: datacentername
backupid:BID vminstanceuuid:instanceUUID

The file names of the restore infiles created by the script follow this pattern: restoreinfile_BID.txt

After the files are created you can review and revise them, if applicable, before using them to restore your environment with vmcli restore commands.

Send email reports for backups

Problem

Tivoli Storage FlashCopy Manager for VMware does not provide the capability to send email notification when a backup task completes to indicate whether the backup was successful or not.

Solution

Write a simple shell script that captures the output from the vmcli backup command and sends this information to the system administrator as email. In the example in Listing 4, the script reruns an existing backup task so we hardcoded the ID of the task in the script.

Listing 4. Sending email reports automatically
#!/bin/bash

#temporary text file which stores the content of the mail
DAILY=mail.txt

# task id of the backup task we want to use.
TSKID=3619
echo "Starting task with task id $TSKID!!!" >>$DAILY 2>&1
/opt/tivoli/tsm/tdpvmware/common/scripts/vmcli -f backup -T $TSKID --runnow >>$DAILY 2>>&1
if test $? -ne 0; then
  echo "!!!!!!!!!!!!!!!!Backup has failed.!!!!!!!!!!!!!!!!!!"  >>$DAILY
else
  echo "!!!!!!!!!!!!!!!! Backup successful. ||||||||||||||||"  >>$DAILY
fi
mail -s "daily backup notification" "admin@company.com" <$DAILY

To schedule this script to run every day at 2:00 am, you could use the crontab of the tdpvmware user.

Listing 5. Crontab entry for email reporting
$> crontab -e
# Minute Hour  Day of Month      Month         Day of Week       Command    
# (0-59) (0-23)   (1-31)   (1-12 or Jan-Dec)  (0-6 or Sun-Sat)          
    0       2        *            *              *         /home/tdpvmware/dailyBackup.sh

Conclusion

Using IBM Tivoli Storage FlashCopy Manager for VMware with the vCenter GUI plug-in is easy and convenient and in many cases provides sufficient flexibility for backing up and restoring your VMware vSphere environment. However, in highly dynamic VMware vSphere environments with virtual machines being automatically migrated back and forth and datastores being frequently restructured, the vCenter GUI plug-in has its limitations.

Using the Tivoli Storage FlashCopy Manager for VMware command-line interface in combination with simple scripting gives you more flexibility and helps you automate routine tasks. The intent of this article was to give you an understanding of the basic concepts of the vmcli, the vSphere inventory model, and the vSphere SDK for Perl. Applying this knowledge opens up possibilities far beyond the examples demonstrated in this article.


Download

DescriptionNameSize
Sample shell and Perl scripts for this articlesm-flashcopy-vmware-automation.zip3KB

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Tivoli (service management) on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Tivoli (service management), Tivoli
ArticleID=847645
ArticleTitle=Learn advanced automation concepts with FlashCopy Manager for VMware
publish-date=11262012