IBM PureApplication System backup and restore, Part 3: Workload backup

IBM® PureApplication® System is a cloud computing system-in-a-box with integrated hardware and software for deploying and executing workloads in a cloud. As with other systems, administrators should maintain a current set of backups of the system and its workloads. While the system backup feature in PureApplication System captures the system’s configuration, other backup techniques are needed if you need to recreate or restore cloud, workload, or data components at a more granular level. This article explains how to backup workload components, and also how to add configuration to patterns so that their pattern instances will backup their application data. This content is part of the IBM WebSphere Developer Technical Journal.

Bobby Woolf, PureApplication System Consultant, IBM

Bobby Woolf is a Technical Account Manager for the PureStart program that helps customers get started with PureApplication System. He's spent years helping customers develop business applications leveraging cloud technologies, service-oriented architecture, event-driven architecture, and application integration. He is the author of Exploring IBM SOA Technology and Practice, and a co-author of Enterprise Integration Patterns and The Design Patterns Smalltalk Companion.


developerWorks Master author
        level

Tamiko W Brown (twbrown@us.ibm.com), Software Test Engineer , IBM

Tamiko Brown is a Software Test Engineer with over 13 years experience in the IT industry. Her focus currently is software verification testing for PureApplication System in the areas of system backup and restore and user security.



09 April 2014

Also available in Japanese

Introduction

IBM PureApplication System (W1500 and W1700 v1.0 and v1.1) is a cloud computing system-in-a-box, with integrated hardware and software to deploy and execute workloads in a cloud — in other words, everything an enterprise data center needs to add a private cloud environment. As with other systems, administrators should maintain a current set of backups of the system and its workloads.

The system backup feature in PureApplication System v1.1 captures the system’s configuration, but other backup techniques are needed to enable you to more granularly recreate the cloud configuration, restore workload components, and restore application data.

This series of articles cover the different aspects of backup and restore on PureApplication System:

  • Part 1: Recovery scenarios An overview of the issues involved with backup and restore on PureApplication System and the recovery scenarios for which you should be prepared.
  • Part 2: System backup Details for system administrators on how to configure system backup, how to use restore, and how backup storage is arranged.
  • Part 3: Workload backup – Details for workload administrators on how to back up the workload catalog (patterns, and so on) and the data in running applications.
  • Part 4: Configuration management – Details for (primarily) workload administrators on how to use backup components to manage their versions and copy them from one system to another.

This article, Part 3 explains how to backup workload components, and also how to add configuration to patterns so that their pattern instances will backup their application data. By following these practices, workload administrators can ensure that they always have a current backup of their patterns and the application data within the pattern instance needed for restore.

This article assumes that the user performing these steps has been granted the Workload resources administration (Full permission) security role. See “Managing administrative access in IBM PureApplication System” in References. This role is the easiest way — and depending on how access has been authorized on individual components, might be the only way — to perform these export and editing tasks across all patterns.


Recovery scenarios

Part 1 described the various recovery scenarios for PureApplication System. This article addresses three of them:

Scenario 1: Restore application data

  • Context: The system and workloads are available, but the application resources are corrupted.
  • Procedure: Use the same program that was used to make the file system or application database backup to restore the backup.
  • Example: Backup an application’s database (schema and data). Then, when the pattern instance is still running correctly — but the application is not because the data in its database is corrupted, restore the database to make the application start running correctly again.

Scenario 2: Restore workloads selectively

  • Context: The workload components are intact, but a workload has become corrupted.
  • Procedure: Delete the workload, redeploy the pattern, and restore its application data backup.
  • Example: When a server (such as those in an IBM WebSphere® Application Server cell or an IBM DB2® instance) won’t run properly (perhaps because an emergency fix failed to apply successfully), redeploy the pattern. The new pattern instance contains a new set of servers. Then, restore the application data.

Scenario 3: Restore system or workload components selectively

  • Context: The system is functional, but system or workload components are corrupted.
  • Procedure — option 1: Use the CLI to delete the components and import their backup.
  • Procedure — option 2: Use the system restore functionality to import workload components that were included with the system backup.
  • Example: Backup all of the custom patterns. Then, when a pattern developer changes a pattern for the worse, or the pattern librarian deletes one that is still needed, restore the pattern.

Thus these procedures help prepare you in case corruption occurs in your patterns, your workloads (pattern instances), or your workloads’ data (application data).


Component export

System backup exports the system’s own configuration and internals. As shown in Part 2, the component specific backup can also be used to export some types of system components. A more customizable approach, however, is to export individual workload components.

Command line interface (CLI) scripts should be used to export key workload components regularly, especially when configurations are edited.

The process is fairly simple:

  1. To backup a component, use a CLI script to export it.
  2. To restore a corrupt component, delete the corrupt component, and then use a CLI script to import the backup.

CLI commands to export and import components transmit the data over the management network, specified as CUSTMGMT on the Customer Network Configuration page (System Console > System > Customer Network Configuration). The file system specified in the commands must be accessible from this CUSTMGMT VLAN.

Below are some examples of how to export components. The CLI client includes a directory with a set of sample scripts, pure.cli\samples. The examples shown here demonstrate how to use two of those sample scripts ( exportPattern.py and importPatterns.py) to make backups of components and restore those.

Example: Virtual System Pattern export

A virtual system pattern can be exported using a CLI sample script exportPattern.py. The CLI command line to run the script looks like this:

Listing 1.
C:\pure.cli\bin>pure -h <pureapp_host> -u <username> -p <password> -a -f
exportPattern.py --pattern "ER_SETTINGS" --target <export_dir>\ER_SETTINGS032013.tgz

This command uses the script to export the pattern named “ER_SETTINGS” to the file “ER_SETTINGS032013.tgz”. To run the command, you need to fill in these details specific to your environment:

  • pureapp_host: The hostname for your system
  • username: A user with read access to the pattern
  • password: The user’s password
  • export_dir: A directory on the same host as the CLI client

Ultimately, the script uses the Pattern object’s save method to perform the export. When run for the data in this pattern, the script generates this output:

Listing 2.
Created a temporary directory "\temp\exporti_rf_h" for export
Exporting pattern "ER_SETTINGS" ...
Exporting script package "ER_Settings" to "script_packages/CSC_ER_Settings_Test.zip" ...
Exporting script package "ER_JVM_Policy" to "script_packages/CSC_ER_JVM_Policy.zip" ...
Exporting add-on "Default add disk" to "add_ons/defaultadddisk.zip" ...
Export pattern "ER_SETTINGS" successfully

Example: Virtual System Pattern import

A virtual system pattern can be imported using a CLI sample script importPatterns.py. The CLI command line to run the script looks like this:

Listing 3.
C\pure.cli\bin>pure -h <pureapp_host> -u <username> -p <password> -a -f
importPatterns.py -s <export_dir>\ER_SETTINGS032013.tgz
  • pureapp_host: The hostname for your system
  • username: A user with write access to the pattern
  • password: The user’s password
  • export_dir: A directory on the same host as the CLI client

Ultimately, the script uses the Pattern object’s load method to perform the import.

When run for the data in this pattern, the script generates this output:

Listing 4.

Click to see code listing

Listing 4.

Importing script package "ER_Settings" from "\script_packages/CSC_ER_Settings_Test.zip"
                    ...
                    Importing script package "ER_JVM_Policy" from "\script_packages/CSC_ER_JVM_Policy.zip"
                    ...
                    Importing pattern "ER_SETTINGS" ...
                    Import pattern "ER_SETTINGS" successfully.

Example: Virtual Application Pattern export

A virtual application pattern can be exported using the CLI, specifically the Application object’s download method. That method would be used in a CLI command or script. For example, the command to download the pattern TWB_DTL to the file TWBDTLApp.zip looks like this:

Listing 5.
deployer.applications['TWB_DTL'][0].download("<export_dir>/TWBDTLApp.zip")

This command will export the VApp Pattern as one zip file, including the virtual application model .json files and the archives (for example, .war or .ear file for IBM WebSphere Application Server, .sql file for an IBM DB2 component, and so on).

Example: Virtual Application Pattern import

A virtual application pattern can be imported using the CLI, specifically the Applications object’s create method. For example, the command to upload the VApp patterns and components in the file TWBDTLApp.zip looks like this:

Listing 6.
deployer.applications.create("<export_dir>/TWBDTLApp.zip")

When run for the data in this file, the command generates this output:

Listing 7.
{
                    "access_rights": (nested object),
                    "acl": (nested object),
                    "app_id": "a-f4bcda89-a996-4398-bebb-2bceaff12379",
                    "app_name": "TWB_DTL",
                    "app_type": "application",
                    "artifacts": (nested object),
                    "content_md5": "0b0e5dd0af25417eaa569d9f424ab56d",
                    "content_type": "application/json",
                    "create_time": "2014-03-07T15:06:49Z",
                    "creator": "admin",
                    "last_modified": "2014-03-07T15:06:49Z",
                    "last_modifier": "admin",
                    "patterntype": "webapp",
                    "version": "2.0"
                    }

Examples using the integrated console

The system’s integrated console includes buttons to import and export virtual application patterns, as shown in Figure 1. To open the Virtual Application Patterns page, navigate to Workload Console > Patterns > Virtual Applications. To export a pattern, select it first.

Figure 1. Virtual application pattern import and export buttons
Virtual application pattern import and export buttons

Click to see larger image

Figure 1. Virtual application pattern import and export buttons

Virtual application pattern import and export buttons

The Import button opens a file selection dialog. The Export button sends a file to your web browser to download; your browser will typically prompt you with a file save dialog.


Application data backup

If an application becomes corrupted, it can be recovered simply enough: Delete the pattern instance and deploy the pattern again. However, most applications have state, and any state that is part of the pattern instance has to be recovered as well.

The approach for backing up an application running as a workload is very similar to that for backing up the application running on traditional hardware. You need a backup solution such as IBM Tivoli® Storage Manager. For each application, you install an agent for the backup solution on the same server as the middleware that contains the data. The agent must be configured in two aspects:

  • Backup server: The agent must know how to connect to the backup server to which it will be transmitting backups. All agents typically use the same backup server.
  • Target data: The agent must know how to gather the data from the middleware so that it can be backed up. This is different for each middleware product and can depend on what data you decide should be backed up.

Configuring DBaaS backup

Configuring a pattern to backup its data is seen easily in the Database as a Service (DBaaS) virtual application pattern (go to Workload Console > Instances > Databases). It includes a system plug-in called tsm that includes a Tivoli Storage Manager backup agent configured to backup the database. You simply need to configure it to connect to your Tivoli Storage Manager server using the properties dialog shown in Figure 2 (go to Workload Console > Cloud > System Plug-ins and select the plug-in named tsm 1.1.0.7). This configuration is shared by all DBaaS instances, so all of the databases will be backed up to the same server.

Figure 2. Configuration dialog for the database as a service backup agent
Configuration dialog for the database as a service backup agent

Click to see larger image

Figure 2. Configuration dialog for the database as a service backup agent

Configuration dialog for the database as a service backup agent

To enable automatic database backups, select a database (go to Workload Console > Instances > Databases) and press Manage to open the Database Service Console. In the console, with DB2 selected, expand Fundamental > Automatic scheduled database backup and, as shown in Figure 3, set the frequency to daily or weekly. This frequency can be set differently for each database.

Figure 3. Schedule automatic database backups
Schedule automatic database backups

Click to see larger image

Figure 3. Schedule automatic database backups

Schedule automatic database backups

Configuring virtual system instance backup

For the database part in a virtual system pattern (like configuring the DBaaS pattern), you can use a script package to install a backup agent and configure it with what to back up and where the server is. You can also install the agent on other virtual machines to back up other directories like configuration settings, log files, and so on.

The download file included with this article, installTSM.zip, shows the contents of a script package to install a TSM client. The main script is installTSM.sh, a UNIX® shell script. It installs the package for the Tivoli Storage Manager client, updates the configuration files for the client and its scheduler, and schedules it to run automatically.

Backup image management

Your backup solution might mandate various conventions in the backup image that can be problematic for cloud environments. These conventions fit two main categories:

  • Image naming convention

    You should name your backup image after the application that was the source for the data. Typically, the pattern and pattern instance are also named after the application, so likewise name the backup image after the application, perhaps with a version indicator as well. If you back up separate parts of the application into separate backup images, the images’ names should include the name of the part to make them unique.

    For example, you might have an application called Stock Trader whose currently deployed version is 1.1. You might backup its databases and its transaction logs separately. In this case, the image names for the backup might be:

    • Stock Trader v1.1 databases
    • Stock Trader v1.1 translogs

    Presumably, the backup solution would also add the timestamp the backup was taken to each backup image name so that you would see this pair of image names for each time the backup was performed. If not, you should specify the timestamp in the image filenames. For example, for the set of Stock Trader v1.1 backups made on January 31, 2014, the image names could be:

    • Stock Trader v1.1 databases 20140131
    • Stock Trader v1.1 translogs 20140131

    Some backup solutions do not use such descriptive names. Rather, they name the backup image after the IP address or hostname of the virtual machine (VM) that contained the source data. This practice is not very cloud friendly. The workload may well be deleted, after which you’ll have no record of what was deployed to that IP address or hostname when the backup was made. You’ll need to make a note of the application that was running in that VM so you’ll know what data was stored in that backup image.

  • Restore target

    You need to be able to configure the restore process so that it specifies the application instance into which the backup image’s data will be restored. How this is done depends on whether the restore is run from the backup solution’s server or its agent.

    • Restore from backup agent: The backup solution can run the restore process in the backup agent running in the application. (More specifically, the agent will be running in the application’s VM where the data needs to be restored.) Configure the restore process with the name of the backup image to restore and the agent will know where to restore the data to.
    • Restore from the backup server: The backup solution can run the restore process in the backup server shared by all applications. Configure the restore process with the name of the backup image to restore and the IP addresses/hostnames of the VMs to restore the data to.

    Some backup solutions embed within the backup image the IP address/hostname of the VM that was the source for the data. The assumption is that the image will be restored to the same IP address/hostname. This restore target is not configurable in the restore process because the process automatically restores to the target embedded in the image. Again, this practice is not very cloud friendly. Commonly, especially in cloud computing, a backup image taken from one set of infrastructure needs to be restored in a different set of infrastructure, and the second infrastructure is likely to have different IP addresses/hostnames than the first.

    If your backup solution embeds IP addresses/hostnames, look into how to use it to backup from one application environment (say Development) and restore into another application environment (such as Test). These two environments will be provisioned with different sets of IP addresses/hostnames, so the backup is taken from one set and restored to another set. To do this, a solution might require a process something like this:

    1. Make the backup image from the first set.
    2. Clone the image for the second set.
    3. Restore the cloned backup image.

Whatever process your backup solution requires to restore to a different environment, you’ll need to use that process to restore your backup to a new pattern instance.

Backup network

You might wish to isolate the application data backup network traffic on its own VLAN. This won’t necessarily conserve bandwidth, but it will logically separate the backup traffic for privacy from the network traffic for standard application operations. This technique assumes that the backup server host is connected only to the backup data VLAN and not to the VLANs used for standard application data. Otherwise, backup data could transmit over either VLAN, which defeats the purpose of a separate VLAN just for backups.

To use a VLAN for application data backups, each virtual machine (VM) with data to backup must be configured with a separate virtual network adapter configured with an IP address for the backup VLAN. Then configure the backup agent to connect to the backup server. When the backup runs, the agent will need to use to use the adapter, also called a network interface card (NIC), to connect to the backup server.

For example, in a virtual system pattern, consider a part that will have a backup agent where that agent should connect to the server using the backup VLAN. Edit the pattern. For each part that will have a backup agent, add the Default add NIC add-on, as shown in Figure 4, which configures the VM with an additional network adapter.

Figure 4. Adding a NIC to a virtual system pattern's part
Adding a NIC to a virtual system pattern's part

Click to see larger image

Figure 4. Adding a NIC to a virtual system pattern's part

Adding a NIC to a virtual system pattern's part

In the cloud group that will need to use the backup VLAN, add an IP group configured for the backup VLAN with some IP addresses for using it.

When you deploy the pattern, you’ll see not one but two network interfaces listed in the dialog to configure the part’s property values (Figure 5), each of which needs an IP group (assuming the environment profile specifies that IP addresses are provided by IP group). For network interface 1 (which is the second NIC, the one you added to the part), specify the IP group for the backup network.

Figure 5. Configure part deployment dialog
Configure part deployment dialog

With this configuration, the deployed VM will contain two logical network adapters:

  • NIC 0, which connects to the application data VLAN; this is typical for every user workload VM on the system.
  • NIC 1, which connects to the backup data VLAN; this is the new feature to enable the VM to use a separate VLAN for backups.

Configure the backup agent with the hostname for the backup server. Since the backup agent can only reach the backup server via NIC 1’s VLAN, that’s the VLAN that will connect the agent to the server and therefore transmit the backup data.


Conclusion

This article explained what is necessary to backup a system’s patterns and the application data in its workloads. It has addressed the two pattern types in IBM PureApplication System, virtual application patterns and virtual system patterns. By following these procedures, your workloads will be backed up and you will be prepared to recover them if and when needed.


Acknowledgements

The authors thank these IBMers for their help with this article: George Owen, Tom Alcott, Paul Rockwood, and Allan Weeks.


Download

DescriptionNameSize
Sample codeinstallTSM.zip2 KB

Resources

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into WebSphere on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere, Cloud computing
ArticleID=967937
ArticleTitle=IBM PureApplication System backup and restore, Part 3: Workload backup
publish-date=04092014