Optimize your BAR Performance using parallel backups with Informix Dynamic Server

For optimum database backup and restore performance for your IBM® Informix® Dynamic Server (IDS) database, take advantage of parallelization with the ON-Bar utility. This article introduces ON-Bar, describes the parallel backup and restore sequence, and includes a discussion of configuration parameters and dbspace sizing to achieve your performance objectives.

Share:

Shweta Sinha, System Software Engineer, IBM

Shweta Sinha photoShweta Sinha works in the IBM Software Lab, India, as a QA-Engineer in the IDSQA team across various IDS products.



16 November 2006

Also available in Russian

A brief introduction to ON-Bar

The ON-Bar utility offers a robust and scalable backup solution for Informix Dynamic Server databases. It reads the data from the database and then communicates directly with the storage manager to store the data to tape, as shown in Figure 1. This storage manager keeps track of where that data went, and how long it needs to be retained. Examples of storage managers are NetWorker by Legato and NetBackup by VERITAS.

Figure 1. On-Bar architecture
ON-Bar architecture

ON-Bar offers three different levels of backups:

  • Level 0, which backs up all data on the system
  • Level 1, which backs up all data that has changed since the last level 0
  • Level 2, which backs up all data that has been changed since the last level 1 or level 0 backup

These different levels can be used in conjunction with one another to shorten the backup window.

Parallel backup

Parallel backups allow the processing of several dbspaces ($ONCONFIG parameter BAR_MAX_BACKUP) simultaneously, thus reducing the overall backup time if the infrastructure is adequate. However, in the case of a restore, On-Bar relies on the logical logs to reach a point of consistency.

It is essential that you ensure your logical logs are backed up regularly if you plan to do parallel backups. Informix versions before IDS V10 did not prevent you from performing parallel backups without saving your logical logs. However, in the case of a restore, without the logical logs you will not be able to recover all of your data because the logical logs are needed to reach a point of consistency for backups performed in parallel mode.

Make sure that the $ONCONFIG parameter LTAPEDEV is set to some different path than /dev/null and the ALARMPROGRAM is capable of doing log backups with ON-Bar. If LTAPEDEV is set to /dev/null, a logical log will be marked as backed up as soon as a log switch occurs. The logical log will not be saved.

Parallel backup sequence in IDS

If you specify a parallel backup or restore, the onbar-driver (onbar_d) creates child onbar_d processes, as shown in Figure 2, that perform backup and restore operations. Each child processes one storage space, then returns. ON-Bar processes log files serially. If you specify a serial backup or restore, the onbar-driver performs the operation one object at a time. The onbar_d processes write status and error messages to the ON-Bar activity log and write information to the emergency boot file that is used in a cold restore.

Figure 2. On-Bar child processes
ON-Bar child processes

The workflow shown in Figure 3 depicts the ON-Bar backup sequence. When you issue a backup command, the onbar-driver builds a list of storage spaces and creates a backup session.

Figure 3. On-Bar workflow
ON-Bar workflow

In a parallel backup (if BAR_MAX_BACKUP is not set to 1), the onbar-driver starts one or more onbar_d child processes and assigns backup tasks to them. Each onbar_d child process backs up one storage space. Each onbar_d child disappears when the backup of its storage space is done. The onbar-driver keeps creating new children until all the storage spaces are backed up. Then the onbar-driver backs up the logical logs. If you specify a whole-system backup or set BAR_MAX_BACKUP to 1, the onbar_driver backs up the storage spaces and logical logs serially, and only 1 onbar_d child process will be created.

When the backup is complete, the onbar-driver determines whether an error occurred and returns the status in the ON-Bar activity log. After each object is backed up, information about it is added to the emergency boot file on the database server and to the sysutils database.

Parallel restore sequence on IDS

If BAR_MAX_BACKUP is set to 1, ON-Bar restores the storage spaces serially. If you did not perform a whole-system backup, you must use the onbar -r command to restore the data. While using the onbar -r command, ON-Bar restores the storage spaces in parallel and replays the logical logs once.

Figure 4. Restore workflow
Restore workflow

In a parallel restore (if BAR_MAX_BACKUP is not set to 1), the ON-Bar driver starts onbar_d child processes. The onbar_d processes transfer data between the storage manager and the database server until the warm restore is complete. Each onbar_d process processes one storage space.

Note:

In a serial restore, the onbar-driver restores the storage spaces one at a time. Then the onbar-driver performs the logical backup and restore.

After each object is restored, information about it is added to the sysutils database. For each storage space, ON-Bar restores the last level-0 backup, the level-1 backup (if it exists), and the level-2 backup (if it exists). After the physical restore is complete, ON-Bar backs up the logical logs to get the latest checkpoint and then restore the logical logs. This logical backup allows data to be restored up to the moment of failure.

If you set BAR_MAX_BACKUP to 0, the system creates as many ON-Bar processes as needed. The number of ON-Bar processes is limited only by the number of storage spaces or the amount of memory available to the database server, whichever is less. The amount of memory available is based on SHMTOTAL.

On-Bar performs the following calculation where N is the maximum number of ON-Bar processes that are allowed:

N = SHMTOTAL / (# transport buffers * size of transport buffers / 1024)

If SHMTOTAL is 0, BAR_MAX_BACKUP is reset to 1. If N is greater than BAR_MAX_BACKUP, ON-Bar uses the BAR_MAX_BACKUP value. Otherwise, ON-Bar starts N backup or restore processes.

Configuration parameters

In addition of regular ON-Bar configuration parameters, Dynamic Server uses the following parameters for parallel backup/restore:

Config. ParametersRangeOnconhfig.std valueEffect
BAR_MAX_BACKUP 0 = maximum number of processes allowed on system
1 = serial backup or restore
n = specified number of processes allotted
0Specifies the maximum number of parallel processes that are allowed for each onbar command
BAR_NB_XPORT_COUNT3 to unlimited 10Specifies the number of data buffers that each onbar_d process can use to exchange data with the database server. The value of this parameter affects onbar performance
BAR_RETRYBAR_ABORT(0), BAR_CONT(1), or n 1If set to BAR_ABORT, onbar aborts the backup or restore session when an error occurs for a storage space or logical log
If set to BAR_CONT, onbar aborts the backup or restore attempt for that particular storage space
If set to a specific number (n), onbar attempts to back up or restore this storage space or logical log the specified number of times before it gives up and moves on to the next one

Sizing of dbspaces for ON-Bar parallelization

To help in understanding the concept of sizing dbspaces to take advantage of parallelization with ONBAR utility, we'll look at the following example. Here, we will present a layout of dbspaces where ON-Bar will not be able to take advantage of parallelization. Later, we will present a layout of dbspaces which will be suitable for parallelization.

Suppose a customer has a requirement for creating an Informix instance with capacity 200 GB. Let us say, we will have dbspaces for:

  • rootdbs
  • llogdbs
  • plogdbs
  • tempdbs1
  • tempdbs2
  • database1dbs
  • userdbs1
  • userdbs2
  • ....
  • ....
  • userdbsN

How do we go about sizing these dbspaces? Here, we will present one layout of dbspaces where ON-Bar will not be able to take advantage of parallelization. Second, we will come up with a layout better suited for ON-Bar parallelization.

  1. Example with Large dbspaces:
    In this example, suppose we have,
    dbspaceSize
    userdbs140 GB
    userdbs240 GB
    userdbs320 GB
    With layout as above, when Onbar is backing up userdbs1, only one thread is associated with this backup. This will leave other Onbar threads idle.
  2. Sizing of dbspace to take advantage of ONBAR parallelization:
    dbspaceSize
    userdbs1a 8 GB
    userdbs1b 8 GB
    userdbs1c 8 GB
    userdbs1d 8 GB
    userdbs1e 8 GB
    userdbs2a 8 GB
    userdbs2b 8 GB
    userdbs2c 8 GB
    userdbs2d 8 GB
    userdbs2e 8 GB
    userdbs3a 8 GB
    userdbs3b 8 GB
    userdbs3c 8 GB
    userdbs3d 8 GB
    userdbs3e 8 GB

In this case, as shown in Figure 5, 40 GB dbspace is split across five dbspaces. Five On-Bar threads can potentially be backing up the 40 GB dbspace in parallel. Thereby, we will be taking advantage of On-Bar parallelization. Here, the limitation will be with respect to hardware capacity.

Figure 5. Parallel backup
Parallel backup

Similar advantages are gained when restoring dbspaces.

Conclusion

We have seen that the sizing of dbspaces is important with respect to ON-Bar. By properly sizing the dbspaces, you enable the ON-Bar utility to back up Informix instance faster since it uses multiple threads to back up the dbspaces. When you have large dbspaces, a single ON-Bar thread works to back up the dbspace, even though the other threads are idle.

Acknowledgement

I would like to thank Srinivasachar Prasad and Uday Kale for their support, guidance and ideas.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Information management on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Information Management
ArticleID=174734
ArticleTitle=Optimize your BAR Performance using parallel backups with Informix Dynamic Server
publish-date=11162006