Answer

The MEMORYEFFICIENTBACKUP option changes the manner in which the Tivoli Storage Manager client processes files for a backup there are 3 possible settings: NO, YES, and DISKCACHEMETHOD.

When the option is set to No (the default setting), the client builds a list in local memory of each directory and file on the client machine that fits the backup criteria for the entire filesystem. It then checks each file and directory against a list received from the Tivoli Storage Manager server of previously backed up files and directories to determine if the file needs to be included in the incremental backup.

When the option is set to YES the client does the same process, but it does only one directory at a time, instead of one filesystem at a time, using much less memory.

When the option is set to DISKCACHEMETHOD the backup works like using NO, except the client uses hard disk space on the client machine for the sorting, this option can help in situations where memory is constrained or where a single directory has millions of files, and even YES can run out of memory. This option can use a significant amount of disk space (as much as 5gb per 1 million files and directories).

For the purposes of this document a normal client is one in which there is a directory structure with files fairly evenly distributed through out the structure; an abnormal client is one on which the majority of files are concentrated in only a few large directories

1. In a normal client with no memory constraints that would normally complete its backup without problems, setting MEMORYEFFICIENTBACKUP YES may cause a slight increase in overall time for the backup over the default of NO. Using DISKCACHEMETHOD on this client may slightly slow the backup as well but will use significantly less memory than NO.

2. On a normal client that may experience a lack of memory resources, setting MEMORYEFFICIENTBACKUP YES should let the client backup complete without error. This is especially useful on older/slower/under-powered hardware. Using DISKCACHEMETHOD on this client will again use much less memory, and can be used if free disk space is not an issue on the client.

3. In a normal client with a very large number of directories/files (this number is OS specific and often influenced by hardware) setting MEMORYEFFICIENTBACKUP YES should also allow the client to complete its backups without error. It will also avoid problems with IDLETIMEOUTs with the Tivoli Storage Manager server because of the extended set up time that may occur on clients of this type even if the memory/paging space is sufficient to backup without the MEMORYEFFICIENTBACKUP YES. Using DISKCACHEMETHOD on this client will also allow the backup with the use of less memory but will use disk space and if disk space is not sufficient it may fail.

NOTE: As indicated in point 3 above setting MEMORYEFFICIENTBACKUP YES will cause files to begin being backed up to the Tivoli Storage Manager server sooner than with it set to NO. On a small client this is likely not to be noticed, but on a very large system (as referenced in #3 above) it can be a significant time difference.

In the case of a client which has directories with a large number of files in a single directory, MEMORYEFFICIENTBACKUP is much less likely to have an affect. If sufficient memory resources do not exist to create the backup lists then the backup will still fail. In these cases using DISKCACHEMETHOD should help if disk space is sufficient. If there is not sufficient disk space to use DISKCACHEMETHOD then only adding additional memory/paging space and/or adding a subdirectory structure will allow a backup to complete. A good indication that a directory is a candidate for this type of failure is if the results of an 'ls' command take a long time to display, especially on UNIX systems.