How to upload large AIX snap Files

Answer

Customers may need to split large files containing snap data in order to upload this data to IBM. There may be constraints on their firewalls as to how large files can be or they may run into transmission issues. This technote discusses how to resolve the problem.

The snapsplit command

AIX has a shell script called snapsplit. The script can be used to split and to join files. The script splits a file using the "split" command and joins files using the "cat" command. Because of this, it is rarely necessary to use snapsplit to join files as you can use "cat" directly. The main benefit is in splitting files.

Splitting a file

The snapsplit works on an existing snap file. For this discussion we will use a file called snap.pax.Z. Please bear in mind that you will need enough file system space to hold the original snap.pax.Z file and the resulting split files.

# ls -l snap.pax.Z
-rw------- 1 root system 197661449 May 21 14:49 snap.pax.Z

If no options are used to snapsplit it will create split files in 1MB chunks. For the file shown above, this would create twenty files. The common way to use snapsplit is to specify the size of the chunks you want using the "-s" option in order to control the size of the output files. Using the above file to make 50 MB files:

# snapsplit -s 50 snap.pax.Z

If the file is named something other than snap.pax.Z use the "-f" option to specify the file you want. For example:

# snapsplit -s 50 -f mysnap.pax.Z

Looking at the output from the original command:

# ls -l snap*
-rw-r--r-- 1 root system 52428800 May 21 15:06 snap.ernie.052115150600.pax.Zaa
-rw-r--r-- 1 root system 52428800 May 21 15:06 snap.ernie.052115150600.pax.Zab
-rw-r--r-- 1 root system 52428800 May 21 15:06 snap.ernie.052115150600.pax.Zac
-rw-r--r-- 1 root system 40375049 May 21 15:06 snap.ernie.052115150600.pax.Zad
-rw------- 1 root system 197661449 May 21 14:49 snap.pax.Z

The snap.pax.Z file is split into three files of equal length.  The fourth file is the remainder. The default output from snapsplit will put the host and machine names in the resulting files.

The above example is for demonstration purposes only. Customers should split files into the largest possible chunks they can upload without running into firewall or transmissions issues. Most customers will use an option of 1024 or 2048 to split files into 1 or 2GB chunks.

Renaming the split files

The snapsplit program creates files that are named in lexicographic order. It is important to preserve this order if  renaming the files with the PMR or SalesForce number prefixed prior to uploading them. Assuming the PMR number is 12345,678,000, the files would be named as follows:

snap.ernie.052115150600.pax.Zaa -> 12345.678.000.pax.Zaa
snap.ernie.052115150600.pax.Zab -> 12345.678.000.pax.Zab
snap.ernie.052115150600.pax.Zac -> 12345.678.000.pax.Zac
snap.ernie.052115150600.pax.Zad -> 12345.678.000.pax.Zad

The files can then be uploaded to either testcase.software.ibm.com or to ecurep.ibm.com, using either sftp or https. Please be aware that most browsers have a 2GB limit for file sizes. If a file needs to be resent retain the lexicographic order as much as possible; for example, 12345.678.000.pax.Zac can be resent as 12345.678.000.pax.2.Zad.

Please make known to support personnel the size and names of the files being transmitted. This is to make sure of the integrity of what is being received and helps speed up the process of reassembling and extracting the data.

Joining split files

Split files uploaded to the ftp servers are not joined by tools on those servers. This means each file needs to downloaded to the server where analysis will be done. All transfers must be in binary mode and again, lexicographic order is important. Ensure that all files except the last are of equal size. Files of unequal length indicate a transmission problem and files must be resent as noted above.

To join the files all that is needed is to use the "cat" command:

$ cat 12345.678.000.pax.Zaa 12345.678.000.pax.Zab 12345.678.000.pax.Zac 12345.678.000.pax.Zad > snap.pax.Z

The snap file can then be extracted for examination.

Further references:

The snapsplit man page

A FAQ on snapsplit: http://www-01.ibm.com/support/docview.wss?uid=swg21569990

Submitting a Testcase for Dump Analysis: http://www-01.ibm.com/support/docview.wss?uid=isg3T1000654