Data compression

Content Manager OnDemand can compress report data by using several different data compression algorithms before storing the data in cache storage and archive storage. The compression ratio that Content Manager OnDemand can achieve has a significant impact on the amount of space required to store reports.

The compression ratios that Content Manager OnDemand can achieve vary widely depending on the type of data and the format of the data. You cannot always accurately estimate the compression ratio by simply examining the data. On average, you can expect to achieve between 2:1 and 15:1 compression for AFP documents and up to 30:1 compression for line data reports. Compression for AFP documents is based on the output data file produced by ACIF, and not the input file, which might be line data. When ACIF formats line data with a page definition, it might increase the size of the data by adding AFP controls for positioning text.

To properly estimate the amount of storage space required by a report, it is recommended that you measure the compression ratio achieved on a sample of the report. You can measure the compression ratio by using the ARSADMIN program. For example:
  • For reports that contain logical items, such as statements and policies, use the following example: arsadmin compress -l 200000 -s inputFile -o outputFile

    Where inputFile is the report that you want to measure and outputFile is the compressed output.

    To determine the compression ratio, divide the size of outputFile value by the length (-l 200000). For example, if the size of outputFile value is 66 000 bytes, then the compression ratio is 66000/200000 or 0.33 (3:1 compression).

  • For reports that contain line data and include a sorted transaction value, such as a general ledger, first determine the size of an indexed group of pages, for example, 100 pages. Then, extract a group of pages from a larger report and process them with the ARSADMIN program, for example: arsadmin compress -s groupPages -o outputFile

    Where groupPages is a file that contains a representative group of pages from a larger report and outputFile is the compressed output.

    To determine the compression ratio, divide the size of outputFile value by groupPages value. For example, if the size of outputFile value is 40 000 bytes and the size of the group of pages is 200 000 bytes, then the compression ratio is 40000/200000 or 0.20 (5:1 compression).

To run the ARSADMIN program, the input file must exist in HFS. The output file will be written to the directory that contains the input file. See the IBM® Content Manager OnDemand for z/OS®: Administration Guide for more information about the ARSADMIN program.
Beginning with APAR PI41677, Content Manager OnDemand supports the zEnterprise Data Compression (zEDC) available with z/OS 2.1 for OD77 and OD77Lite compression methods. Content Manager OnDemand will automatically attempt to use the zEDC unless configured not to by using the ARS_USE_ZLIB_HW=0 parameter in the ARS.CFG file. In order for that attempt to succeed, the following conditions must occur:
  1. The IFAPRDxx PARMLIB member must specify that the z/OS zEDC software feature is ENABLED;
  2. The RACF users for which use of the zEDC hardware is allowed must have READ access to FPZ.ACCELERATOR.COMPRESSION in the FACILITY CLASS; and
  3. Buffer sizes must meet the minimum sizes specified by the IQPPRM PARMLIB member. If the zEDC hardware cannot be used, software-based compression and decompression will be performed instead.
Additionally, the _HZC_COMPRESSION_METHOD=software environment variable can be used to force the zEDC to perform software-based compression and decompression.

The OD77, OD77Lite, and zEDC compression types implement the same compression and decompression algorithm. That is, something compressed with one can be decompressed by the other. However, all three have been tuned differently, so that compressed sizes are different. The OD77 compression tends to compress better than the zEDC, which in turn tends to compresses better than OD77Lite. Remember that different compression methods result in different CPU consumption, with greater compression consuming greater CPU. Experimentation should be performed with specific examples of data to determine what is the optimal compression to use.