IBM Support

IBM FlashSystem Comprestimator

Download


Downloadable File

File link File size File description
http://ftp.software.ibm.com/storage/san/sanvc/comprestimator_1.0.1/setup_014b.exe 3839448 Comprestimator install file v1.0.1

Abstract

IBM FlashSystem Comprestimator is a command line host-based utility that can be used to estimate expected compression rate for block-devices. The IBM FlashSystem Comprestimator utility uses advanced mathematical and statistical formulas to perform the sampling and analysis process in a very short and efficient way. The utility also displays its accuracy level by showing the maximum error range of the results achieved based on the formulas it uses. The utility runs on a host that has access to the SVC, Storwize or FlashSystem that will be analyzed, and performs only read operations so it has no effect whatsoever on the data stored on the device.

Download Description

 

Using Comprestimator

To use Comprestimator on a Linux, ESXi server:

  1. Log in to the server by using the root account;
  2. Obtain the list of device names:
    • In Linux: Use the "fdisk –l” command;
    • In ESXi 4: Use the “esxcli corestorage device list | grep Dev" command;
    • In ESXi 5 or later: Use the “esxcli storage core device list | grep Dev” command.
  3. Run Comprestimator with the -d <device_name> flags to analyze a device or a partition.

Comprestimator is designed to scan any block device that is readable by the OS itself. This typically includes devices managed by logical volume managers (LVMs) or partitioned by the OS. However, for practical reasons, since compression is applied to physical volumes, it is recommended to estimate compression by running Comprestimator on the same block device/physical volume that will be compressed, and not on a logical volume, which may be spanning on those volumes. It is thereby highly recommended to always analyze the native block-device when using Comprestimator.

Some volume managers "reserve" some of the LUN capacity for internal use. Since Comprestimator reads directly from the block device, some of these IOs may fail. The tool will tolerate up to 1% failed IOs and a scan will be aborted if this threshold is reached.

Note that, for Flashsystem, the Maximum effective capacity per enclosure is limited based on the number of modules configured in the system. This configuration will limit the total physical and effective capacity on the enclosure.

 

To use Comprestimator on a Windows server:

  1. Log in to the server using an account with administrator privileges;
  2. Open an elevated Command Prompt with administrator rights (Run as Administrator);
  3. Run Comprestimator with the -l flag to list the disk;
  4. Run Comprestimator with the –n flag to analyze the specific disk;
  5. Alternatively, run Comprestimator with –d and the device name. Note that when using –d option with the long device name, the device name must be wrapped in double quotation marks.

Syntax

Using Linux, ESXi:

comprestimator -s DRP|FCM [-h | -d device] [-c file_name] [-v] [-p number_of_threads]  [-P] [-I] [--config= task_file]< --stripewidth

Using Windows:

comprestimator -s DRP|FCM [-h | -l | -d device | -n disk_number] [-c file_name]  [-v] [-p number_of_threads] [-P] [-I] [--config=task_file] --stripewidth

Options Description
-s, --storageSys Specify storage system type. Supported values are: DRP, FCM
-d Specifies the device name. Note that the name must be wrapped in double quotation
device Path of device to analyze (e.g. /dev/sda in Linux).
--stripewidth Stripe width for the planned FCM array. Allowable values: 5 - 16. Default 12.
Stripe width is the width of a single unit of redundancy in the array. Arrays with smaller stripe width are quick to complete rebuild operations. But such arrays also use more capacity for parity, and parity is usually less compressible than user data.
-n disk_number Only in Windows – The disk number. Identify this number by running Comprestimator with the –l flag first.
-l List the disk numbers and names available to use by Comprestimator. The drive numbers reported match the “Disk #” reported in Windows Disk Management.
-p Specifies the number of processes (or threads in Windows) between 1 and 50. The default value is 10.
number_of_processes The number of processes (or threads in Windows) used by Comprestimator.
-P Displays the results using a paragraph format.
-c

Export the results to a CSV-formatted output file.

file name File name of CSV-formatted output file.
-h Display usage information.
-v Display the results of every few samples. Note that the output is extended to a few thousands of lines as a result.
--config Configuration file that contains list of devices to analyze. Used for automation.
file Configuration file that contains list of devices to analyze. The file should contain a list of devices in the same format of the –d option, one device in each line.
For example:

# cat devlist_file
/dev/sda1
/dev/sda2
 
Explanation of scan results
Sample# The number of the current sample reported.
Device The device name used in the scan.
Size(GB) The total size of the device as reported by the operating system, in gigabytes.
Compressed Size(GB) The estimated size of the device if it will be in gigabytes.
Total Savings(GB) The total estimated savings from thin-provisioning and compression, in gigabytes.
Total Savings(%) The estimated savings from thin-provisioning and compression, in percentage of the size of the device.
This value is calculated in the following method:
Total Savings(%) = 1-( Compressed Size(GB) / Size(GB) )
Thin Provision Savings (%) The estimated savings from thin-provisioning. Shows how much of the volume is user data vs unmapped/zero.
Compression Savings(%) The estimated savings from compression.
Compression Accuracy The accuracy of the estimate provided by Comprestimator. The results provided are estimated based on samples from the device and therefore may be lower or higher than the actual compression that would be achieved. The approximate accuracy of the results is represented as a percentage of the total size of the device.
For example, if the estimated Compression Savings (%) is 67%, and the Compression Accuracy Range is 5%, the actual compression savings (in percentage) if this device will be compressed is between 62% and 72%.
Range(%)
 

Comprestimator can also be used without any flag to get help and usage information. Following is an example of the output you should expect for each analyzed device/partition:

Sample#
Device
Size (GB)
Compressed Size (GB)
Total Savings (GB)
Total Savings (%)
Thin Provision Savings (%)
Compression Savings (%)
Compression Accuracy Range %
3425
/dev/dm-2
500
112.8
387.1
77.4%
39.6%
62.6%
4.7%
 

Examples

Linux

To display the usage information:

# ./comprestimator

To list all the available device names in Linux:

# fdisk -l

Disk /dev/sda: 78.3 GB, 78383153152 bytes
255 heads, 63 sectors/track, 9529 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000021
Device Boot Start End Blocks Id System
/dev/sda1 * 1 8555 68718006 83 Linux
/dev/sda2 8556 8924 2963992+ 5 Extended
/dev/sda5 8556 8924 2963961 82 Linux swap / Solaris

Use the device names listed in the output, in this example: /dev/sda1, /dev/sda2, /dev/sda5.

To run a standard scan of /dev/sdb device (Linux version):

# ./comprestimator –d /dev/sdb -s DRP

Version: 1.5.3.1 (Build w0117)
Start time: 13/07/2013 15:54:02.918234
Device name: /dev/dm-2
Device size: 500.0 GB
Number of processes: 10
Exhaustive: no
Sample# | Device | size (GB) | Compressed Size(GB) | Total Savings(GB) | Total Savings(%) |Thin Provision Savings(%) | Compression Savings (%) | Compression Accuracy Range(%)
3425 | /dev/dm-2 |500.0|112.8| 387.1 |77.4% |39.6% |62.6% |4.7%

 

VMWare vSphere ESXi 4

The device names that can be scanned using Comprestimator on an ESXi 4 Server can be obtained by using the following command when logged in to the ESXi server with root account using SSH:

~ # esxcli corestorage device list

To display a shorter, more readable list, you can use the following command to filter out unnecessary information:

~ # esxcli corestorage device list | grep Dev
    Device Type: Direct-Access
    Devfs Path:
/vmfs/devices/disks/naa.600605b0042ff75017a86e2e531227e7
    Device Type: Direct-Access
    Devfs Path: /vmfs/devices/disks/naa.6005076801ac861ef800000000000007
    Device Type: Direct-Access
    Devfs Path: /vmfs/devices/disks/naa.6005076801ac861ef800000000000008
    Device Type: Direct-Access
    Devfs Path: /vmfs/devices/disks/naa.6005076801ac861ef80000000000000a
    Device Type: Direct-Access
    Devfs Path: /vmfs/devices/disks/naa.6005076801ac861ef80000000000000b
    Device Type: Direct-Access
    Devfs Path: /vmfs/devices/disks/naa.6005076801ac861ef80000000000000d
    Device Type: CD-ROM
    Devfs Path: /vmfs/devices/cdrom/mpx.vmhba0:C0:T0:L0

This will list only the device types and their respective path names. The device path names can then be used as arguments for Comprestimator.

To scan a device in ESXi 4:

 # ./comprestimator -d /vmfs/devices/disks/naa.600605b0042ff75017a86e2e531227e7

 

VMWare vSphere ESXi 5 or later

The device names that can be scanned using Comprestimator on an ESXi 5 Server can be obtained by using the following command when logged in to the ESXi server with root account using SSH:

~ # esxcli storage core device list

To display a shorter, more readable list, you can use the following command to filter out unnecessary information:

~ # esxcli storage core device list | grep Dev

This will list only the device types and their respective path names. The device path names can then be used as arguments for Comprestimator.

To scan a device in ESXi 5:

# ./comprestimator -d /vmfs/devices/disks/naa.60050768029c00e8140000000000002d

 

Windows

To scan a device in Windows, first identify the disk number using the -l argument:

# comprestimator –l

Drive number [0]        \\?\ide#diskhitachi_hts725032a9a364pc3zc70f#5&1e544866&0&0.0.0#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}

The drive numbers reported match the “Disk x” reported in Windows Disk Management. Now run a standard scan of disk 0:

# comprestimator –n 0 -v
Version: 1.5.1.1 (Build w0082)
Start time: 17/07/2013 17:33:08.728421
Device name: \\?\ide#diskhitachi_hts725032a9a364pc3zc70f#5&1e54866&0&0.0.0#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}
Device size: 298 GB
Number of processes: 10
Exhaustive: no
...
...

Alternatively, the drive numbers reported match the “Disk #” reported in Windows Disk Management. Now run a standard scan of disk 0:

# comprestimator –d "\\?\ide#diskhitachi_hts725032a9a364_________________pc3zc70f#5&1e544866&0&0.0.0#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}"


Known issues

The following is a list of known issues in the software current release. These issues are due to a combination of environment constraints and current version limitations.

ID Details
12599 Comprestimator reports incorrect results (i.e. -1.$%) for devices smaller than 1GB. Use Comprestimator only with devices larger than 1GB.
 

Troubleshooting

Linux, ESXi

  1. Run the tool with –v option – check output;
  2. Check the contents of the log file – comprestimator.log in the current working directory;
  3. Confirm that the device is readable using standard UNIX commands:
    bash-4.2# dd bs=262144 if=/dev/rdsk/c1t0d0s2 of=/dev/null
  4. If the device cannot be read, verify that it is indeed a raw device path name;
  5. If the utility crashes or hangs, rerun the utility in verbose mode, and with trace information:
    # strace –o /tmp/strace.txt –f ./comprestimator –v –d /dev/sdb
  6. Check the contents of the log file – "comprestimator.log" in the current working directory.

Prerequisites

Comprestimator is supported and can be used on the following client operating system versions:

  • Windows 2003 Server, Windows 7, Windows 2008 Server , Windows 8, Windows 2012
  • ESXi 4, 5, 6
  • ESXi 6.5, 6.7 (default number of threads only)
  • Red Hat Enterprise Linux Version 5.x, 6
  • SUSE SLES 11
  • Ubuntu 12
  • CentOS 5.x

Installation Instructions

Read this first!

IBM FlashSystem Comprestimator can only be installed on supported Windows operating systems (see Pre-requisites).
After installation completes, the binary files for other supported operating systems are available in the Windows installation folder.

By default, the files are copied to:
In Windows 64-bit: c:\Program Files (x86)\IBM\IBM FlashSystem 9100 Comprestimator
In Windows 32-bit: c:\Program Files\IBM\IBM FlashSystem 9100 Comprestimator

This section provides useful information on installing IBM FlashSystem Comprestimator on a host and using it to analyze devices on that host. Depending on the environment configuration, in many cases Comprestimator will be used on more than one host, in order to analyze additional data types.

To install IBM FlashSystem 9100 Comprestimator on a Linux:

  1. Log in to the host using the root account;
  2. Copy the Comprestimator binary file from the installation folder to any folder on the host using an SCP utility such as WinSCP or PuTTY;
  3. Log in to the UNIX host, and confirm that the file has executed permissions, to add execute permissions to the binary file, type chmod +x comprestimator.

To install IBM FlashSystem 9100 Comprestimator on ESXi:

  1. Enable SSH on the ESXi server. Check the following URL for instructions or skip this step if SSH is already enabled: https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.esxi.install.doc/GUID-F9234F4A-FD6A-477B-B7BA-28EB6B08CE82.html
  2. Copy the ESX/comprestimator_esx binary file to any folder on the ESXi server. You can copy the file to the ESXi server by using the secure copy protocol (SCP), for example ‘scp’ on Linux or ‘WinSCP’ on Microsoft Windows

To install IBM FlashSystem 9100 Comprestimator on a Windows host:

  1. Log in to the host;
  2. Copy the appropriate Comprestimator binary file from the corresponding folder to any folder on the host.

MD5 Checksum = fc4e2e6125f1f3efe975a9c6e8110690

Off
[{"DNLabel":"Comprestimator v1.0.1","DNDate":"11 Jul 2018","DNLang":"English","DNSize":"3750000 B","DNPlat":{"label":"Windows","code":"PF033"},"DNURL":"https://ftp.software.ibm.com/storage/san/sanvc/comprestimator_1.0.1/setup_014b.exe","DNURL_FTP":"","DDURL":null}]

Document Location

Worldwide

[{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"STSLR9","label":"IBM FlashSystem 9x00"},"ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Version(s)","Line of Business":{"code":"LOB26","label":"Storage"}},{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"STSLR9","label":"IBM FlashSystem 9x00"},"ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Version(s)","Line of Business":{"code":"LOB26","label":"Storage"}},{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSA76Z4","label":"IBM FlashSystem 7x00"},"ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Version(s)","Line of Business":{"code":"LOB26","label":"Storage"}},{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"ST3FR9","label":"IBM FlashSystem 5000"},"ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Version(s)","Line of Business":{"code":"LOB26","label":"Storage"}}]

Document Information

Modified date:
06 March 2023

UID

ibm16209688