IBM Support

HMC Scanner for POWER Server Config and Performance Stats

How To


Summary

Use the HMC Scanner to quickly extract all the details of the POWER Servers the HMC is connected too and saved in a Microsoft Excel spreadsheet.

Steps

HMC Scanner Overview

HMC Scanner is a Java program that uses SSH to connect to an HMC or SDMC or FSM or IVM (IVM is experimental), downloads the system configuration and produces a single Excel spreadsheet that contains the configuration of servers and LPARs. The result is a simple way to document configuratioun and to easily look at most important configuration parameters. In PureFlex environments, only Power Systems compute nodes are detected.

Information is organized in tabs:

  • System summary: name, serial number, cores, memory, service processor IP for each server
  • LPAR Summary: list of all LPAR by serve with status, environment, version, processor mode
  • LPAR CPU: processor configuration of each LPAR
  • LPAR MEM: memory configuration of each LPAR
  • Physical Slots: list of all slots of each system with LPAR assignment, description, physical location and drc_index
  • Virtual Ethernet: network configuration of each virtual switch and each LPAR
  • Virtual SCSI: configuration of all virtual SCSI adapters, both client and server
  • VSCSI Map: devices mapped by each VIOS to partitions
  • Virtual Fibre: virtual fibre channel configuration of client and server with identification of physical adapter assigned
  • SEA: SEA configuration and statistics for all VIOS
  • SW Cores: LPAR and virtual processor pool configuration matrix to compute the number of software licenses. Simulation of alternative scenarios is possible.
  • CPU Pool Usage: monthly average history of CPU usage of each system. Based on last 12 months of lslparutil data.
  • Sys RAM Usage: monthly average history of physical memory assignement to each LPAR. Based on last 12 months of lslparutil data.
  • LPAR CPU Usage:monthly average history of CPU usage of each LPAR. Based on last 12 months of lslparutil data.
  • CPU Pool Daily Usage: 1 year of CPU usage of every pool and subpools of each system. Based on daily averages.
  • LPAR Daily Usage: 1 year of CPU usage of every LPAR of each system. Based on daily averages.
  • CPU Pool HourlyUsage: 2 months of CPU usage of every pool and subpools of each system. Based on hourly averages.
  • LPAR Hourly Usage: 2 months of CPU usage of every LPAR of each system. Based on hourly averages.

NOTE: in order to get CPU usage data you need to enable HMC performance data collection on each managed system. Data collection is off by default. Check the hmcScanner FAQ section below for details.

The tool is constantly updated to match changes in tool syntax, to manage new data types and to add new features. If you have any problem or any suggestion to improve HMC Scanner, please send me (Federico Vagnini in IBM, Italy) an mail (vagnini@it.ibm.com). Since new features are mostly suggested by users, propose yours! Please always include the downloaded files to allow be proper debugging.

HMC Scanner is packaged in a single ZIP file available in the download section of this page. A sample report is included. Updates are provided from time to time so check the Wiki or contact the author.

This tool is not officially supported by IBM. No guarantee is given or implied, and you cannot obtain help from IBM. It a personal project of the author, Federico Vagnini (IBM Italy).

See also pGraph page form the same author. It allows multiple nmon, vmstat, iostat, sar, lslparutil files to be loaded regardless of their size. Graphs are provided together with a very useful "time zoom" that allows you to select the time period to show.

The code exploits features from other software. See text file in the ZIP file for license details.

Check the hmcScanner FAQ section below for common issues in using the tool.

Installation instructions

HMC Scanner only requires Java 1.6 installed (it may work with previous Java versions but issues are known with Java 1.5 and -stats flag).

Select a system with Java installed that is capable of contacting HMC/SDMC/FSM/IVM with a direct SSH connection or using an SSH connection through a HTTP, SOCKS4 or SOCKS5 proxy server. The HMC/SDMC/FSM/IVM must allow remote SSH command execution.

Unzip the downloaded file and run the hmcScanner.bat or hmcScanner.ksh script if they are kept located in the same directory of Java classes. The script use the internal BASE variable to point to the directory where the ZIP file has been decompressed. By default it uses the directory where batch file is located. You can update the variable in the batch file if needed.

Fixes are provided in download section. Download the new jar file and overwrite the original hmcScanner.jar file.

 

Parameters

There are few parameters that can be changed in order to override default setup. Just run the batch file with no parameters to get syntax help:

hmcScanner.Loader <HMC name/IP> <user> [-p <password>] [-dir <local dir>] [-perf <start> <end> <d|h>] [-readlocal] 
                                       [-key [file] ] [-stats] [-sanitize] [-csv] [-csvchar <char>] [-html]
                                       [-log <file>] 
                                       [-proxy <http|socks4|socks5> <host> <port> [user] [password] ] 
                                       [-m <managed system>] [-timeout <secconds>] [-novios]

    -dir <local dir> is the directory where data will be stored. Default is current directory.
    -perf <start> and <end> is data collection retrieval interval. Syntax is: YYYYMMDD
               <d|h> d=daily data samples; h=hourly data samples
    -readlocal will force reading of existing local data without contacting HMC
    -key will use OpenSSH private key (default $HOME/.ssh/id_rsa)
    -stats will produce system statistics. It does NOT require -perf!
    -sanitize will remove sensitive data.
    -csv creates one .csv file for each sheet in a separate directory.
    -csvchar defines che separator char is CSV (default is comma).
    -html creates HTML report in a separate directory.
    -log <file> logs SSH activity into file
    -proxy <http|socks4|socks5> <host> <port> [user] [password]
           select proxy type, host, port to use for SSH communication
    -m <managed system> restrics the scan to the provided managed system
    -timeout <seconds> Timeout SSH connection after idle seconds. Use with care! Default is no timeout.
    -csvdir <directory> directory where csv files will be written. Default is <hmc>_<date>_<time>_CSV
    -htmldir <directory> directory where html files will be written. Default is <hmc>_<date>_<time>_HTML
    -novios do not collect data from VIOS to speed up pure hw config scanning.

The basic usage is hmcScanner.ksh HMC_name hscroot -p password -stats

It will open a single SSH connection to the HMC with hscroot user and the provided password. The tool will identify all systems managed by HMC and download their configuration as text files. All files will be stored in a new directory with the nameof the HMC, under the current working directory. Once download is finished, a spreadsheed named <HMC_name>_scan.xls is created in the current working directory. The downloaded files are not deleted.

It is possible to use OpenSSH keys to connect to HMC without providing a password. If no password is provided, HMC Scanner will use the public key located in $HOME/.ssh/id_rsa. If a different key has to be loaded, use the -key parameter to provide the right file. Only OpenSSH keys with private key not protected by a passphrase is supported: if you are using other key types, please use keys generated by OpenSSH (you can use the package available for AIX)

The default location of downloaded files can be modified by using the -dir parameter. A new directory will be created under the given directory.

The spreadsheet file can be generated again from already downloaded files by using the -readlocal switch. HMC will not be contacted.

In order to have in the spreadsheet the tabs with the history of CPU and memory usage, the HMC must collect performance data and you have to provide the -stats switch. Allow some more time to download the data.

The -perf switch has no effect in spreadsheet content. It is just a simple way to download lslparutil data from HMC in raw format. It can be later loaded by tools like pGraph.


Known issues

  • LPAR names with commas will prevent hmcScanner from working. Best practice is to have alphanumeric characters and underscore.
  • On big environment you may get Java memory errors. Change maximum Java allowed values by editing script and adding "-Xmx1024m" before "hmcScanner.Loader"
  • VIOS partitions with a huge set of disks may require a long time to download data. Allow hmcScanner to run and finish or use -novios to skip data collection from VIOS

Sample execution

The only required parameters are HMC's IP or hostname and the user to log on. Usually you also provide the password with -p flag unless you have exchanged SSH keys. This example also shows how to download last 1 year of performance data to be included in the Excel output, plus the generation of a report with no sensitive data included.

C:\Federico\HMCscanner>hmcscanner 9.71.196.28 hscroot -p abc123 -rowmode -stats
-sanitize
hmcScanner version 0.10
Detecting manager type: HMC
Detecting managed systems: 16 systems present.
Starting managed system configuration collection:
   Scanning 750e-8233-E8B-SN1003C8P:      ............... DONE
   Scanning New_DNS:                      ............... DONE
   Scanning PowerHA-1:                    ............... DONE
   Scanning PowerHA-2:                    ............... DONE
   Scanning Server-8406-70Y-SN06AD0EA:    ............... DONE
   Scanning Server-8406-70Y-SN06AD10A:    ............... DONE
   Scanning Server-8406-71Y-SN06ACE4A:    ............... DONE
   Scanning Server-9110-51A-SN10AC29C:    ............... DONE
   Scanning i7-8233-E8B-SN10080EP:        ............... DONE
   Scanning op1:                          ............... DONE
   Scanning p6-8203-E4A-down:             ............... DONE
   Scanning p6-8203-E4A-up:               ............... DONE
   Scanning p7-8233-E8B-SN100805P:        ............... DONE
   Scanning pSAP:                         ............... DONE
   Scanning pSeries-9113-550-SN65FDE9C:   ............... DONE
   Scanning tsia:                         ............... DONE
Collection successfully finished. Data is in C:\Federico\HMCscanner\9.71.196.28\

Performance data collection:
   Loading 750e-8233-E8B-SN1003C8P:      .......... .
   Loading New_DNS:                      .. .
   Loading PowerHA-1:                    . .
   Loading PowerHA-2:                    ... .
   Loading Server-8406-70Y-SN06AD0EA:    . .
   Loading Server-8406-70Y-SN06AD10A:    . .
   Loading Server-8406-71Y-SN06ACE4A:    . .
   Loading Server-9110-51A-SN10AC29C:    .. .
   Loading i7-8233-E8B-SN10080EP:        .................. ..
   Loading op1:                          .. .
   Loading p6-8203-E4A-down:             ....... .
   Loading p6-8203-E4A-up:               .............. ..
   Loading p7-8233-E8B-SN100805P:        ................................. ..
   Loading pSAP:                         .... .
   Loading pSeries-9113-550-SN65FDE9C:   ............... ..
   Loading tsia:                         .. .
................................................ DONE
Starting Excel file creation. Done: C:\Federico\HMCscanner\9.71.196.28_20130718_111826_scan.xls
Starting Excel file creation. Done: C:\Federico\HMCscanner\9.71.196.28_20130718_111826_scan_safe.xls

HMCscanner contacts the HMC, detects the presence of 14 managed systems and downloads their setup. This is the base usage.

The -stats flag activated a performance data download for each system that is related to the last 12 months and it is included in the generated Excel tabs.

Usually only one Excel file is generated, but the -sanitize file requires a second file to be generated with sensitive data masked (IPs, MAC, WWPN, serials, etc).

 

Sample output

HMCscanner produces a single Excel file with multiple tabs. The following pictures show samples of the report. Full report example is available in the download section. The "-rowmode" is the suggested output mode that is shown in these samples.

Configuration of all managed systems
LPAR list
CPU setup of LPARs
Memory configuration of LPARs
Physical slots in the system
Virtual Ethernet configuration
Virtual SCSI setup
Virtual fibre channel configuration

SEA Configuration and statistics

SEA data handling is new in version 0.10 and it will be improved in the next future. The goal is to provide specific SEA information that is available only within the VIOS environment. The following pictures show some examples:

Complete lists of all SEAs in the environment, together with their setup.

Setup of all Etherchannel:

SEA statistics extremely useful for correct sizing of buffers. The most important cells are highlighted in yellow and red.

When hypervisor send failures are not null, they are highlighted in yellow. Virtual Ethernet buffers are set in yellow when the "max allocated" value is greater then "min buffers" and are set in red when they are equal to "max buffer".

Pool and LPAR statistics

The "-stats" flag makes hmcScanner download lslparutil data from the HMC. Hourly data is downloaded for the latest two month and daily data is collected for the latest 12 months. Statistics are then provided for global pool, each virtual pool and each LPAR. Data is organized in tabs: 12 months based on monthly averages, 12 months based on daily averages, 2 months based on hourly averages.

Values can be easily used to produce graphs by exploiting Excel graphing capabilities (or your favorite compatible spreadsheet).

In order to ease the identification of patterns, the cells have different formats based on the value compared to entitlement and maximum available size.

Please note: Performance data collection is performed by HMC, but this capability is disabled by default. You need to enable it for each managed system in order to see data in hmcScanner's output. If you see a blank cell or a value of 0 in the "Perf Sample Rate" column of System Summary tab, the collection is disabled. Change sampling rate on HMC to something like 5 or 30 minutes: select managed systems and go to Operations->Utilization Data->Change Sampling Rate

Pool statistics

Data is shown based on global and virtual pools. Cell format changes depending on the value of the sample: if value is at 90% or more of the pool size, the cell is red, if value is at 70% or more of the pool size the cell is yellow. If more that 25% of samples of a specific pool is colored, the pool name gets the same color.

On top of values there is also a summary of pool data:

  • Days: number of samples shown in the column
  • Reds%: percentage of red cells in the samples
  • Yellows%: percentage of yellow cells in the samples
  • Max: max value in all samples
  • Avg: average value in the samples
  • StdDev: standard deviation
  • 90 Perc: assuming a normal distribution of values, 90% of samples are below or equal to tis value
  • Last size: the pool size at the time hmcScanner was run

LPAR statistics

Data is shown for each LPAR known by the manager. Data can be split in multiple tabs for readability.

Cell format changes depending on the value of the sample: if value is at 90% or more of maximum size, the cell is red, if value is at 70% or more of the maximum size the cell is yellow. If more that 25% of samples of a specific LPAR is colored, the LPAR name gets the same color. The maximum size is the minimum value of Virtual Processor, virtual pool availability of global pool availability in a specific moment. For example, a LPAR may have a red cell because the virtual shared pool it belongs is full, even if global pool has free resources available and virtual processors are higher than processor consumed.

Cells showing a consumption higher than the entitlement in that sample are highlighted in bold.

On top of values there is also a summary of pool data:

  • Days: number of samples shown in the column
  • Reds%: percentage of red cells in the samples
  • Yellows%: percentage of yellow cells in the samples
  • Over Cap %: percentage of samples that show usage over entitlement
  • Max: max value in all samples
  • Avg: average value in the samples
  • StdDev: standard deviation
  • 90 Perc: assuming a normal distribution of values, 90% of samples are below or equal to tis value
  • Last Ent: the entitlement size at the time hmcScanner was run

Graphs (under developement)

Using the 0.10.5 version with the new "-html" flag, you can create performance graphs under the HTML directory. Point your browser on the "index.html" file to have a menu of all pictures. Two PNG files are created for each LPAR, for each system and for each virtual pool: 1 year data based on daily averages and 2 months data based on hourly averages.

The following picture is an example for a single LPAR.

The red line is the number of online Virtual Processors, that is the theoretical available CPU the LPAR can access if uncapped and if the pool has enough resources. The green area is the entitled capacity or the CPU resource that is guaranteed to be available to the partition. The blue line is the processor that the LPAR has used.

The yellow area is the additional quantity of CPU that was available to the LPAR but not used. If the partition is capped, it matches with the entitled capacity. If the partition is uncapped, it shows the quantity of free resource in the global pool or in the virtual pool the partition belongs to (it will never exceed the number of virtual processors).

Yellow area represents the CPU that the partition could get if needed but did not request. If blue line gets into the yellow area the partition is getting more resources than entitled; if it gets near the grey area, the partition had to compete with other partitions, either in the global pool or in its virtual pool. In the example, the partition did not compete with others and was in a virtual pool that was near to be full.

HTML and CSV output

hmcScanner will always produce a report in Excel format in a single file. You have the option to request also an HTML or CSV output formatt by using -html or -csv flags. The output is exactly the same with the following excaptions:

  • HTML and CSV obviously do not have formulas
  • SW Cores tab is not available since it depends on formulas
  • If you combine -stats with -html flags you will also have performance graphs of LPARs and pools in the HTML report




hmcScanner FAQ

  • Which kind of information can be retrieved by hmcScanner and how?
    • hmcScanner runs commands on HMC only and can retrieve only information known by HMC. The LPARs present on managed systems are not contacted by any means. The HMC uses RMC protocol to contact the LPAR to get some information but is it not able to run operating system commands. Only exception is VIOS: HMC can use RMC protocol to run padmin commands and hmcScanner exploits this option to get VIOS specific configuration.
  • The data related to VIOS configuration is missing in hmcScanner report
    • When RMC protocol is not working HMC can not run commands on a VIOS. Please ensure that there is network connectivity between HMC and VIOS and that no firewall blocks TCP/UDP on port 657 bidirectional. The protocol is the same used by dynamic LPAR reconfiguration. The RMC status is shown in hmcScanner report under LPAR_Summary tab under RMC State column.
  • The operating system version in the report is missing or non accurate
    • Operating system version is detected by HMC by using RMC protocol. If protocol is not working the value can not be identified or updated. Updates are not immediately available. Sometimes a HMC reboot helps. Unfortunately the scanner only gets HMC provided values. If you think there is a problem, please contact your local IBM support.
  • There is no performance data in the report even if I use -stats flag, or data is present only for some LPARs
    • Performance data needs to be enabled on the HMC for each managed system. By default it is not activated. You can see the status in the report in the System_Summary tab under the Per Sample Rate column: if the cell is empty or has a zero value, data collection is not active. You can enable the data collection on the GUI looking at managed system menu or by using the CLI: chlparutil -r config -s <seconds> -m <managed system> (I suggest 30 minutes) or chlparutil -r config -s <seconds> for all managed systems. If you add a new managed system it will not have collection enabled. If you have a dual HMC setup and want both HMCs to collect data, you need to enable collection on both HMCs.
  • I see performance data on the report but I was expecting more data
    • There is currently a bug on HMC software that prevents a full extraction of performance data using lslparutil command as hmcScanner does. In order to fix the issue you need to upgrade the HMC code version V8 R8.2 Service pack 2 or later (PTF MH01488 HMC V8 R8.2.0 Service Pack 2)




Additional Information


If you find errors or have question, email me: 

Document Location

Worldwide

Download HMC Scanner

Sample spreadsheets:

HMC Scanner code: Always use the latest version since it fixes bugs and introduce new capabilities. Check often!

Latest releases are on top. Read the "Changes" column for installation details. You need the latest "zip" file (the full package) and the latest "jar" fix. The "jar" file is the latest fix level that needs to be renamed as "hmcScanner.jar" to replace the file in the latest zip package.

I keep old versions just in case but check the list from the top.

Version Date posted Package

Changes

0.11.42

(current)

23 May 2019

Update 42 of version 0.11.

Full package: download and unzip. Use bat or ksh script.

This version has an update to SSH library that is required to connect to an HMC of version 930 or higher

0.11.35

09 May 2017

Update 35 of version 0.11

Check you have version 0.11.24 installed, download the jar file and rename it as "hmcScanner.jar"

0.11.24 28 April 2016

Update 24 of version 0.11. It requires Java 1.6

Download and unzip the file in a separate folder. Run the hmcScanner.bat or hmcScanner.ksh with no flags to check syntax.

0.11.17

22 January 2015 No longer supported

Update 17 of version 0.11

Download the hmcScanner.jar.0.11.17 file and rename it into hmcScanner.jar, overwriting the existing one. You must be at 0.11 version to apply the update.

Run hmcScanner.bat (or .ksh) with no parameters to see all options. I will slowly update the Wiki description.

0.11.0

20 December 2013
No longer supported

The SSH library has changed and should remove all communication issues that occurred in many FSM installations and some HMC environments.

Added proxy capabilities for SSH communication (HTTP, SOCKS4 and SOCKS5)

Added capability to restrict scanner to a single managed system (-m flag)

-rowmode flag is now always implicit

HTML output directory now containg HMC and generation date in name

0.10.8

(latest 0.10 version)

10 October 2013
No longer supported

Latest fix level for version 0.10

Please use version 0.11

0.9 14 March 2013
No longer supported
Added E:VP% field and -sanitize flag

NOTE: If you do not use "-sanitize" flag you will get a Java Exception you can ignore.
0.8 22 February 2013
No longer supported
0.7 16 October 2012
No longer supported
16 October 2012
No longer supported

[{"Business Unit":{"code":"BU009","label":"Systems - Cognitive"},"Product":{"code":"SWG10","label":"AIX"},"Component":"","Platform":[{"code":"PF002","label":"AIX"}],"Version":"All Versions","Edition":""},{"Business Unit":{"code":"BU009","label":"Systems - Cognitive"},"Product":{"code":"HW1W1","label":"Power ->PowerLinux"},"Component":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"All Versions","Edition":""},{"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"SWG60","label":"IBM i"},"Component":"","Platform":[{"code":"PF012","label":"IBM i"}],"Version":"All Versions","Edition":""}]

Document Information

Modified date:
29 November 2019

UID

ibm11117515