PowerVP version 1.1.3 Agent Data Store

Configuring the Data Store

This function is not enabled by default. When sampling every second for 24 hours, the size of these files can get very large, for example, on a Power 8 system with 24 cores, the file size for 24 hours was 455,626,157 bytes. Also, if the partition has a maximum file size, you will need to configure PowerVP so that it doesn’t exceed that maximum. Therefore, you need to plan for the PowerVP data store files. This configuration is done with 5 directives in the configuration file. The configuration file is located in /QIBM/UserData/PowerVP/powervp.conf on IBM i and in /etc/opt/ibm/powervp/powervp.conf on AIX/VIOS and Linux. If you change the directives in the configuration file, you will need to stop the PowerVP agent and start it again to pick up the changes. If the PowerVP agent encounters an error due to running out of space or hitting the maximum file size, it will simply stop logging data, but continue to run. This will enable the PowerVP GUI to still display live performance data after encountering problems writing the data store.

LogData directive

The LogData directive directs the PowerVP agent if it is to do logging and what information is logged. The default for this directive is No. A value of Yes will enable the PowerVP agent to log both system level and partition level information to the file. A value of System will enable the PowerVP agent to log only the system level performance data. A value of Partition will enable the PowerVP agent to log only the partition level performance data.

On the partition level agents, you simply need to decide if you want to have the partition level information logged in a file. This information is the basic partition CPU utilization, Disk I/O, Ethernet I/O, and the cycles per instruction (CPI) information you see on the partition drilldown. If you decide you want this information to be logged, use the Partition value for the LogData directive on your partition level agent.

On your system level agent, you will need to decide if you want to log the partition level information in addition to the system level information. If you are running the system level agent on your VIOS, the partition drill down information is more than likely not necessary to retain. Therefore, you will probably use the System value for the LogData directive on your system level agent.  If your system level agent isn’t a VIOS, you have the same decision you have on the partition level agents, as to whether you want the partition information logged.

SampleInterval directive

The PowerVP agent gets the performance metrics based on the value of the SampleInterval directive. It defaults to 1 second, which means the agent will retrieve new metrics every second and if you are using the data store, it will write that information to the file every second. You may want to consider using a larger sample interval to help reduce the size of the data store file. The SampleInterval directive has just one value, numeric, in seconds, with valid values from 1 to 120.

LogFilePath directive

The location of the data store is configurable through the LogFilePath directive. The default location on IBM i is /QIBM/UserData/PowerVP/logs and on AIX/VIOS and Linux, the default location is /opt/ibm/powervp/logs. The PowerVP agent can create the log file in any location that the operating system can access, so you can put the file where you want to.

LogFileRotation

The LogFileRotation directive controls when the PowerVP agent will close the current data store file and start logging to a new data store file. The PowerVP agent will always rotate files at midnight. If you want more frequent rotation, you can provide a value from 1 to 24 hours. You can instead provide a file size in megabytes or gigabytes for the LogFileRotation directive which will cause the PowerVP agent to rotate the file when it reaches the configured size. The value is numeric for both options. It is followed by an H for hours, M for megabytes, or G for gigabytes, for example, 12H or 40M or 8G. If you use more frequent log file rotation, you can move the old log files off of your partition more frequently to conserve space. You may find it beneficial to rotate on size if you want to analyze the data using a spreadsheet application. Smaller files will load quicker and take less time to process.

LogFileArchive

The LogFileArchive directive controls when the PowerVP agent will archive or delete old data store files. This directive defaults to 7 days. At midnight, the PowerVP agent will check its LogFilePath directory for log files that are older than the configured number of days and delete files older than that. Again, you can specify this in days, using a smaller value to cause PowerVP to reclaim the disk used by old files. This will also give you time to move the files to another location if you want to retain them longer.

Data Store Data

The data store file is a standard comma separated values (CSV) format, allowing you to import the data into spreadsheet applications for further manipulation. You can sort the records, and then apply your own formulas to generate graphs and charts for your system utilization over time.

The first part of every file contains information records that describe what is contained in the data records.  Column 1 is used as the row type. Column 2 is the “occurrence” count, and an occurrence count of 0 indicates the record is column labels defining the data records. Column 3 of every row except the AAA and ZZZZ rows is the time column which correlates the data record to the time it occurred. Let’s take a look at each of the records. We will include a brief description of the record followed by a table that contains the column headings and one or more data records so you can get an idea of what the data will look like.

AAA – General information records

These records are always located at the beginning of each file. These records consist of 3 columns. Column 2 contains the type of information and column 3 contains the value. 

ZZZZ – Time records.

There is a time record for every sample interval. Column 2 is the time “id” which is used in the other data records so they can be correlated to a time. All of the other data records have this time “id” in column 3.

The rest of the entries use column 2 as an occurrence count, for example, if the data record is “by core”, that data record will occur once for each core per time interval. The occurrence count is incremented for each core.

SYSTOP – System topology record.

There is normally only 1 system topology record in the file. If the system topology changes (the number of nodes or chips or cores or virtual processors changes), this record will be rewritten to show the change.

CHIPTOP – Chip topology records.

There is one chip topology record per chip. If the system topology changes, these records will be rewritten to show the change. Note that this record is 30 columns and since that doesn’t fit well on this page, we are showing it as continued.

CORETOP – Core topology records

There is one core topology record per physical core. If the system topology changes, these records will be rewritten to show the change. Note that this record is 15 columns and since that doesn’t fit well on this page, we are showing it as continued.

REGLPARS – Registered partition records 

There is one per partition level agent registered to the system level agent and column 2 is used to count the number of partitions. These records are rewritten whenever a partition registers or de-registers from the system level agent. Note that this record is 12 columns and since that doesn’t fit well on this page, we are showing it as continued.

AFFDTOP – Affinity information by domain

Affinity information domain records define the affinity domains on the system. There are one of these for each affinity domain on the system. If the system topology changes, these records will be rewritten to show the change. Note that this record is 11 columns and since that doesn’t fit well on this page, we are showing it as continued.

AFFPTOP – Affinity information by partition

Affinity information by partition defines the affinity for each partition. Each partition has AFFPELE records associated with it. If the system topology changes, these records will be rewritten to show the change.

AFFPELE – Affinity elements for the partition

The affinity elements go with the AFFPTOP affinity information by partition records. There can be several of these per partition. If the system topology changes, these records will be rewritten to show the change. Note that this record is 11 columns and since that doesn’t fit well on this page, we are showing it as continued.

AFFVPROC – Affinity domain information by virtual processor

The Affinity domain information by virtual processor records define the virtual processor affinity of the system. If the system topology changes, these records will be rewritten to show the change.

The next entries are data records that are written on the system level agent each sample interval.

SCPUBC – CPU utilization by core

The CPU utilization by core records contain the information on CPU utilization for each physical core. There is a row for each core. PowerVP computes the core utilization on POWER 7 by dividing the number in column 5 by the number in column 9 and on POWER 8 by dividing the number in column 5 by the number in column 6. Then to determine system CPU utilization, it adds the core utilizations for all the cores together.

SCYCBC – Time base cycle utilization by core

The time base cycle utilization records contain the CPU cycles delta and timebase delta for each core for the sample period. There is a row for each core. 

SBUSBCH – bus utilization by chip

The bus utilization by chip records contain the bus utilization percent for each power bus on the system for each chip. There is a row for each chip.  Note that this record is 31 columns and since that doesn’t fit well on this page, we are showing it as continued.

SCPUBP – CPU utilization by partition

The CPU utilization by partition records contain the CPU utilization for each partition. There is a row for each partition. PowerVP uses this information to calculate the LPAR CPU utilization by adding the capped cycles and uncapped cycles, subtracting the idle cycles, and dividing  the result by the entitled capacity.

SMETRICBP – partition metrics records

The partition metrics record contain some metrics for each partition. There is a row for each partition on the system. Note that this record is 16 columns and since that doesn’t fit well on this page, we are showing it as continued.

The next entries are data records that are written for partition level data each sample interval.

PSTAT – partition configuration information for the partition

The partition configuration information record contains specific configuration information for the partition. This is partition specific information, so there is only one record per sample interval.

PCPI – partition cycles per instruction information

The partition CPI (cycles per instruction) information is retrieved from the PMU. PowerVP collects event counts for the following groups on POWER8: pm_utilization, pm_cpi_stack2, pm_cpi_stack4, pm_cpi_stack15, pm_cpi_stack18, pm_dsource1, pm_dsource4, pm_dsource5, pm_dsource6, pm_dsource7, and pm_dsource8; and for the following groups on POWER7: pm_dlatencies3, pm_cpi_stack1, pm_cpi_stack2, pm_cpi_stack7, pm_dsource1, pm_dsource2, pm_dsource3, pm_dsource4, pm_dsource5, pm_dsource6, pm_psource10, pm_dsource12, and pm_prefetch2. Note that this record is 10 columns and since that doesn’t fit well on this page, we are showing it as continued.

PENET – Ethernet throughput

The Ethernet throughput records show the throughput for each Ethernet interface on the partition.

PDISK – disk I/O information

The disk I/O information records show I/O information for each disk on the partition.

PCPU – CPU cycles

The CPU cycles records show the CPU used by the partition. This information is the same information as shown in the SCPUBP record, but only for the specific partition.