Planning Active Memory Expansion
Today, memory capacity is one of the major bottlenecks that prevents a workload to scale linearly with the power of processors. The processor's power capacity increases year-by-year, and to feed fast processors the system should include sufficient memory. Memory constraint is one of the key reasons for under-utilized processing power.
Active Memory Expansion (AME) is a feature which helps to increase the effective memory size of a system without increasing the physical memory size. This is achieved by packing more data into smaller memory footprint, thereby giving space for storing more data.
The effective increase in memory capacity, due to AME, leads to higher throughput and increased utilization of the LPAR. Memory savings can be used to add more LPARs or additional workloads to the system to do more work.
The operating system manages AME, and it is configurable on a per LPAR basis by the virtualization control point. Users can unlock the AME capability of a system by entering a VET activation code. Once AME has been unlocked for a system, an administrator is free to create LPARs with memory expansion enabled. AME can be disabled at any point of time if it does not solve the problem.
When AME is enabled for an LPAR, the operating system starts compressing and decompressing data in memory based on the workload's memory access pattern. This results in the creation of two pools of memory: compressed and uncompressed.
The operating system controls the pool sizes and is responsible for moving pages between the compressed and uncompressed pools based on workload.
When an application needs to access data that is compressed, the operating system automatically decompresses the data and moves it from the compressed pool to the uncompressed pool making it available to the application. When the uncompressed pool is full, the operating system compresses the data and moves it from the uncompressed pool to the compressed pool. There will be some additional utilization of CPU based on the workload and the level of memory expansion.
The target memory expansion factor is user-configurable. The user can specify the target memory expansion factor for a workload. This factor is treated as a multiple of the actual logical memory size of the LPAR. The range of values supported for this factor are 1.0 (i.e. 100%) to 10.0 (i.e. 1000%).
This memory expansion factor provides input to the operating system on the effective memory capacity the user is trying to achieve, and thus, the amount of data the OS should try to fit into memory.
An uncompressed pool of 20GB can hold up to 20GB of data. The amount of data that can fit into the compressed pool varies based on the compressibility of the data. If data can be compressed by a factor of 2 to 1, the amount of data that can fit into a 20GB compressed memory pool will be 40GB. If the data can only compress by a factor of 1.8 to 1, the amount of data that can fit into a 20GB compressed pool will be 36GB.
Memory Expansion factor = Expanded Memory Size / Actual Memory Size = 60 / 40 = 1.5X
This is achieved with a compression ratio of 2:1.
The benefit of AME varies based on workload. Hence, the workload has to be monitored for some period to determine the memory expansion factor. The Active Memory Expansion Planning and Advisory Tool, amepat, assists in planning the deployment of a workload in the Active Memory Expansion environment and provides guidance on the level of memory expansion a workload can achieve.
Factors like data compressibility, memory usage and access patterns decide the benefit of AME which are not commonly known. The amepat tool considers these factors and generates a report with projection of expansion factors. Hence, it is recommended to run amepat for guidance before enabling AME.
The amepat tool reports AME information and statistics and helps to monitor the effects of enabling AME for a workload. The amepat tool helps to plan an initial AME configuration and monitor and fine tune an active AME configuration. The amepat tool can be invoked when AME is both enabled or disabled.
The amepat tool should be run at a workload's peak utilization period to ensure that the tool captures the utilization and memory usage at a workload's peak. Once the tool is invoked, it starts gathering metrics periodically to determine the compressibility. This tool reports projections of various expansion factors with CPU overhead at the end of command execution. This helps customers to decide what would be the optimal expansion factor based on their current compression ratio and CPU overhead.
This command is available from AIX 6.1 TL4 SP2 and can be run on the older system starting from POWER4 to estimate the workload for use on the POWER 7 system with AME. The amepat tool can either be invoked through command line or smit.
Consider an example, where amepat is run on a workload's peak utilization time for 3 hours to plan AME.
The amepat tool monitors the system and generates a report after 3 hours. There are two sections in the generated report: system statistics and AME statistics.
System configuration and resource statistics
Figure 1. The amepat report - command information, system configuration and system resource statistics
The Command Information Section provides details about the arguments passed to the amepat tool, time of invocation, the total time monitored, and the number of samples collected. For example, if amepat is invoked as amepat 2 3, then the total monitored time would be six minutes and three samples would be collected in interval of 2 minutes. If the sample value is not mentioned, amepat calculates the interval and samples automatically. The actual monitoring time can be higher than the duration specified based on the workload.
The System Configuration Section provides details about the system configuration. This provides details about partition name, processor type, processor implementation mode, number of logical CPUs, processor entitled capacity (capacity entitlement of the partition, represented in the unit of number of physical processors), processor max capacity, SMT threads (this can be 1, 2 or 4), shared processor mode (the possible values are Disabled, Enabled-Capped, Enabled-Uncapped), Active Memory Sharing (indicates whether Active Memory Sharing is enabled or disabled), Active Memory Expansion (indicates whether Active Memory Expansion is enabled or disabled).
When AME is enabled, the following two fields are also displayed:
- Target expanded memory size in MB for the LPAR
- Target memory expansion factor configured for the LPAR
System resource statistics provides details about the system resource utilization from CPU/memory standpoint. Minimum and maximum values get displayed when duration/interval value is specified.
This gives details about:
- A partition's CPU utilization in the units of number of physical processors
- The percentage of CPU utilization against the maximum capacity
- Active virtual memory size in MB
- The percentage virtual memory size against the true memory size
- The amount of the LPAR's real physical (or logical) memory in MB
- The percentage of true memory size against the true memory size/expanded memory size
- The pinned memory size in MB
- The percentage of pinned memory size against the true memory size
- The non-computational file cache size in MB
- The percentage of file cache size against the true memory size
- The size of the memory available in MB for application execution
- The percentage of size of the memory available against the true memory size/expanded memory size
- AME statistics and modeled statistics
- AME modeled statistics provides details about the modeled statistics for AME. This displays details about the size of expanded memory that is used to produce the modeled statistics.
- The average compression ratio of the in-memory data of the workload
- The modeled target memory expansion factor
- The modeled true memory size (real physical or logical memory)
- The amount of memory the partition can gain by enabling AME for the reported modeled expansion factor
- The percentage of modeled memory gain against the target expanded memory size
- The amount of processing capacity in units of physical processors that would be used for memory compression activity.
- The percentage of utilization against the maximum capacity
In this example, the fifth row indicates that 1.25GB of memory can be saved by running the workload in an LPAR with 2.75GB of physical memory and a memory expansion factor of 1.54. The estimated CPU usage for this activity is 0.07.
Figure 2. The amepat report - AME modeled statistics and recommendation
When AME is enabled, amepat displays the AME statistics. This provides details about:
- The CPU utilization for AME activity in units of physical processors
- The percentage of utilization against the maximum capacity
- The total amount of virtual memory in MB that is compressed
- The percentage against the target expanded memory size
- Compression Ratio
- The size of the expanded memory in MB deficit for the LPAR,if the LPAR has a memory deficit.
- The percentage against the target expanded memory size,if the LPAR has a memory deficit.
Figure 3. The amepat report - AME statistics section
Recommendation provides details about the AME configuration that would provide optimal benefits to the current running workload.amepat selects one of the configuration from modeled table as recommendation. Any of the configurations listed are valid and can be used an initial configuration to try.
Customizing amepat report
amepat analyzes the system configuration and performance statistics, collected in real time or from the user specified recording file, to generate workload utilization and planning reports.
There are options in amepat to customize the AME modeled statistics.
When amepat is run without options,it generates report with snapshot of system configuration and AME performance metrics (if AME is enabled).
To generate an AME report for workload planning with modeled memory expansion factors between 1.2 and 1.8 at 0.1 incremental factor, the -e option can be used. This accepts input in the following format:
startexpfactor [ :stopexpfactor [ :incexpfactor ] ] amepat -e 1.2:1.8:0.1 2
Figure 4. Using amepat with the modeled memory expansion factors
To generate an AME report for workload planning with capping the modeled AME CPU usage to 30%, and starting modeled memory gain of 800 MB – c and –m options are used.
amepat -c 30 –m 800 2 3
Figure 5. Using amepat with the -c and –m flags
To generate an AME report for workload planning by modeling a minimum uncompressed pool size 700 MB with only 3 entries in the modeled table, -u and –n options are used.
amepat –u 2000 –n 3 1
Figure 6. Using amepat with the -u and -n flags
To generate an AME report with the modeled target expanded memory size as 5000 MB. –t option can be used.
amepat –t 5000 10
Figure 7. Using amepat with the modeled target expanded memory size
To generate an AME report estimated based on the current memory usage of the workload excluding the available memory size, -a option can be used.
To generate an AME report by disabling AME modeling with log file, -N and –v options are used.
amepat –v –N
Figure 8. Using amepat with the –v and –N flags
The verbose log also contains detailed information on all the samples collected. This would be useful to find the statistics for a particular time when amepat is run for long period with workload getting changed frequently. It also logs paging space in and outs, file page in and outs, candidate memory and memory references.
Figure 9. The amepat log file
The amepat tool – Recording mode
In the recording mode, amepat records systems configuration and various performance statistics into a user specified recording file. The recorded data can be post processed later using the -P option.
Whenever a workload's utilization level goes high, the -R option can be used to record memory and CPU usage. This recorded file can be further processed as per requirements using various command options and expansion factor can be set based on the report.
To generate the recording file for 60 minutes, -R option can be used.
amepat -R myrecord_amepat 60
To generate a report with the modeled target expanded memory size as 5000MB using the recorded file, use this code:
amepat –t 5000 -P myrecord_amepat
To generate a report for workload planning with the modeled memory expansion factors ranging between 2 to 4 with 0.5 delta factor, use this code:
amepat -e 2.0:4.0:0.5 -P myrecord_amepat
- Learn more about IBM Active Memory Expansion
- This AME white paper provides an overview of POWER7's Active Memory Expansion technology