A tour of the hardware in IBM PureApplication System: The second generation
IBM PureApplication System is a cloud computing system-in-a-box, with integrated hardware and software to deploy and execute workloads in a cloud - in other words, everything an enterprise data center needs to add a private cloud environment.
This article, specific to IBM PureApplication System software v1.1, explains the changes introduced in the second generation of system hardware. It fits in with a group of previously published articles that together describe the hardware and software foundation that PureApplication System provides for hosting application workloads. These articles cover:
- Hardware: A tour of the hardware in IBM PureApplication System explains the hardware included inside PureApplication System. This current article augments that earlier one with details for the second generation hardware. It doesn’t explain all of the hardware in the system, but rather focuses on what has changed since the first generation.
- Virtualized hardware: Best practices using infrastructure as a service in IBM PureApplication System explains how PureApplication System virtualizes its hardware to implement infrastructure as a service (IaaS).
- Cloud environment: Managing application runtime environments in IBM PureApplication System explains how to design the cloud environment that hosts workloads, which is built on PureApplication System’s virtualized hardware.
Each article builds on its predecessor to fully explain this foundation.
The second generation of system hardware became available in June 2014. In a nutshell, here is what is different in this new generation:
- The second generation hardware introduces two new architecture
- W2500 (Intel®)
- W2700 (Power®)
- An Enterprise system now has two chassis, not three, for a maximum of 384 cores.
- In the W2700, the IBM PureFlex® System Manager is now virtual and runs as a service on the PureSystems® Manager
- A Mini system is now housed in a 42U tall rack, not the shorter 25U rack, yet still contains one chassis.
- A second generation Enterprise compute node contains twice as much memory as its first generation counterpart.
- All second generation compute nodes (Enterprise and Mini) contain faster processors and/or faster memory than their first generation counterparts.
- Second generation compute nodes can be added to first generation systems.
Let’s review each of these changes in detail.
System families and classes
A system’s workload hosting environment is determined by its architecture. PureApplication System, both first and second generations, is offered in two architectures, as shown in Table 1.
Table 1. PureApplication System architectures
|System architecture||Machine type||Processor||Hypervisor software||Workload operating system|
|Intel||8283||Intel Xeon||VMware vSphere Hypervisor (ESXi)||Red Hat Enterprise Linux (RHEL)|
|Power||8278||IBM POWER7+||IBM PowerVM||IBM AIX|
A system’s workload type and capacity is determined by its family, class, and size. PureApplication System offers two new families that embody the second generation of the two architectures (Table 2).
Table 2. PureApplication System families
|Generation||Intel Family||Power Family|
Like the first generation, the second generation hardware consists of two classes of system. As shown in Table 3, the classes are the same but the contents of each class have evolved somewhat.
Table 3. PureApplication System second generation classes
|System class nickname||Chassis||CPU cores (system size)|
|Mini||One chassis||32, 64, 96, or 128|
|Enterprise||Two chassis||32, 64, 96, 128, 160, 192, 224, 320, or 384|
A system’s size describes the number of CPU cores in the system. The size of each family is upgradeable within its class in 32-core increments to a maximum size of 128 and 384 cores, respectively. Upgrades to the size of the system can be performed without taking an outage, as can upgrades to the system software.
Table 4 shows a quick comparison of the hardware in each family and class.
Table 4. PureApplication System second generation hardware family and class comparison
|Component||W2500 Mini||W2700 Mini||W2500 Enterprise||W2700 Enterprise|
|Rack||42U - 2.0 M 19” Enterprise Rack|
|Node chassis||1 Flex System Chassis||2 Flex System Chassis|
|Processor||8 core, 2.6 GHz Intel Ivy Bridge EP||8 core, 4.1 GHz POWER7+||8 core, 2.6 GHz Intel Ivy Bridge EP||8 core, 4.1 GHz POWER7+|
|Compute nodes||2 , 4, 6, or 8||21, 2, 3, or 4||2, 4, 6, 8, 10, 12, 14, 20, or 24||21, 2, 3, 4, 5, 6, 7, 10, or 12|
|CPU cores||32, 64, 96, or 128||32, 64, 96, 128, 160, 192, 224, 320, or 384|
|Memory||0.5, 1.0, 1.5, or 2.0 TB RAM||1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 10.0, or 12.0 TB RAM|
|Storage nodes||1 V7000 Controller|
1 V7000 Expansion
|2 V7000 Controllers|
2 V7000 Expansions
|Storage drives||6 x 400 GB 2.5 in. SSDs|
40 x 600 GB 2.5 in. HDDs
|16 x 400 GB 2.5 in. SSDs|
80 x 600 GB 2.5 in. HDDs
|Storage capacity||2.4 TB (1.6 TB usable) SSD|
24.0 TB (21.6 TB usable) HDD
|6.4 TB (4.8 TB usable) SSD|
48.0 TB (43.2 TB usable) HDD
|Management nodes||2 PSMs|
|2 PSMs||2 PSMs|
|Network||2 IBM RackSwitch G8264 64-port 10 Gb Ethernet switches|
|Power2||4 PDUs: 30A 1ph or 32A 1ph||4 PDUs: 60A 1ph or 32A 3ph|
1 About the smallest W2700 systems: As Table 4 shows, the two smallest sizes in the W2700 family (Mini and Enterprise) have two compute nodes, both for the 32-core size and the 64-core size. In the 32-core models, only half of the hardware in each W2700 (and W1700) compute node is functional. This enables the system to contain two compute nodes for redundancy and yet provide half the computational capacity of a 64-core model.
2 About the PDUs: Additional line cord options are available. For example, the Enterprise class systems also support 60A 3ph.
The naming convention for a system model specifies the system’s family, its size, and its class. For class, a Mini system’s name appends an “m” after the number of cores, whereas the name for an Enterprise system does not append a letter. IBM uses a 7-digit machine type model (MTM) number to uniquely identify each model. The first four digits are the machine type, which identifies the product and architecture: 8283 for Intel and 8278 for Power, regardless of generation. The last three digits are the model number, which identifies the system’s size, class, and generation. By knowing the MTM for a system, an IBM service technician knows all of the components in the rack and each one’s position, including the position of each management and compute node within the chassis. Table 5 shows some examples of how systems are named and numbered.
Table 5. Examples of PureApplication System model names and numbers
|Model name||MTM number||Description|
|W2500-32||8283-2A3||An Enterprise system with 32 Intel cores|
|W2500-32m||8283-12Y||A Mini system with 32 Intel cores|
|W2700-128||8278-2D3||An Enterprise system with 128 Power cores|
|W2700-128m||8278-18Y||A Mini system with 128 Power cores|
|W2500-384||8283-2L3||An Enterprise system with 384 Intel cores|
|W2700-384||8278-2L3||An Enterprise system with 384 Power cores|
The second generation of PureApplication System consists of twenty-six different models, plus custom sizes. The main differences between two models with the same family and size but different classes is that the Mini has less storage and a lower maximum size.
Anyone familiar with the first generation hardware in PureApplication System will recognize the components in the second generation:
- Flex chassis containing management nodes and compute nodes.
- Storage nodes.
- Top of rack switches (ToRs).
- Service terminal.
- Power distribution units (PDUs).
However, the second generation hardware makes some adjustments to the ratio of components, their placement in the rack, and — for some classes of system — the size of the rack itself. It also makes adjustments to the contents of a compute node.
Figure 1 illustrates the layout of the hardware components in a W2500 Enterprise system.
Figure 1. IBM PureApplication System W2500-384 hardware
Much about the second generation enterprise class system hardware is equivalent to that in the first generation enterprise class:
- Cabinet: The cabinet that houses the system is the same size: A PureFlex System 42U Rack, 42 rack units (2.0 meters) tall for components 19 inches wide.
- Node chassis: These are still IBM Flex System Enterprise Chassis Type 7893 chassis.
- LAN network: Similar LAN hardware provides the same bandwidth: IBM System Networking RackSwitch G8264 64-port 10 Gb Ethernet switches and 66-port IBM Flex System Fabric EN4093R 10Gb Scalable Switch Ethernet switches.
- SAN network: The same SAN hardware provides the same bandwidth: 24-port IBM Flex System FC5022 16Gb ESB SAN Scalable Switch SAN switches.
- Storage: The IBM Storwize V7000 storage units are
newer models that contain the same capacity as before:
- Hard disk drives: 48.0 TB (43.2 TB usable) HDD
- Solid state drives: 6.4 TB (4.8 TB usable) SSD
- Power: Four PDUs provide the same amount of power and redundancy.
As always, every hardware component is redundant to avoid a single point of failure.
The biggest change between the generations of systems is the amount of computational capacity. The chassis are still Flex System chassis, but a second generation Enterprise class system now has two chassis, not three. This lowers the maximum number of compute nodes and thus the maximum number of CPU cores a system can contain, from 608 cores to 384 cores. Fewer compute nodes means a more favorable ratio of storage and network bandwidth per core.
The layout of the components within the rack has changed. Chassis 1 and 2 are still in the same place, but with no Chassis 3, the storage modules have been moved down, which lowers the rack’s center of gravity. Because the ToRs are still at the top of the rack (as their name implies), this creates unused space in the rack between the storage modules and the system network switches. While there are still two storage modules, each containing a controller node and an expansion node, the order of the nodes has been reversed to put each controller node on top of its corresponding expansion node.
The service laptop has been replaced with a service terminal. The service terminal occupies the same drawer that the service laptop did. It connects to the virtual service console, a service that runs in the PSM. As always, this is still for IBM’s use, to administer the system.
Figure 2 illustrates the layout of the hardware components in a W2700 Enterprise system.
Figure 2. IBM PureApplication System W2700-192 hardware
The system-level components in a W2700 Enterprise system are positioned the same as in a W2500 Enterprise system. Like in the W1700, the W2700 compute nodes are twice as wide, so a given number of cores is provided by half the number of nodes. This illustration also shows that the distribution of the compute nodes amongst the chassis has changed in the second generation hardware. While the PureSystems Manager management node still occupies Bay 2 of both chassis, the compute nodes are no longer evenly distributed amongst the chassis. Rather, compute nodes in both the W2500 and W2700 fill Chassis 1 completely first before starting to fill Chassis 2. As Figure 2 shows, the six compute nodes occupy all of the available bays in Chassis 1, while the compute node bays in Chassis 2 remain empty. Because Chassis 1 is full before any compute nodes are added to Chassis 2, and because each chassis is filled from the bottom up, this lowers the center of gravity within the rack.
The W2700 also makes a change to the virtualization management nodes. While the PureSystems Manager still exists, the second generation hardware no longer requires a separate management node for the PureFlex System Manager. Rather, the PureFlex Systems Manager has now become a virtual PureFlex Systems Manager, a service that runs in the PureSystems Manager. The Virtualization System Manager on the W2500 is still physical: a management node housed in Bay 1 in Chassis 1 and 2, as shown in Figure 1. But the PureFlex Systems Manager in the W2700 is virtual, so Bay 1 in Chassis 1 and 2 are empty, as shown in Figure 2. The virtual PureFlex Systems Manager was introduced in the W1700 Mini; it contains two physical PureSystems Manager management nodes (in Bays 2 and 4) but no physical PureFlex Systems Manager management nodes (Bays 1 and 3 are empty). This virtual PureFlex Systems Manager design is preserved in the W2700 Mini and is now introduced to the W2700 Enterprise.
Figure 3 illustrates the layout of the hardware components in a W2500 Mini system.
Figure 3. IBM PureApplication System W2500-128m hardware
As with the first generation, the second generation Mini systems contain the same types of hardware components as the Enterprise systems, just fewer of those components for lower overall capacity and cost. The chassis are the same, but only one instead of two. The LAN and SAN network switches are the same for the same network bandwidth. The storage is housed is a single controller/expansion node pair with just under half the capacity. The power is supplied by the same number of PDUs though each PDU has lower capacity.
Although nicknamed Minis, the smaller PureApplication System is only miniature compared to its Enterprise class counterpart. What PureApplication System calls miniature still contains 32-128 CPU cores and 0.5-2.0 TB of RAM, a 16 Gb SAN with 26.4 TB of internal storage, and a 10 Gb LAN — still enough capacity to run a number of enterprise applications.
The most noticeable change between the first and second generation Minis is the size of the rack. Whereas the first generation Mini is housed in a short rack (25U or 1.3 meters tall), the second generation Mini is housed in the same tall rack as the Enterprise class systems (42U or 2.0 meters tall). A shorter rack can suffer airflow issues that the taller rack corrects. With the taller rack, from the outside, a Mini class system doesn’t look very "miniature."
What makes the Mini smaller is the number of chassis and number of storage modules compared to the Enterprise. Like the first generation Mini, the second generation Mini contains a single chassis, fewer than the two chassis in the second generation Enterprise. Even though the second generation Mini is now housed in a large rack, the components still occupy the same positions as they did in the first generation rack. The single storage module is positioned above the chassis, and the service terminal drawer and the ToRs are positioned in rack unit 22 and rack units 24-25 respectively. Like the Enterprise class systems - and unlike the first generation Minis - the PDUs in the second generation Mini systems are positioned in the sides of the rack. Therefore, the rack now contains unused space below and above the system’s network switches and service terminal.
The component type that has changed the most from the first to second generation of PureApplication System hardware is the compute nodes. Compared to the first generation, the second generation compute nodes contain:
- Faster CPU chips and/or faster memory.
- Twice as much memory (for the Enterprise class systems).
Let’s look at the details.
W2500 compute node
Like the W1500 system, the W2500 system contains Intel compute nodes, specifically the IBM Flex System x240 compute node. The new compute node model is still called x240, but it contains newer chip sets.
The second generation Intel compute node contains faster CPU chips, as shown in Table 6.
Table 6. Intel compute node CPU chips
|First||2 CPUs||Sandy Bridge||8 core, 2.6 GHz Intel Xeon Processor E5-2670 (115 W)|
|Second||2 CPUs||Ivy Bridge||8 core, 2.6 GHz Intel Xeon Processor E5-2650 v2 (115 W)|
While their clock speeds are the same, the newer Intel chip offers better performance, is smaller, and uses less power.
Table 7 shows the memory in an Intel compute node. The second generation Intel compute node contains faster memory chips. The compute nodes in the Enterprise class systems contain twice as much memory.
Table 7. Intel compute node memory
|First||256 GB of RAM||8 2x16 GB, 1333 MHz, DDR3, LP RDIMMS (1.35 V)|
|Second (Mini)||256 GB of RAM||16 1x16 GB, 1866 MHz, DDR3, LP RDIMM (1.5 V)|
|Second (Enterprise)||512 GB of RAM||16 1x32 GB, 1866 MHz, DDR3, LP LRDIMM (1.5 V)|
W2700 compute node
Like the W1700 system, the W2700 system contains Power compute nodes, specifically the IBM Flex System p460 compute node. The new compute node model is still called p460, but it contains newer chip sets. As in a W1700, a W2700 compute node is twice as wide as a W2500 compute node and essentially contains twice as much hardware: twice as many CPUs that provide twice as many cores, twice as much memory, and twice as many LAN and SAN adapter cards that provide twice as many ports.
The second generation Power compute node contains faster CPU chips (Table 8).
Table 8. Power compute node CPU chips
|First||4 CPUs||8 core, 3.61 GHz POWER7+ (190 W)|
|Second||4 CPUs||8 core, 4.116 GHz POWER7+ (190 W)|
Table 9 shows the memory in a Power compute node. The memory chips in the second generation Power compute node run at the same clock speed as those in the first generation. The compute nodes in the Enterprise class systems contain twice as much memory.
Table 9. Power compute node memory
|First||512 GB of RAM||16 2x16 GB, 1066 MHz, DDR3, LP RDIMMS (1.35 V)|
|Second (Mini)||512 GB of RAM||16 2x16 GB, 1066 MHz, DDR3, LP RDIMMS (1.35 V)|
|Second (Enterprise)||1.0 TB of RAM||16 2x32 GB, 1066 MHz, DDR3, LP RDIMMS (1.35 V)|
Expanding a first generation system
What if you already own a first generation system? There’s good news for you also.
Second generation compute nodes can be added to first generation systems. The chassis are the same between generations; the compute node case remains the same, and so a second generation compute node fits in a first generation chassis just fine and works just fine. For a Mini system, a second generation compute node contains faster CPU and/or memory. For an Enterprise system, a second generation compute node contains faster chips, but also - and more importantly - twice as much RAM.
A second generation compute node has a few prerequisites. Expanding a system with additional compute nodes, whether first or second generation, must be performed by an IBM Customer Engineer. The firmware installed in a second generation compute node is part of system software v220.127.116.11, which means that before a second generation compute node can be installed in a first generation system, the system must be upgraded to Version 18.104.22.168 Interim Fix 1 (or later).
For a system that contains both first and second generation compute nodes, the best plan is to separate them using separate cloud groups. A cloud group tends to assume that its compute nodes are homogeneous, so a cloud group that is all first generation compute nodes and another that is all second generation compute nodes will work best. Perhaps use one for development and another for production.
If you mix first and second generation compute nodes into a single cloud group on an Enterprise system, there are some issues to be aware of.
- Placement: During the pattern deployment process, the placement engine determines which compute node in which to instantiate each virtual machine. The placement engine in system software v22.214.171.124 hasn’t been optimized for heterogeneous cloud groups. It expects a cloud group’s compute nodes to be homogeneous; in a heterogeneous cloud group, its usage of the extra memory might not be as optimal. This is especially true for a dedicated type cloud group because it doesn’t overcommit CPU; average type cloud groups are better at reserving all available memory.
- Reserve resources: The reserve resources for availability feature will cause the extra memory to be ignored because it plans for the worst case scenario by setting all reservations equal to the lowest common denominator.
You can avoid this latter issue by turning off reserve resources, although that has negative implications for the high availability of that cloud group. If you make the cloud group’s nodes homogeneous, you can enable reserve resources and still avoid both issues.
This article reviewed the second generation of IBM PureApplication System hardware, focusing on differences from the first generation. It explained what hardware components are contained in the various models, and the hardware improvements inside the compute nodes. With this information, you now have a better understanding of the hardware included in your new PureApplication System.
The author thanks fellow IBMers Nik Teshima, Brad Crater, Jim Robbins, David Rainey, Jose De Jesus, and Bob Ringo for their help with this article.
- IBM PureApplication System
- IBM Knowledge Center
- IBM PureSystems Centre
- IBM PureApplication System resources on developerWorks
- IBM PureApplication System W2500 and W2700 next-generation hardware includes performance improvements and drives greater flexibility
- Adoption Scenarios for IBM PureApplication System
- A tour of the hardware in IBM PureApplication System
- Best practices using infrastructure as a service in IBM PureApplication System
- Managing application runtime environments in IBM PureApplication System
- Achieving business continuity in IBM PureApplication System
- IBM PureApplication System backup and restore, Part 1: Recovery scenarios