By: Anirban Chatterjee.
month, the PowerLinux team is announcing the biggest technology change in PowerLinux servers
since we launched, with the availability of our POWER7+ chips on the platform.
POWER7+ is more than just a speed bump on our POWER7
processors. Our hardware teams have
worked hard to increase the flexibility of the platform, bringing
balanced performance increases while keeping other factors like energy
consumption at bay. Some examples:
doubled the memory capacity in servers like the 7R1 and 7R2. We’ve also doubled the number of virtual
machines you can allocate to a single processor core. This means we’ve dramatically increased
the system’s flexibility when it comes to deploying virtualized workloads
… in many cases, this will eliminate memory as the gating factor, allowing
users to drive utilization rates even higher and boost system efficiency.
reduced the feature size in the chips from 45 nm to 32 nm. This not just a simple die shrink,
though … with every shrink, the chip team has to work even harder to
ensure the computational and thermal stability of the chip while driving
higher clock speeds. In PowerLinux
servers like the 7R2, the new chips now top out at 4.2 GHz.
- Because we have more available chip real estate now that we’ve shrunk the die
size, we’ve bumped up the L3 cache from 4 MB to 10 MB. This significantly boosts performance in
workloads that are memory dependent, like Java and big data applications.
feature additions to POWER7+ allow us to improve chip reliability and
boost energy savings. We’ve added
self-healing capabilities and automatic processor reinitialization to
increase system robustness, and we’ve introduced a new energy saving mode
that saves 45% more energy than before when the processor is idle.
The new performance capabilities afforded by POWER7+ enable
some pretty interesting possibilities when it comes to reducing costs. For example, we’ve found that people
typically need just two dual-socket (16 core) PowerLinux 7R2s to do what it
would take three dual-socket (16 core) Xeon servers to do. Given the already competitive pricing on the
7R2s, this means that you can potentially save north of 40% on your costs of acquisition by choosing PowerLinux.
These changes make PowerLinux an ideal platform for the most
critical workloads your business runs today, like your customer facing web
applications, or your ERP system.
Customers like Kwik Fit (PDF) and IT Informatik (PDF) are already realizing the benefits. Click the links to read the
case studies on these customers.
But PowerLinux is also a great platform for today’s growth
workloads, like development and deployment of mobile and web applications. To make it easier for businesses to create
and launch these types of client experiences, we’re introducing a new solution for WebSphere mobile and web applications that leverages the lightweight
WebSphere Liberty Profile software. This
is a light, easily reconfigured web app environment that makes it simple for
developers to test and deploy applications.
As 2013 progresses, we'll continue to bring you more
announcements that improve the PowerLinux platform's ability to reduce costs while improving
efficiency, enabling new and growth workloads, and giving you a better overall
Modified on by jerberstark
By: Breno Leitão.
This tutorial explains how to create a RAID device on PowerLinux machines using an array of disks. This step by step tutorial includes identifying the disks, formatting them, combining them in a RAID array, creating a partition and, finally, creating a file system on this partition.
The PowerLinux machines support a RAID (Redundant Array of Independent Disks) card. A RAID card is a device that combines a set of physical disks into a logical unit to achieve a better performance and more data redundancy. A RAID array could also be created by the operating system (known as Software-based RAID), and it consumes some CPU cycles from the machine to manage and control the disk array. On the other side, a RAID card, as the one embedded on PowerLinux machines, offers a Hardware-based RAID, meaning that the operations on the disk array are offloaded to the RAID card, not utilizing CPU cycles managing the disks, thus, being more efficient than Software-based RAID solutions.
The RAID adapters on PowerLinux machines support several different RAID protection levels. Depending of the protection level, you might have different benefits, as potentially achieving a higher data transfer, a smaller latency and data redundancy when compared to a single big disk. You might also want to combine these benefits all together in the same disk array, which is also a possible depending on the RAID protection level.
Using RAID is usually a trade-off between disk space and redundancy, so, depending on the RAID protection level, part of the disk space is used to save redundant data, thus, part of the disk space is not available for general usage. The real space available to the users varies from 50% to 100% of the total disk space.
The RAID protection levels supported by most PowerLinux RAID adapters are:
RAID 0: On this configuration, a block of data is striped in different disks on the array, so, the read/write operations on the disks could happen in parallel on the disks in the array. On this configuration, there is no fault tolerance i.e., if a disk fail, the whole data is lost. This level usually improves the data throughput.
Requires at least 1 disk. In a single disk RAID 0 configuration, no striping occurs.
RAID 1: On this level, the data is written at the same time on 2 disks. As both disks have the same data, a read operation will occur on the disks that has the smaller latency. On this case, if one disk fails, the whole data will continue be preserved on the other disk. Once the other disk is replaced, the RAID would be reconstructed. As expected, this level improves the data redundancy.
Requires at least 2 disks. Note: The PowerLinux RAID adapters refer to this RAID level as RAID 10.
RAID 5: On this level, the data and parity bits are spread all across the disks. If one disk fails, then all the data is still available, once the original data could be reconstructed using parity data from the other disks. If more than one disk fails, then the data will be corrupted. (If a disks fail happen, the operations may happen in a slower fashion, since the data being accessed is on the lost disk, then the data will need to be reconstructed.). One disk's worth of capacity is consumed for redundancy information for the array.
Requires at least 3 disks.
RAID 6: The same as RAID 5, but up to 2 disk can fail, and the data will still be preserved. Two disk's worth of capacity is consumed for redundancy information for the array.
Requires at least 4 disks.
RAID 10: This level combines the best concepts of RAID 1 and RAID 0. On RAID 10, the data is striped on a set of hard disks, and these hard disks are mirror to another set of hard disks. So, you have a very good throughput and also a data redundancy.
Requires at least 2 disks for mirroring and striping. In a two disk RAID 10 configuration, no striping occurs.
RAID card on PowerLinux
The PowerLinux machines come with an embedded RAID controller that supports up to 6 SAS or SDD disks and RAID levels 0, 5, 6 and 10 on machines 7R1
. RAID 1 is also supported as a subset of RAID 10, since the RAID controller allows you do create a RAID 10 with just two disks as part of the array. In this case, since there is no enough disk to mirror and strip, the data just gets mirrored on both disk, instead of striped, which is what a RAID 1 does. So, the cards also supports, in a different form, a RAID 1.
On Linux, the device is listed as a PCI-E device named "IBM Obsidian-E PCI-E SCSI controller ".
In order to manage this controller, there is a set of tools on the packaged called iprutils that helps the system administrator to create, configure and delete disks and arrays using the RAID controller.
The iprutils package provides the iprconfig application. The iprconfig is the tool responsible for configuring the RAID devices on your machine, and will be the tool that will be covered below.
The device driver
The device driver for the PowerLinux RAID controller is named ipr.ko. It's currently part of the Linux kernel and comes with all the supported Linux Distros for PowerLinux. So, it's recommended to always use the last supported kernel version from the distro in order to take the best from you PowerLinux machine.
Using iprconfig tool
The iprconfig tool is a very easy application to use. It's a text-based (TUI) application that helps you to list and configure the RAID controller and the disks on your system. iprconfig also allows you to check the controller log, and upgrade the card firmware. An example of the iprconfig screen could be seen at Figure 1.
Now on, a step-by-step tutorial will show how to format a set of disks, combine them together in a RAID mode, create a partition over this array and, then, create a file system over this partition. This is an easy process that might take less than 30 minutes to be accomplished. For this tutorial, we are going to create a RAID 5 device, meaning that the array will have the data mirrored and striped over the disks arrays.
You can use iprconfig just as a command line option, in this case, you need to pass the parameters you want in the command line. For example, in order to see what are the RAID levels supported on a controller (sg5
), the following command should be used:
# iprconfig -c query-supported-raid-levels sg5
0 5 10 6
Formatting a disk in RAID mode.
In order to use a disk as part of a disk array, it needs to be formatted specifically for being part of RAID, also known as, advanced function
mode. If the disks is not formatted properly, you can not add it to a RAID array. In order to format a disk in advanced function mode, the following steps should be followed:
Launch the iprconfig tool on a console.
Select the menu Work with disk array (as shown in Figure 1)
In order to do it, press 2 and then enter.
Select Format device for RAID function
In order to do it, press 5 and then enter.
Then select the disks you want to format (all of those that are going to take part of the disk array) and continue. (As shown in figure 2)
In order to select the disks, you must use the up/down arrows, and press 1 to select the devices you want to select.
Wait until the disks are formatted as shown in Figure 3. (It takes some minutes until the disks are formatted.)
Figure 3: Disks being formatted
Creating an RAID disk array
As explained above, in order to add a disk into the RAID array, the disk need to be formatted in RAID mode. Once the disks are formatted in RAID mode, they can be available to be added to a disk array, and the RAID device could be created. You might want to create as many RAID devices you want, and give them a set of disks. Let' s go through the process of creating an array device. It is recommended when creating a RAID array to format the devices for RAID as described in the previous step, then create the disk array following the steps below without exiting the iprconfig
tool. Exiting the tool will result in the loss of knowledge that the disks have just been zeroed and it will take longer for the array to initialize.
Launch the iprconfig tool.
Select Work with disk arrays, as shown in Figure 1.
Select Create a disk array.
Select the controller you want (You might have just one controller), as shown in Figure 4.
Select the disk that will be part of the array, as shown in Figure 5
Press '1' over the disks you want to select.
Select the RAID type, as shown in Figure 6.
Go into the Protect Level and press 'c' to change the RAID level.
Select the disks that will take part of that disk array
It may take a while to have the array created.
Figure 4: Selecting the RAID controller
Figure 6: Select the RAID type
Start using the RAID array
Once you had the RAID array created, it becomes a block device as any other block device on the system. You can create a partition on the device, make a file system on it, and start using. The next steps will help you to create a partition and a file system on the array partition. On this tutorial, I will create just one partition using the whole array and format it using EXT4 file system, as shown below:
Figure 8: Creating a EXT4 file system on the partition over the RAID device
Once the file system is created in the partition, you can use this partition as a traditional file system, i.e, mount it on a directory and start using it. All the RAID operation, as managing, striping, checksumming or mirroring the data will be offloaded to the RAID card, and happen transparently.
By: Timothy Noonan.
Recently, Red Hat announced that Red Hat Enterprise Linux 6 conforms with the USGv6 Host profile. See Red Hat Ready to Serve U.S. Government with IPv6 Conformity. Likewise, we can assert that IBM POWER7 Systems™, as well as SUSE Linux Enterprise Server 11, have met the National Institute of Standards and Technology’s USGv6 evaluation requirements.
Why is this important? U.S. government agencies are migrating to Internet Protocol Version 6 (IPv6), as the pool of IPv4 addresses is being depleted. The move to IPv6 also allows scalability of government networks to take advantage of new technologies such as cloud computing. The National Institute of Standards and Technology (NIST) provides the technical standards and testing program, USGv6, to certify products as conforming to IPv6. This certification is required to be considered by the U.S. government for new IT purchases. POWER7 Systems, Red Hat Enterprise Linux 6, and SUSE Linux Enterprise Server 11 conform with the USGv6 profile, making PowerLinux™ a trusted platform.
To achieve conformity, IBM POWER7 Systems, Red Hat Enterprise Linux 6, and SUSE Linux Enterprise Server 11 underwent rigorous testing by the University of New Hampshire’s InterOperability Laboratory (UNH-IOL). UNH-IOL is one of two accredited third-party labs approved for USGv6 testing. Testing included addressing and security protocol requirements. For specific UNH-IOL test results for POWER7 Systems, Red Hat Enterprise Linux 6, and SUSE Linux Enterprise Server 11, see the following:
· IBM: https://www.iol.unh.edu/services/testing/ipv6/usgv6tested.php?company=2443&type
· Red Hat: https://www.iol.unh.edu/services/testing/ipv6/usgv6tested.php?company=6164&type
· SUSE: https://www.iol.unh.edu/services/testing/ipv6/usgv6tested.php?company=105&type
By: Jessica Erber-Stark.
Mel Beckman at PowerITPro recently published a very nice overview of IBM PowerLinux systems. Mel states: "IBM’s PowerLinux
brings big-iron reliability and scalability to Linux while still
providing cost-competitive, Linux-friendly, entry-level server
Check out the article "PowerLinux Pumps Up Linux Apps
" on PowerITPro to get Mel's take on:
- Application portability to PowerLinux systems
- Virtualization performance advantages
- PowerLinux 7R2 system summary
- Big Data Analytics solution, using Hadoop
- Power systems features key to Linux users
Modified on by jhopper
zswap" is discussed, with some initial performance data provided to demonstrate the potential benefits for a system (partition or guest) which has constrained memory and is beginning to swap memory pages to disk. The technique improves the throughput of a system, while significantly reducing the disk I/O activity normally associated with page swapping. We also explore how zswap works in conjunction with the new compression accelerator feature of the POWER7+ processor to potentially improve the system throughput even more than software compression alone.
This article is a good example of the ongoing collaboration that occurs in the Linux open-source community. New implementations are proposed, discussed, debated, refined and updated across developers, community members, interested customers, and performance teams. Here on the PowerLinux technical community, we are working to highlight more of these examples of work-in-progress from the broader Linux community. These proposals are applicable to both x86 systems and Power systems, so examples shown below cover both realms.
What is zswap?
Zswap is a new lightweight backend framework that takes pages that are in the process of being swapped out and attempts to compress them and store them in a RAM-based memory pool. Aside from a small reserved portion intended for very low-memory situations, this zswap pool is not pre-allocated, it grows on demand and the max size is user-configurable. Zswap leverages an existing frontend already in mainline called frontswap. The zswap/frontswap process intercepts the normal swap path before the page is actually swapped out, so the existing swap page selection algorithms are unchanged. Zswap also introduces key functionality that automatically evicts pages from the zswap pool to a swap device when the zswap pool is full. This prevents stale pages from filling up the pool.
The zswap patches have been submitted to the Linux Kernel Mailing List
(lkml) for review, you can view them in this post
Instructions for building a zswap-enabled kernel on a system installed with Fedora 17 can be found on this wiki
What are the benefits?
When a page is compressed and stored in a RAM-based memory pool instead of actually being swapped out to a swap device, this results in a significant I/O reduction and in some cases can significantly improve workload performance. The same is true when a page is "swapped back in" - retrieving the desired page from the in-memory zswap pool and decompressing it can result in performance improvements and I/O reductions compared to actually retrieving the page from a swap device.
Using the SPECjbb2005 workload for our engineering tests, we gathered some performance data to show the benefits of zswap. SPECjbb2005 uses a Java™ benchmark that evaluates server performance and calculates a throughput metric called "bops" (business operations per second). To find out more about this benchmark or see the latest official results, see the SPEC web site
. Note that the following results are not tuned for optimal performance and should not be considered official benchmark results for the system, but rather results obtained for research purposes. We liked this benchmark for this use case because we could more carefully control the amount of active memory being used in increments.
The SPECjbb2005 workload ramps up a specified number of "warehouses", or units of stored data, during the run. The number of warehouses is a user-controlled setting that is configured depending on the number of threads available to the JVM. As the benchmark increases the number of warehouses throughout the run, the system utilization level increases. A bops score is reported for each warehouse run. For this work, we focused on the bops score from the warehouse that keeps the system about 50% utilized. We also increased the default runtime for each warehouse to 5 minutes since swapping can be bursty and a longer runtime helps to achieve more consistent results.
For these results, the system was assigned 2 cores, 10 GB of memory, and a 20 GB swap device. A single JVM was created for the SPECjbb2005 runs, using IBM Java. First, a baseline measurement was taken where normal swapping activity occurred, then a run with zswap enabled was measured to show the benefits of zswap. We gathered results on both a Power7+ system and an x86 system to observe the performance impacts on different architecture types. The mpstat, vmstat, and iostat profilers from the sysstat package were used to record CPU utilization, memory usage, and I/O statistics. We would recommend taking advantage of the lpcpu
package to gather these data points.
To demonstrate the performance effects of swapping and compression, we started with a JVM heap size that could be covered by available memory, and then increased the JVM heap size in increments until we were well beyond the amount of free memory, which forced swapping and/or compression to occur. We recorded the throughput metric and swap rate at each data point to measure the impacts as the workload demanded more and more pages.
Settting up zswap
With the current implementation, zswap is enabled by this kernel boot parameter:
We looked at several new in-kernel stats to determine the characteristics of compression during the run. The metrics used were as follows:
pool_pages - number pages backing the compressed memory pool
reject_compress_poor - reject pages due to poor compression policy (cumulative) (see max_compressed_page_size sysfs attribute)
reject_zsmalloc_fail - rejected pages due to zsmalloc failure (cumulative)
reject_kmemcache_fail - rejected pages due to kmem failure (cumulative)
reject_tmppage_fail - rejected pages due to tmppage failure (cumulative)
reject_flush_attempted - reject flush attempted (cumulative)
reject_flush_fail - reject flush failed (cumulative)
stored_pages - number of compressed pages stored in zswap
outstanding_flushes - the number of pages queued to be written back
flushed_pages - the number of pages written back from zswap to the swap device (cumulative)
saved_by_flush - the number of stores that succeeded after an initial failure due to reclaim by flushing pages to the swap device
pool_limit_hit - the zswap pool limit has been reached
There are two user-configurable zswap attributes:
max_pool_percent - the maximum percentage of memory that the compressed pool can occupy
max_compressed_page_size - the maximum size of an acceptable compressed page. Any pages that do not compress to be less than or equal to this size will be rejected (i.e. sent to the actual swap device)
failed_stores - how many store attempts have failed (cumulative)
loads - how many loads were attempted (all should succeed) (cumulative)
succ_stores - how many store attempts have succeeded (cumulative)
invalidates - how many invalidates were attempted (cumulative)
To observe performance and swapping behavior once the zswap pool becomes full, we set the max_pool_percent parameter to 20 - this means that zswap can use up to 20% of the 10GB of total memory.
The following graphs represent the SPECjbb2005 performance and swap rate for a run using the normal swapping mechanism.
Note that as "available" memory is used up around 10GB, the performance falls off very quickly (the Blue Line) and normal page swapping (the Red Line) to disk increases. The behavior is consistent both on Power7+ and x86 systems.
Power7+ baseline results:
x86 baseline results:
As you can see, performance dramatically decreased once the system started swapping and continued to level off as the JVM heap was increased.
The following graphs represent the SPECjbb2005 performance and swap rate for a run when zswap is enabled. In these cases, memory is now being compressed, which significantly reduces the need to go to disk for swapped pages. Performance of the workload (the blue line) still drops off but not as sharply, but more importantly the system load on I/O drops dramatically.
Power7+ with zswap compression:
x86 with zswap compression:
As you can see, the swap (I/O) rate was dramatically reduced. This is because most pages were compressed and stored in the zswap pool instead of swapped to disk, and taken from the zswap pool and decompressed instead of swapped in from disk when the page was requested again. The small amount of "real" swapping that occurred is due to the fact that some pages compressed poorly - which means they did not meet a user-defined max compressed page size - and were therefore swapped out to the disk, and/or stale pages were evicted from the zswap pool.
Looking at the zswap metrics for each run, we can calculate some interesting statistics from this set of runs - keep in mind the base page size is different between Power (64K pages) and x86 (4K pages), which accounts for some of the different behaviour. Also note that we set the max zswap pool size to 20% of total memory for these runs, as mentioned above - this max setting can be adjusted as needed. On Power, the average zswap compression ratio was 4.3. On x86, the average zswap compression ratio was 3.6. For the Power runs, we saw entries for "pool_limit_hit" starting at the 17 GB data point. For the x86 runs, the pool limit was hit earlier - starting at the 15.5 GB data point. For the Power runs, at most the zswap pool stored 139,759 pages. For the x86 runs, the max number of stored pages was 1,914,720. This means all those pages were compressed and stored in the zswap pool, rather than being swapped out to disk, which results in the performance improvements seen here.
POWER7+ hardware acceleration
The POWER7+ processor introduces new onboard hardware assist accelerators that offer memory compression and decompression capabilities, which can provide significant performance advantages over software compression. As an example, the system specifications for the IBM Flex System p260 and p460 Compute Nodes
mention the "Memory Expansion acceleration" feature of the processor.
The current zswap implementation is designed to work with these hardware accelerators when they are available, allowing for either software compression or hardware compression. When a user enables zswap and the hardware accelerator, zswap simply passes the pages to be compressed or decompressed off to the accelerator instead of performing the work in software. Here we demonstrate the performance advantages that can result from leveraging the POWER7+ on-chip memory compression accelerator.
POWER7+ hardware compression results
Because the hardware accelerator speeds up compression, looking at the zswap metrics we observed that there were more store and load requests in a given amount of time, which filled up the zswap pool faster than a software compression run. Because of this behavior, we set the max_pool_percent parameter to 30 for the hardware compression runs - this means that zswap can use up to 30% of the 10GB of total memory.
The following graph represents the SPECjbb2005 performance and swap rate for a run when zswap and the POWER7+ hardware accelerator are enabled. In this case, memory is now being compressed in hardware instead of software, and this results in a significant performance improvement. Performance of the workload (the blue line) still drops off, but even less sharply than the zswap software compression case, and the system load on I/O still remains very low.
Power7+ hardware compression:
As you can see, the swap (I/O) rate was dramatically reduced. This is because most pages were compressed using the hardware accelerator and stored in the zswap pool instead of swapped to disk, and taken from the zswap pool and decompressed in the hardware accelerator instead of swapped in from disk when the page was requested again. The small amount of "real" swapping that occurred is due to the fact that some pages compressed poorly - which means they did not meet a user-defined max compressed page size - and were therefore swapped out to the disk, and/or stale pages were evicted from the zswap pool.
The following graphs show the performance comparison between normal swapping and zswap compression, and the POWER7+ graph also includes the hardware compression results, showing that the hardware accelerator provides even more performance advantages over software compression alone:
Power7+ performance comparison:
x86 performance comparison:
As you can see, this workload shows up to a 40% performance improvement in some cases after the heap size exceeds available memory when zswap is enabled, and the POWER7+ results show that the hardware accelerator can improve the performance by up to 60% in some cases compared to the baseline performance.
Swap (I/O) comparison
The following graphs show the swap rate comparison between normal swapping and zswap compression, and the POWER7+ graph includes the hardware compression results, showing that the hardware accelerator also reduces the swap rate dramatically. Swap rates are dramatically reduced on both architectures when zswap is enabled, including the POWER7+ hardware compression results.
Power7+ swap I/O comparison:
x86 swap I/O comparison:
The new zswap implementation can improve performance while reducing swap I/O , which can also have positive effects on other partitions that share the same I/O bus. The new POWER7+ on-chip memory compression accelerator can be leveraged to provide performance improvements while still keeping swap I/O very low.
By: Carlos Seo.
The IBM Advance Toolchain for PowerLinux is a set of open source
development tools and runtime libraries that allow users to take leading
edge advantage of IBM's latest POWER hardware features on Linux. A new
update release is now available, and it includes the following:
- Time zone automatically set to system's time zone.
- New advance-toolchain-at6.0-selinux rpm, which sets the correct labels for key files so the Advance Toolchain works when SELinux is enabled.
- New ldconfig wrapper (installed in /usr/sbin/) that wraps up the ldconfig of the system and all installed AT's.
- GDB for remote debugging now included in the cross compiler.
- The following fixes included in GCC:
- The following fixes included in GDB:
- The following fixes included in OProfile:
- Fix to address over-counting user context lost samples.
- Fix to record samples from forked processes.
Please let us know if you have any questions about this release.
By: Aravinda Prasad.
Problem determination is
definitely a key area when it comes to systems administrators
(sysadmins). Sysadmins tend to spend hours debugging and trying to
find out what is wrong with the system. Engineers at the IBM Linux
Technology Center around the world are working on ways to simplify the
experience of sysadmins in managing IBM systems.
An outcome of such
an effort is the upcoming facilities like Light Path Diagnostics,
improved diagnostic tools and other features in IBM PowerLinux, which
integrates well with the existing Reliability Availability and
Serviceability (RAS) capabilities. IBM believes that such facilities will help sysadmins to perform the administration tasks easily and
In this article, we emphasize the PowerLinux RAS advantage for
sysadmins in determining and resolving the problem from the
administrator point of view.
A PowerLinux sysadmin
receives a notification alert of a serviceable event. The sysadmin
knowing that the service log infrastructure of RAS on PowerLinux is
capable of sending such notifications, logs into the system and
checks the service log event using the servicelog
tool to get more details about the serviceable event. The detailed
log by servicelog mentions
that one of the Ethernet cards has gone bad, giving additional
information about the location code, device serial number etc.
firstname.lastname@example.org ~]# servicelog --dump
Servicelog ID: 27
Log Timestamp: Fri Nov 30 10:44:02 2012
Event Timestamp: Fri Nov 30 10:44:02 2012
Update Timestamp: Fri Nov 30 10:44:02 2012
Type: Operating System Event
Severity: 6 (ERROR)
Node Name: ras.ibm.com
Reference Code: BF778E00
Serviceable Event: Yes
Predictive Event: No
Disposition: 1 (Unrecoverable)
Call Home Status: 1 (Call Home Candidate)
Kernel Version: #1 SMP Wed Jun 13 18:19:27 EDT 2012
Message forwarded from syslog:
Fri Nov 30 10:44:02 ras kernel: e1000e 0001:00:01.0: Invalid MAC Address
Description: The MAC address read from the adapter's EEPROM is not a valid Ethernet
Action: 1. Execute diagnostics on the adapter, using "ethtool -t".
2. Check the EEPROM level on the failing adapter.
3. Replace the adapter.
<< Callout 1 >>
Procedure Id: see explain_syslog
The clever sysadmin,
knowing the RAS capabilities behind this entire setup, recalls that
the OS running on the PowerLinux server which detected the bad
Ethernet card has logged an error in /var/log/messages,
which was converted into service log event by syslog_to_svclog
tool by logging the event into service log database. The service log
database upon receiving a serviceable event has sent the
notification. The sysadmin also quickly recalls that the serviceable
events are not only restricted to Ethernet devices, but are also
supported on SCSI enclosures for which events are logged to service
log database by diag_encl tool
and RTAS related events, which are logged into the database by
The sysadmin orders a new
Ethernet card after collecting the required information like model,
serial number etc of the bad Ethernet card with the help of the new
-l flag to lscfg
command, which takes location code, which was logged in service log,
as input and prints VPD (Vital Product Data) information.
email@example.com ~]# lscfg -vl U78A5.001.WIH8464-P1
0001:00:01.0 eth1 ethernet U78A5.001.WIH8464-P1
Port 2 - IBM 2 PORT 10/100/1000
Base-TX PCI-X Adapter (14107910)
Machine Type-Model........82546GB Gigabit Ethernet Controller
The sysadmin is also very
happy to know that with the new light path diagnostics facility (coming in RHEL and SLES service pack updates in the future) and the
service log notifier would have notified the light path diagnostics
subsystem lp_diag about the
bad Ethernet card and the light path infrastructure would have
enabled the fault indicator for the bad Ethernet card slot helping in
easy identification of the physical location of the Ethernet card.
The sysadmin replaces the
bad part by identifying the slot with the help of fault indicator
LEDs. The hot-plug facility automatically identifies and initializes
the newly plugged card. The sysadmin closes the serviceable event
using log_repair_action tool
and the service log facility upon closure of serviceable event
notifies the light path infrastructure to turn off the fault
indicators. The sysadmin now updates the VPD using vpdupdate
tool to reflect the changes in the hardware.
The sysadmin appreciates
the RAS capabilities of PowerLinux and its seamless integration with
the OS, which helped in quickly identifying and resolving the
problem. The sysadmin checks for new notifications knowing that the
PowerLinux RAS is not just restricted to identifying faulty devices
but is capable of lot more things and provides many service and
For more information about service and productivity (aka RAS) tools for your PowerLinux system, see the related article
in the Linux Information Center.
By: Fabio Dassan dos Santos
One of the new and noteworthy features for this 5.3 release, the LPAR Cloning and Restoration tool, focuses on extending value in this category by providing a quick and easy way of creating re-usable system images across LPARs.
Through very few steps, it is possible to achieve the following with this new tool:
- Save all available devices of the LPAR in backup images;
- Use compression methods to decrease the size of the backup image;
- Store these images in a NFS server share;
- Associate previously saved images with available devices of a LPAR and restore the system;
This function is specially useful in those situations where there is a need to preserve a certain system level, or even quickly replicate system images to multiple LPARS, in a virtualized environment.
By: Anibal Carvalho
One of the
main new features of IBM® Installation Toolkit for PowerLinux version
5.3 is the support for facilitating Linux installations, on top of multipath devices.
Multipath I/O Linux layers can be enabled on IBM PowerLinux servers with multiple physical paths to storage devices (through SANs or direct attached), where they can be used to increase storage path redundancy, as well as to enhance server performance though typical features, such as:
- Dynamic load balancing
- Traffic shaping
- Automatic path management
- Dynamic reconfiguration
With this new IBM Installation Toolkit feature, once you are done with the installation wizard, your Linux system will be able to leverage the the multipath I/O capabilities from your IBM PowerLinux server, without any extra configuration steps.
By: Ricardo M. Matinata.
About IBM Installation Toolkit for PowerLinux version 5.3
The IBM® Installation Toolkit for PowerLinux is an optional tool that speeds up the installation of Linux, and gives you access to the IBM value-added software necessary to fully use the capabilities of the Power® platform. You can use the IBM Installation Toolkit on PowerLinux servers, including IBM PowerLinux™ and IBM Power Systems™ servers running Linux.
New in IBM Installation Toolkit version 5.3:
- Support for installing Linux on multipath devices
- Linux LPAR cloning and restoration tool
- Improved UI
- New PowerLinux packages: IBM Performance Manager Monitor, IBM Large Page Analysis tool
There is a previous article that explains all the Linux Installation Options for IBM Power Systems
and how IBM Installation Toolkit fits into the picture.
Also, take a look at the Linux Information Center for Installing Linux with the IBM Installation Toolkit for PowerLinux
You can use the IBM Installation Toolkit to:
- Install and configure Linux on a non-virtualized Power Systems server.
- Install and configure Linux on servers with previously-configured logical partitions (LPARs, also known as virtualized servers).
- Install IBM service and productivity tools on a new or previously installed Linux system. The IBM service and productivity tools include dynamic logical partition (DLPAR) utilities.
- Integrate with the IBM PowerLinux Tools (YUM) Repository
- Upgrade system firmware level on Power Systems servers.
- Perform diagnostics or maintenance operations on previously installed systems.
- Access documentation resources included with IBM Installation Toolkit.
- Migrate a LAMP server (software stack) and application data from a System x® to a System p® system.
If you have comments on the IBM Installation Toolkit, please drop us a line here or in the message boards. Also, check out the IBM Installation Toolkit for Linux landing page