There is no doubt the acceptance and growing popularity of Linux on Power is here. With POWER based systems providing advantages over x86 systems in performance, capacity, memory and IO bandwidth, and reliability (I have intentionally left out the CAPI technology and am saying that discussion for another BLOG that can delve into that and the incredible capabilities it provides) and with the latest Linux distributions from RedHat, SUSE, and Ubuntu exploiting POWER8 technology, customers are now looking at hosting mission critical applications on Power Systems with Linux for a better customer experience at a lower TCO. (and Linux is Linux, right??....by the way, not really! But that will be the subject of a different BLOG)
Yes, Linux on Power has great capabilities and will expand rapidly, but I also believe you all have several critical decisions to understand as you start your implementations. As not everyone is actively running production on Linux on Power, I suspect that most all of you will start with a Proof of Concept or a Proof of Technology. And this is where decisions have to start be made and options chosen. In my opinion your decisions have to start at the application layer and then work down to the hardware server selection. Several years ago the choices were much easier since there were only specific elements available such as the virtualization layer (it was only PowerVM), the Linux distribution (previously only SUSE and RedHat were certified), the Cloud (Smart Cloud Entry only), Endianness (Big only), and the hardware support of POWER6 or POWER7 really only differentiated by the SMT levels. Now this has changed with many new options and the IBM support of key Open Sources technologies like OpenStack and KVM. And like Indiana Jones was told by the Grail Knight in the Last Crusade, “Choose wisely”. Your choices should also be chosen wisely and with forethought.
A suggestion, of mine, is to start at the application. First the application must be available (eg complied) for POWER or must be a script or scripting tool or Java application that is ISA neutral or can be compiled to the POWER ISA (PPC) such as Open Source code or your own “home grown” application. Choose whatever application you will (eg Websphere, SAP HANA, TSM, MariaDB, Redis, home grown, etc.) for the POC or POT but you then must determine the Endianness of the application or the Endianness that you will compile for. The reason is that the Endianness of the application MUST match that of the Linux Distribution. So, if you choose SAP, which is Big Endian, you need to choose a BE distribution like RedHat 6.5, RedHat 7.1 BE (not the LE RedHat 7.1 which is a separate SKU), or SLES 11. If you had chosen SLES 12 or Ubuntu you would have a problem since these are Little Endian distributions and your SAP HANA test would fail immediately. Now, skipping forward several choices, we must recognize that Little Endian applications can only run on POWER8 hardware. It can run in a partition, IFL on a POWER8 server (S8xx, E8xx) or on an S8xxL system, but not on POWER7 nor POWER7+ nor POWER6.
Many people ask, “what distribution should I use?”. First you must choose the correct distribution based on LE or BE capabilities. Then the differentiator normally is based on consistency. RedHat, SUSE, Ubuntu, Debian, Fedora, OpenSuse are all based on the Linux kernel for sure. But, they are all at different kernel levels, have different “support packages” included in the distro, have different administrative procedures for such items as tuning, install, management, EUI, support, etc. You should review specific requirements that may lead to a multi distribution environment versus consistency within the environment for a single skill set and single set of procedures. It should be noted that the interface to the Linux OS is consistent for applications, so that is not a major consideration although certain ISV applications may only be certified on specific distributions and releases. An example is that SAP HANA is only supported on SLES 11 at this time.
Another point, though not that significant, is that the different distributions have different prices and T&Cs such as pricing by cores or sockets and/or virtual machines. These should be reviewed and ordered correctly for entitlement compliance.
Now choose the support structure for Linux on Power. You have two choices basically…support from the distributor or support from IBM GTS. Each are good choices as both IBM and the distributor interact for fix resolution but there are differences in costs..eg, IBM charges for support for the entire server at a single price and distributors charge based on the entitlements. Another point is that if Ubuntu support is wanted, it can only be provided by IBM since Ubuntu is normally community support. Again, consistency within the Enterprise should lead to a clear decision here.
Now you need to decide on your virtualization technology. Your choices are PowerVM, KVM, or no virtualization, sometimes called bare metal. Bare metal implies no virtualization and associated overhead and as such can provide the highest performance of Linux applications on POWER8 Systems. If this is what you wish for example with analytics, HPC, etc, you can only run Linux on Power on POWER8 Scale Out systems (S8xx L’s) since only they have a hardware interface layer called OPAL (Open Power Abstraction Layer) that the Linux distributions must be enabled for in order to operate. OPAL as of today only is available on the S8xx L systems and not the Enterprise systems (E8xx) nor POWER7 systems. Also, although it will change soon, only Ubuntu (LE only) and SLES 12 (LE only) are enabled for bare metal (direct interface to OPAL). PowerVM is the tried and true high function virtualization technology for Power Systems. As of the 126.96.36.199 release, PowerVM will support concurrent usage of BE and LE Linux distributions with equally supported functionality. However PowerVM is specific to Power Systems and some customers may wish a consistent virtualization technology between Power and x86 systems. If this is the case, then KVM, specifically the PowerKVM product from IBM, should be considered. PowerKVM needs the OPAL interface to access the hardware and as such will only run on the POWER8 Scale out Linux only servers at this time. So, no running PowerKVM on POWER7 systems nor on Enterprise Power Systems. But if chosen to be the virtualization layer on the POEWR8 Scale Out Linux only servers, PowerKVM will support BE and LE distributions and applications. Since most of the internals are rooted in KVM, consistency with x86 procedures can be implemented both administratively and skill wise. However, if you are going to use RedHat distributions of Linux and RedHat as your Linux support provider, realize that RedHat support does not recognize PowerKVM as a supported environment and you should obtain RedHat Smart Virtualization as the supported by RedHat virtualization Layer.
So far I have a targeted application, Linux distribution, support, and a virtualization layer. Want Cloud enablement for automated provisioning? The Open Source community is working on OpenStack and IBM has embraced this in PowerVC…which is OpenStack productized for Power architecture. PowerVC has replaced the predecessor product VMControl which ran with System Director. OpenStack provides the ability to capture, deploy, and destroy VMs in the Power environment. You can run with PowerVC as your highest level or you can all the self-provisioning capabilities provided by IBM Cloud Manager with OpenStack or IBM Cloud Orchestrator. ICO provides broader platform support and additional business oriented functions. By the way, some people ask about VMware. The answer is no, VMware is only supported running on x86 systems.
Choosing the cloud layer and the virtualization layer also involves some planning. Since ICMOS contains Open Stack, you cannot run both PowerVC and ICMOS on PowerKVM…reason is you cannot have two OpenStacks managing one virtualization layer. But, if you choose to run PowerVM, both ICMOS and PowerVC talk to the APIs of the HMC (which must be at the 8.x.x level) to provide the automated virtualization. Again, plan ahead.
Finally you need to select a hardware platform. POWER7 is perfectly valid but you must recognize that if that is your choice you cannot run LE nor KVM. With POWER7 you will get BE support from SUSE, Redhat, Fedora and Open SUSE, but no Ubuntu (again LE only). As you move to the POWER8 technology, realize that at this time, OPAL is only implemented on the POWER8 ScaleOut systems for Linux only (S8xx-L models). Since OPAL and pHyp are both shipped on the “L” models, you can reboot and switch back and forth to support PowerKVM or PowerVM. If you plan to do this for test environments, it may be easier to order the PowerVM version and order PowerKVM as a MES later since PowerVM requires vet codes and PowerKVM does not.