Deploying an OpenStack-based private cloud and Hortonworks Data Platform (HDP) on a Linux on IBM Power Systems server
Overview
This article describes the deployment of a private cloud with OpenStack and Linux® on IBM® Power® System LC servers that are running kernel-based virtual machines (KVMs). This paper also covers the deployment of Hortonworks Data Platform (HDP) using OpenStack.
Details
The following requirements are met in this cloud deployment:
- Deployment of RDO distribution of OpenStack
- Deployment of virtual machines (VMs) to the KVM (RHEV) hypervisor that is running Linux on IBM POWER8® processor-based systems
- Deployment of IBM Storwize® V5000 storage to provide block storage to VMs
- Creation of multiple cinder backend systems based on disk types to cater to different performance requirements of the VMs
- Deployment of HDP
Hardware infrastructure
This section describes the hardware used for deploying the private cloud.
Server
The setup consists of IBM Power System S822LC servers. Each of these Power S822LC server is a 20-core two-socket server with 256 GB of RAM. Two internal disks of 1 TB each are configured on each system. Two 4-port (10 Gb + 1 GbE) network cards and two 2-port 16 Gbps adapters are provisioned on each server.
Figure 1. Server network layout

Storage
IBM Storwize V5300 storage is deployed along with the servers. Storwize V5300 is used as the primary storage and provides data logical unit numbers (LUNs) for the VMs that are running on the Power S822LC servers. Storwize V5300 has a mix of solid-state drive (SSD) and serial-attached SCSI (SAS) disk types. Disks are allocated based on performance requirements of the VMs. The following storage pools are configured on the Storwize V5300 system:
- Hybrid pool: Contains both SSD and SAS drives. IBM Easy Tier® has been enabled in this pool. Production VMs use this pool.
- SAS pool: Contains only SAS drives. Hadoop management node uses this pool.
- SSD pool: Contains only SSD drives. Hadoop data nodes use this pool.
Network connectivity
Network connectivity of the VMs within the physical server and external network is provided by Open vSwitch. In the existing setup, Open vSwitch is configured to use Linux Ethernet bonding using Link Aggregation Control Protocol (LACP) interface as the backend for external network connectivity. Linux Ethernet bond interface is created by using two physical network interface cards (NICs) on each server. On all hosts, an Open vSwitch bridge named data-br is created.
Logically, the OpenStack environment looks as shown in the following figure.
Figure 2. Cloud setup overview

OpenStack controller is implemented in a VM. The controller has one network interface that is connected to the management network through a Linux bridge created on the host. Each compute node has two interfaces: management and provider. The provider interface connects to a generic network that the physical network infrastructure switches/routes to external networks. The Open vSwitch bridge data-br contains port on the provider network interface.
Compute nodes are partitioned into availability zones based on requirement. Instances are started in the required availability zones.
OpenStack deployment
For infrastructure management of a private cloud, OpenStack must be deployed.
Prerequisites
This section describe the prerequisites to deploy OpenStack.
Required packages
The following packages must be installed:
- OpenStack Mitaka noarch packages repo
- OpenStack dependencies repo for ppc64le
- RHEV
- Base ISO image – RHEL 7.2 LE
- Additional packages for KVM (through RHEV subscription)
- rhel-7-server-rhv-4-mgmt-agent-for-power-le-rpms
- rhel-7-server-rhev-tools-for-power-le-rpms
- rhel-7-for-power-le-optional-rpms
- rhel-7-server-rhev-mgmt-agent-for-power-le-rpms
- rhel-7-server-rhv-4-tools-for-power-le-rpms
- rhel-7-for-power-le-rpms
- Utilities
- cloud-init, cloud-utils, and cloud-utils-growpart from the following repository:
Preparing the hosts
- Install the base operating system and KVM (RHEV) packages on all the hosts.
- Create either local yum repositories with the packages (mentioned earlier), or ensure that network connectivity is there to access the publicly hosted repositories.
- Set the time zone and configure Network Time Protocol (NTP).
- Disable the Network Manager service.
- Disable Security-Enhanced Linux (SELinux) or set it to permissive.
- Set up the IP, host name, and other details.
- Install KVM. The following are the minimum rpm packages that are required:
- libvirt
- libvirt-daemon-kvm
- qemu-img-rhev
- qemu-kvm-rhev
- qemu-kvm-common-rhev
- qemu-kvm-tools-rhev
- virt-install
Setting up the controller VM
- Create a Linux bridge on the selected KVM host that is designated to host the controller VM.
- Create a controller VM by using the
virt-install
command. Allocate disks and network connection to the controller VM. - Install RHEL (minimal installation).
- Create yum repository configuration for the OpenStack packages.
- Disable the Network Manager service.
- Disable SELinux or set it to permissive.
- Disable the Firewalld service or enable ports required for OpenStack communication. You can find details about OpenStack ports in the Firewalls and default ports website.
- Set the time zone and configure NTP.
- Set up the IP, host name, and other details.
Installing OpenStack
After the prerequisites are met, you can install OpenStack.
On the controller node:
- Install Packstack by running the following command:
# yum install -y openstack-packstack
- Generate the Packstack sample file by running the following command:
# packstack --gen-answer-file /root/answer.txt
- Update the Packstack answer file with the following information:
CONFIG_DEFAULT_PASSWORD=openstack CONFIG_NEUTRON_ML2_TYPE_DRIVERS=vlan, flat CONFIG_NEUTRON_ML2_TENANT_NETWORK_TYPES=vlan, flat CONFIG_NEUTRON_ML2_VLAN_RANGES=datanet:1:200 CONFIG_NEUTRON_OVS_BRIDGE_MAPPINGS=datanet:data-br CONFIG_NEUTRON_OVS_BRIDGE_IFACES=data-br:enP1p3s0f1 CONFIG_NEUTRON_OVS_BRIDGES_COMPUTE=data-br CONFIG_PROVISION_DEMO=n CONFIG_COMPUTE_HOSTS=192.168.4.3, 192.168.4.4 CONFIG_SAHARA_INSTALL=y
- Run the Packstack using the following command.
# packstack --answer-file /root/answer.txt
- After Packstack completes, log in to the Horizon dashboard:
https://<controller-ip>/dashboard
Additional configuration
The following additional configuration must be performed:
- Enable IBM System Storage® SAN Volume Controller (SVC) Fibre Channel (FC)
multipath devices on each compute node for cinder volumes:
- Update the following parameter in the /etc/nova/nova.conf file on each
compute host:
iscsi_use_multipath = True
- Restart
nova-compute
on each compute host by running the following commands:# systemctl stop openstack-nova-compute.service # systemctl start openstack-nova-compute.service
- Update the following parameter in the /etc/nova/nova.conf file on each
compute host:
- Log in to the controller node and perform the following steps:
- Install the Sahara UI component.
# yum install openstack-sahara-ui
- Restart the
http daemon
service.# systemctl restart httpd
- Populate the environment variables by running the following command using the
file generated by Packstack.
# source /root/keystonerc_admin
- Add the host aggregate and availability zones as per requirement.
- Add the network and subnets based on requirement.
- Add the cinder back end and restart cinder service on the controller.
- Install the Sahara UI component.
- Update the /etc/cinder/cinder.conf file on the controller. Add
one stanza for each storage pool that is defined in the Storwize V5300 storage (use the values as appropriate for your setup):
For example:[swv7000] volume_driver = cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc.StorwizeSVCFCDriver san_ip=192.168.5.0 san_login=openstack san_password=passw0rd storwize_svc_volpool_name=SAS_POOL volume_backend_name=v5300hostname_SAS_POOL
- Update the
enabled_backends
parameter in the /etc/cinder/cinder.conf file on the controller.enabled_backends = lvm, swv7000
- Restart the cinder service by running the following commands:
# systemctl restart openstack-cinder-api.service # systemctl restart openstack-cinder-scheduler.service # systemctl restart openstack-cinder-volume.service
Golden image preparation for HDP
Create a RHEL 7.3 VM and perform the modifications specified in the following sections.
Repository preparation
Ensure that the RHEL 7.3 repositories are configured and accessible. In addition, you must configure the following HDP and Ambari repositories:
- Ambari repositories: http://public-repo-1.hortonworks.com/ambari/centos7-ppc/2.x/updates/2.4.2.4-3/ambari.repo
- HDP repositories: http://public-repo-1.hortonworks.com/HDP/centos7-ppc/2.x/updates/2.5.3.0/hdp.repo
- HDP Utils repositories: http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/ppc64le/hdp-util.repo
Package installation
Complete the following steps to install the packages:
- Install the following packages inside the image:
- ambari-server
- openjdk
- ambari-agent
- ppc64-utils
- librtas
- ppc64-diag
- Create a file, named 99-java.sh, in /etc/profile.d/ with the contents of JAVA_HOME (must be the JDK path). For example, /usr/lib/jvm/jre-1.7.0-openjdk.
- Disable the
ambari-agent
service. - Ensure that the PostgreSQL
initdb
does not exist. If it exists, Ambari server initialization fails. - Disable the Firewalld service or ensure that port 8080 is open.
- Install cloud-init, cloud-utils, and cloud-utils-growpart. These can be obtained from the https://dl.fedoraproject.org/pub/epel/7/ppc64le/ repository.
After all these packages are installed, ensure that the image is configured to
receive a Dynamic Host Configuration Protocol (DHCP) request by checking the
/etc/sysconfig/network-scripts/ifcfg-eth0 file for the entry,
BOOTPROTO=dhcp
.
After the image is created, upload the same to glance.
Configuring Sahara templates
Complete the following procedure to configure Sahara templates:
- Register the VM image in Sahara by using the appropriate plug-in tag for which this image is used. Tags might be ambari, 2.4 (for OpenStack Liberty), ambari, 2.5 (for OpenStack Ocata).
- Create node group templates by providing the generic parameters for the node
group template. If you are planning to cinder volumes, select the appropriate
option. You can also select ephemeral volumes, which would mean that the VM
disks would be local to the host machine.
Figure 3. Configure node group templates
- Configure the node processes. Sahara performs some validations for node
processes as shown in the following figure.
Figure 4. Node group processes
- Create the cluster templates. Cluster-wide configuration settings are provided
here. You must select the node groups as a part of the cluster.
Figure 5. Cluster general information
Figure 6. Cluster node group details
Figure 7. General parameters with HDP and HDP-UTILs repository details
Post deployment
After you launch the cluster, Sahara states of the deployed cluster change as described here:
InfraUpdating -> Spawning -> Configuring -> Starting -> Active.
Figure 8. Cluster status

You can observe the VM deployments on the Instances tab:
Figure 9. Instances tab after successful deployment

Figure 10. Volumes attached to instances

Figure 11. Health of the clusters in green

After successful completion of this step, the Hadoop cluster becomes ready to use.
Summary
This article highlights the key configuration and components implemented in the private cloud using RDO and Linux on IBM Power Systems™.
Resources
- Linux on IBM Power Systems
- IBM Power System S822LC for Commercial Computing
- What is OpenStack?
- RDO
- IBM Power System S822LC Technical Overview and Introduction
Acknowledgments
We would like to thank the following people for their guidance and help to publish this article:
- Dipankar Sarma, Distinguished Engineer, Linux Technology Center, IBM Systems
- Anand Haridass, STSM, Power Integrated Solutions and OpenPOWER India Enablement
- Kuntal Acharya, Consultant, Power Systems and AIX, IBM Systems
- Prerna Upmanyu Consultant, Power Systems and AIX, IBM Systems
- Ashish Kumar, Big Data Analytics Solutions Architect and Business Development, IBM Systems