Contents

Deploying an OpenStack-based private cloud and Hortonworks Data Platform (HDP) on a Linux on IBM Power Systems server

Sudipta BiswasSudipta's Photo and Pradipta BanerjeePradipta's photo
Published on March 09, 2017

Overview

This article describes the deployment of a private cloud with OpenStack and Linux® on IBM® Power® System LC servers that are running kernel-based virtual machines (KVMs). This paper also covers the deployment of Hortonworks Data Platform (HDP) using OpenStack.

Details

The following requirements are met in this cloud deployment:

  • Deployment of RDO distribution of OpenStack
  • Deployment of virtual machines (VMs) to the KVM (RHEV) hypervisor that is running Linux on IBM POWER8® processor-based systems
  • Deployment of IBM Storwize® V5000 storage to provide block storage to VMs
  • Creation of multiple cinder backend systems based on disk types to cater to different performance requirements of the VMs
  • Deployment of HDP

Hardware infrastructure

This section describes the hardware used for deploying the private cloud.

Server

The setup consists of IBM Power System S822LC servers. Each of these Power S822LC server is a 20-core two-socket server with 256 GB of RAM. Two internal disks of 1 TB each are configured on each system. Two 4-port (10 Gb + 1 GbE) network cards and two 2-port 16 Gbps adapters are provisioned on each server.

Figure 1. Server network layout

View image at full size

Storage

IBM Storwize V5300 storage is deployed along with the servers. Storwize V5300 is used as the primary storage and provides data logical unit numbers (LUNs) for the VMs that are running on the Power S822LC servers. Storwize V5300 has a mix of solid-state drive (SSD) and serial-attached SCSI (SAS) disk types. Disks are allocated based on performance requirements of the VMs. The following storage pools are configured on the Storwize V5300 system:

  • Hybrid pool: Contains both SSD and SAS drives. IBM Easy Tier® has been enabled in this pool. Production VMs use this pool.
  • SAS pool: Contains only SAS drives. Hadoop management node uses this pool.
  • SSD pool: Contains only SSD drives. Hadoop data nodes use this pool.

Network connectivity

Network connectivity of the VMs within the physical server and external network is provided by Open vSwitch. In the existing setup, Open vSwitch is configured to use Linux Ethernet bonding using Link Aggregation Control Protocol (LACP) interface as the backend for external network connectivity. Linux Ethernet bond interface is created by using two physical network interface cards (NICs) on each server. On all hosts, an Open vSwitch bridge named data-br is created.

Logically, the OpenStack environment looks as shown in the following figure.

Figure 2. Cloud setup overview

View image at full size

OpenStack controller is implemented in a VM. The controller has one network interface that is connected to the management network through a Linux bridge created on the host. Each compute node has two interfaces: management and provider. The provider interface connects to a generic network that the physical network infrastructure switches/routes to external networks. The Open vSwitch bridge data-br contains port on the provider network interface.

Compute nodes are partitioned into availability zones based on requirement. Instances are started in the required availability zones.

OpenStack deployment

For infrastructure management of a private cloud, OpenStack must be deployed.

Prerequisites

This section describe the prerequisites to deploy OpenStack.

Required packages

The following packages must be installed:

  • OpenStack Mitaka noarch packages repo
  • OpenStack dependencies repo for ppc64le
  • RHEV
    • Base ISO image – RHEL 7.2 LE
    • Additional packages for KVM (through RHEV subscription)
      • rhel-7-server-rhv-4-mgmt-agent-for-power-le-rpms
      • rhel-7-server-rhev-tools-for-power-le-rpms
      • rhel-7-for-power-le-optional-rpms
      • rhel-7-server-rhev-mgmt-agent-for-power-le-rpms
      • rhel-7-server-rhv-4-tools-for-power-le-rpms
      • rhel-7-for-power-le-rpms
  • Utilities
    • cloud-init, cloud-utils, and cloud-utils-growpart from the following repository:

Preparing the hosts

  1. Install the base operating system and KVM (RHEV) packages on all the hosts.
  2. Create either local yum repositories with the packages (mentioned earlier), or ensure that network connectivity is there to access the publicly hosted repositories.
  3. Set the time zone and configure Network Time Protocol (NTP).
  4. Disable the Network Manager service.
  5. Disable Security-Enhanced Linux (SELinux) or set it to permissive.
  6. Set up the IP, host name, and other details.
  7. Install KVM. The following are the minimum rpm packages that are required:
    • libvirt
    • libvirt-daemon-kvm
    • qemu-img-rhev
    • qemu-kvm-rhev
    • qemu-kvm-common-rhev
    • qemu-kvm-tools-rhev
    • virt-install

Setting up the controller VM

  1. Create a Linux bridge on the selected KVM host that is designated to host the controller VM.
  2. Create a controller VM by using the virt-install command. Allocate disks and network connection to the controller VM.
  3. Install RHEL (minimal installation).
  4. Create yum repository configuration for the OpenStack packages.
  5. Disable the Network Manager service.
  6. Disable SELinux or set it to permissive.
  7. Disable the Firewalld service or enable ports required for OpenStack communication. You can find details about OpenStack ports in the Firewalls and default ports website.
  8. Set the time zone and configure NTP.
  9. Set up the IP, host name, and other details.

Installing OpenStack

After the prerequisites are met, you can install OpenStack.

On the controller node:

  1. Install Packstack by running the following command: 
    # yum install -y openstack-packstack
  2. Generate the Packstack sample file by running the following command: 
    # packstack --gen-answer-file /root/answer.txt
  3. Update the Packstack answer file with the following information: 
    CONFIG_DEFAULT_PASSWORD=openstack
CONFIG_NEUTRON_ML2_TYPE_DRIVERS=vlan, flat
CONFIG_NEUTRON_ML2_TENANT_NETWORK_TYPES=vlan, flat
CONFIG_NEUTRON_ML2_VLAN_RANGES=datanet:1:200
CONFIG_NEUTRON_OVS_BRIDGE_MAPPINGS=datanet:data-br
CONFIG_NEUTRON_OVS_BRIDGE_IFACES=data-br:enP1p3s0f1
CONFIG_NEUTRON_OVS_BRIDGES_COMPUTE=data-br CONFIG_PROVISION_DEMO=n
CONFIG_COMPUTE_HOSTS=192.168.4.3, 192.168.4.4 CONFIG_SAHARA_INSTALL=y
  4. Run the Packstack using the following command. 
    # packstack --answer-file /root/answer.txt
  5. After Packstack completes, log in to the Horizon dashboard:
    https://<controller-ip>/dashboard

Additional configuration

The following additional configuration must be performed:

  1. Enable IBM System Storage® SAN Volume Controller (SVC) Fibre Channel (FC) multipath devices on each compute node for cinder volumes:
    1. Update the following parameter in the /etc/nova/nova.conf file on each compute host: 
      iscsi_use_multipath = True
    2. Restart nova-compute on each compute host by running the following commands: 
      # systemctl stop openstack-nova-compute.service 
# systemctl start openstack-nova-compute.service
  2. Log in to the controller node and perform the following steps:
    1. Install the Sahara UI component.
      # yum install openstack-sahara-ui
    2. Restart the http daemon service. 
      # systemctl restart httpd
    3. Populate the environment variables by running the following command using the file generated by Packstack. 
      # source /root/keystonerc_admin
    4. Add the host aggregate and availability zones as per requirement.
    5. Add the network and subnets based on requirement.
    6. Add the cinder back end and restart cinder service on the controller.
  3. Update the /etc/cinder/cinder.conf file on the controller. Add one stanza for each storage pool that is defined in the Storwize V5300 storage (use the values as appropriate for your setup):
    For example: 
    [swv7000]
volume_driver = cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc.StorwizeSVCFCDriver
san_ip=192.168.5.0
san_login=openstack
san_password=passw0rd
storwize_svc_volpool_name=SAS_POOL
volume_backend_name=v5300hostname_SAS_POOL
  4. Update the enabled_backends parameter in the /etc/cinder/cinder.conf file on the controller. 
    enabled_backends = lvm, swv7000
  5. Restart the cinder service by running the following commands: 
    # systemctl restart openstack-cinder-api.service 
# systemctl restart openstack-cinder-scheduler.service 
# systemctl restart openstack-cinder-volume.service

Golden image preparation for HDP

Create a RHEL 7.3 VM and perform the modifications specified in the following sections.

Repository preparation

Ensure that the RHEL 7.3 repositories are configured and accessible. In addition, you must configure the following HDP and Ambari repositories:

Package installation

Complete the following steps to install the packages:

  1. Install the following packages inside the image:
    • ambari-server
    • openjdk
    • ambari-agent
    • ppc64-utils
    • librtas
    • ppc64-diag
  2. Create a file, named 99-java.sh, in /etc/profile.d/ with the contents of JAVA_HOME (must be the JDK path). For example, /usr/lib/jvm/jre-1.7.0-openjdk.
  3. Disable the ambari-agent service.
  4. Ensure that the PostgreSQL initdb does not exist. If it exists, Ambari server initialization fails.
  5. Disable the Firewalld service or ensure that port 8080 is open.
  6. Install cloud-init, cloud-utils, and cloud-utils-growpart. These can be obtained from the https://dl.fedoraproject.org/pub/epel/7/ppc64le/ repository.

After all these packages are installed, ensure that the image is configured to receive a Dynamic Host Configuration Protocol (DHCP) request by checking the /etc/sysconfig/network-scripts/ifcfg-eth0 file for the entry, BOOTPROTO=dhcp.

After the image is created, upload the same to glance.

Configuring Sahara templates

Complete the following procedure to configure Sahara templates:

  1. Register the VM image in Sahara by using the appropriate plug-in tag for which this image is used. Tags might be ambari, 2.4 (for OpenStack Liberty), ambari, 2.5 (for OpenStack Ocata).
  2. Create node group templates by providing the generic parameters for the node group template. If you are planning to cinder volumes, select the appropriate option. You can also select ephemeral volumes, which would mean that the VM disks would be local to the host machine.
    Figure 3. Configure node group templates

    View image at full size

  3. Configure the node processes. Sahara performs some validations for node processes as shown in the following figure.
    Figure 4. Node group processes

    View image at full size

  4. Create the cluster templates. Cluster-wide configuration settings are provided here. You must select the node groups as a part of the cluster.
    Figure 5. Cluster general information

    View image at full size

    Figure 6. Cluster node group details

    View image at full size

    Figure 7. General parameters with HDP and HDP-UTILs repository details

    View image at full size

Post deployment

After you launch the cluster, Sahara states of the deployed cluster change as described here:

InfraUpdating -> Spawning -> Configuring -> Starting -> Active.

Figure 8. Cluster status

View image at full size

You can observe the VM deployments on the Instances tab:

Figure 9. Instances tab after successful deployment

View image at full size

Figure 10. Volumes attached to instances

View image at full size

Figure 11. Health of the clusters in green

View image at full size

After successful completion of this step, the Hadoop cluster becomes ready to use.

Summary

This article highlights the key configuration and components implemented in the private cloud using RDO and Linux on IBM Power Systems™.

Resources

Acknowledgments

We would like to thank the following people for their guidance and help to publish this article:

  • Dipankar Sarma, Distinguished Engineer, Linux Technology Center, IBM Systems
  • Anand Haridass, STSM, Power Integrated Solutions and OpenPOWER India Enablement
  • Kuntal Acharya, Consultant, Power Systems and AIX, IBM Systems
  • Prerna Upmanyu Consultant, Power Systems and AIX, IBM Systems
  • Ashish Kumar, Big Data Analytics Solutions Architect and Business Development, IBM Systems

Downloadable resources

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Linux
ArticleID=1043713
ArticleTitle=Deploying an OpenStack-based private cloud and Hortonworks Data Platform (HDP) on a Linux on IBM Power Systems server
publish-date=03092017