Installing on a Linux management host

Install IBM® Spectrum Symphony on a management host on Linux®. Management hosts provide specialized services to the cluster. The first management host that you install on becomes your primary host.

Before you begin

  • Refer to the minimum system requirements outlined in Requirements and permissions and download the applicable installation package according to Installation packages.
  • If you plan to install IBM Spectrum Conductor after you install IBM Spectrum Symphony, you must install IBM Spectrum Symphony in advanced workload execution mode (WEM) and install with either root or sudo-to-root permission.
  • If you plan to use any of the IBM Spectrum Symphony Advanced Edition features (as described in IBM Spectrum Symphony editions) install IBM Spectrum Symphony Advanced Edition. Additionally, if you plan to install MapReduce, that also requires IBM Spectrum Symphony Advanced Edition.
  • You must know the cluster administrator operating system account, communication ports, database host, workload execution mode (WEM), installation directory, and cluster name.
  • Your hosts require the following utilities. Some of them are typically installed with your operating system. However, if they are removed or are not installed, install them before you install IBM Spectrum Symphony:
    • cURL version 7.28.0 or higher for transferring data using various protocols. Without the correct cURL version, the REST service might encounter errors. To work around this issue, see REST service is in ERROR state. Why?.
    • OpenSSL 1.0.1 or higher to support Transport Layer Security (TLS) and Secure Sockets Layer (SSL) protocols. When OpenSSL is bundled with your cURL installation, the bundled OpenSSL version must be 1.0.1 or higher and the supported TLS version must include TLSv1.2. Without the correct versions, the REST service might encounter errors. To work around this issue, see REST service is in ERROR state. Why?.
    • The bc for mathematical calculations.
    • The gettext library that provides globalization support for translated product messages. Without this library, you will encounter a gettext.sh: file not found or gettext: command not found error during installation. Typically, this library is installed with your operating system. To check your system for this library, run the which gettext command. If the command returns that the gettext library is not found, then you do not have the gettext library installed. To install it, run this command as root: 
      yum install gettext
    • The bind-utils package that provides the nslookup command-line tool.
    • The ed Linux line-oriented text editor, which is required for running certain IBM Spectrum Symphony commands, such as egoinstallfixes.
    • In addition, install these specific Linux packages, as applicable. Ensure you install these packages before you install IBM Spectrum Symphony:
      • For RHEL:
        • On RHEL 8, install the libnsl package. You can run this command as root to install it: 
          yum install libnsl
        • On RHEL 7 or later, install the net-tools package. You can run this command as root to install it: 
          yum install net-tools
        • Ensure that either the fonts local.conf file is configured or the dejavu-serif-fonts package is installed. For more information, see the RHEL documentation:
          • Option 1: Create the /etc/fonts/local.conf file with the following contents:
            <?xml version='1.0'?>
<!DOCTYPE fontconfig SYSTEM 'fonts.dtd'>
<fontconfig>
  <alias>
    <family>serif</family>
    <prefer><family>Utopia</family></prefer>
  </alias>
  <alias>
    <family>sans-serif</family>
    <prefer><family>Utopia</family></prefer>
  </alias>
  <alias>
    <family>monospace</family>
    <prefer><family>Utopia</family></prefer>
  </alias>
  <alias>
    <family>dialog</family>
    <prefer><family>Utopia</family></prefer>
  </alias>
  <alias>
    <family>dialoginput</family>
    <prefer><family>Utopia</family></prefer>
  </alias>
</fontconfig>
          • Option 2: Run the yum install command to install the dejavu-serif-fonts package.
        • On SLES 15, install the net-tools package. You can run this command as root to install it: 
          yum install net-tools
        • For Ubuntu:
          • Install the RPM utility by running the apt-get install rpm command.
          • Install the 32-bit compatibility libraries (lib32z1) by using the aptitude install lib32z1 command.
  • For the cluster administrator (egoadmin), you must configure the required limits for the maximum number of processors (nprocs) and the maximum number of files open (nofile) to 65536 on your hosts. Without this limit, services hang or enter the Error state on cluster startup. This configuration is only required on management hosts.
    In the /etc/security/limits.conf file, define nproc and nofile settings to 65536 for your cluster administrator (egoadmin in the following example) and for the root user:
    egoadmin soft nproc 65536
egoadmin hard nproc 65536
egoadmin soft nofile 65536
egoadmin hard nofile 65536
  • (Optional) If you plan to integrate your Apache Hadoop YARN cluster with IBM Spectrum Symphony during installation, you must install YARN.
  • (Optional) The ServiceDirector and WebServiceGateway services are, by default, set to start manually. If you plan to use these services on a management host, you must manually start them after installation. To start the ServiceDirector service, the host must use glibc version 2.14 or higher.

Procedure

  1. Log in to the host by using the installation user operating system account (cluster administrator, root or sudo to root permission).
  2. Define the cluster properties by setting the following environment variables. If you do not set the optional environment variables, the default values are used.
    To define cluster properties as environment variables, run the following commands corresponding to your shell:
    • sh, ksh, or bash: 
      export VARIABLE_NAME=value
    • csh or tcsh: 
      setenv VARIABLE_NAME value
    To define cluster properties in a file, create a simple text file /tmp/install.config and enter each variable on a new line, in this format:
    VARIABLE_NAME=value
    Any variables set in the environment overwrite the same variables set in the configuration file.
    Option Description
    CLUSTERADMIN Mandatory if you are installing as root. Set to any valid operating user account, which then owns all installation files. For example:
    export CLUSTERADMIN=egoadmin
    CLUSTERNAME Optional. Set to the name of the cluster. The default is cluster1. For example:
    export CLUSTERNAME=cluster123
    Your cluster name is permanent; you cannot change it after installation. Do not use the name of any host or user as the name of your cluster. Consider the following rules when naming your cluster:
    • The cluster name can contain up to 39 alphanumeric characters.
    • The cluster name can contain the underscore character (_), the hyphen character (-), and the period (.). No other special characters are supported.
    • The cluster name cannot contain spaces.
    BASEPORT Optional. Set for the cluster. The default port number is 7869. The cluster uses seven consecutive ports from the base port.

    If you use the default base port (7869), you need ports 7869 to 7875.

    If you use other ports, before installation, make sure that the seven consecutive ports are not in use and set the base port (for example, to set the base port as 14899):
    export BASEPORT=14899
    SIMPLIFIEDWEM Required if you are using simplified WEM. Specify N. For example: 
    export SIMPLIFIEDWEM=N
    IBM_SPECTRUM_SYMPHONY_LICENSE_ACCEPT Mandatory when you install without user interaction (that is, with the --quiet option), to indicate that you accept the license agreement before installing:
    export IBM_SPECTRUM_SYMPHONY_LICENSE_ACCEPT=Y
  3. If you do not want to enable SSL communication for your web servers (enabled by default), set the DISABLESSL environment variable: 
    export DISABLESSL=Y
    Check for port conflicts to ensure that the web servers ports are free. The web servers are accessible on the following default ports:
    Web server With SSL Without SSL
    Web server for the cluster management console 8443 8080
    REST web server 8543 8180
    Important: You must use the same SSL setting for the cluster management console and the RESTful web servers. If you disable SSL for one, you must also disable SSL for the other. This setting also takes effect for cloud bursting with host factory. Ensure that SSL for all these functions is configured consistently in the cluster; without a uniform configuration, errors occur. When SSL is uniformly enabled, you can use different certificates and keys as required.
  4. If you are installing IBM Spectrum Symphony Advanced Edition, define the following environment variables for MapReduce workload:
    Note: To install MapReduce after you have installed your IBM Spectrum Symphony cluster, follow the steps in Adding MapReduce to your IBM Spectrum Symphony cluster instead of completing this step.
    Option Description
    INSTALL_MAPREDUCE Required if you are installing MapReduce. Specify Y. For example:
    export INSTALL_MAPREDUCE=Y

    If not set, or if specified to N, MapReduce is not installed.
    JAVA_HOME Specify the installation directory of Oracle or IBM Java™. For example: 
    export JAVA_HOME=/usr/java/latest
    Note: Your Java installation location is required for enabling the MapReduce framework. You can enable this framework by setting the JAVA_HOME environment variable before you install IBM Spectrum Symphony.

    If you install IBM Spectrum Symphony without setting the JAVA_HOME environment variable, you can do so later by defining the JAVA_HOME in the $SOAM_HOME/mapreduce/conf/pmr-env.sh file.

    HADOOP_HOME Specify the installation directory of the Hadoop distribution. For example:
    export HADOOP_HOME=/opt/hadoop-version
    HADOOP_VERSION Specify the version of the Hadoop distribution. For example:
    export HADOOP_VERSION=version
    DFS_GUI_HOSTNAME Specify the host name of the HDFS NameNode. For example: 
    export DFS_GUI_HOSTNAME=
namenodehost.mydomain.com
    DFS_GUI_PORT Specify the port number for the HDFS NameNode. For example:
    export DFS_GUI_PORT=50070
  5. If you want to use the non-production Derby database for reporting, set the DERBY_DB_HOST environment variable: 
    export DERBY_DB_HOST=hostM
    where hostM is the primary host or another management host that serves as the database host.

    You cannot use the Derby database for production clusters. To produce regular reports for a production cluster, you must configure an external production database after installation. See Setting up an external database for production.

    Without the Derby database or an external database, you cannot generate reports or view the Rack View on the cluster management console. If you do not require these functions, you can manually disable individual data loaders after installation. See Data loaders.

  6. If you are installing IBM Spectrum Symphony Advanced Edition and want to configure Apache Hadoop YARN integration during installation, set the following environment variables:
    Option Description
    JAVA_HOME Specify the installation directory for Oracle or IBM Java. For example: 
    export JAVA_HOME=/usr/java/latest
    Note: To enable YARN integration, you must specify the location of your Java installation by setting the JAVA_HOME environment variable before installation.

    If you install IBM Spectrum Symphony without setting JAVA_HOME, remember to define JAVA_HOME in the $EGO_TOP/integration/yarn/conf/egoyarn-env.sh file.

    HADOOP_YARN_HOME Specify the installation directory for full distribution YARN. For example: 
    export HADOOP_YARN_HOME=/opt/hadoop-version
  7. For Ubuntu installation, run the apt-get install rpm in Ubuntu to install the RPM utility.
  8. Run the IBM Spectrum Symphony installer package.
    • If you want to install with default settings, run:
      sym-version_architecture.bin
      If you are using the evaluation version, run:
      symeval-version_architecture.bin
      Note: Alternative method: If you must use .rpm files instead of .bin, extract the .rpm files, install the ego*.rpm files, then install the soam*.rpm files, in the order described in Files within the installation packages.
      For example, first extract the .rpm files from the sym-version_architecture.bin package by running: 
      sym-version_architecture.bin --extract extract_directory

      where extract_directory specifies the directory to extract .rpm files.

      Next, install each .rpm file in order by running:
      rpm -ivh rpm_file_name
      For example: 
      rpm -ivh egocore-version.architecture.rpm
      Additional note for Ubuntu: To install the .rpm files on Ubuntu, you must also specify the --ignorearch parameter. For example:
      rpm -ivh --ignorearch rpm_file_name
    • If you want to install to a different location other than the default, run:
      sym-version_architecture.bin --prefix install_location --dbpath dbpath_location
      If you are using the evaluation version, run:
      symeval-version_architecture.bin --prefix install_location --dbpath dbpath_location
      where:
      • --prefix install_location specifies the absolute path to the installation directory. The --prefix parameter is optional. If you install without the --prefix option, IBM Spectrum Symphony is installed in its default directory: /opt/ibm/spectrumcomputing on Linux. Ensure that the path is set to a clean directory.
      • --dbpath dbpath_location sets the RPM database to a directory different from the default /var/lib/rpm. The --dbpath parameter is optional.
      Note: Alternative method: If you must use .rpm files instead of .bin, extract the .rpm files, install the ego*.rpm files, then install the soam*.rpm files, in the order described in Files within the installation packages.
      For example, first extract the .rpm files from the sym-version_architecture.bin package by running: 
      sym-version_architecture.bin --extract extract_directory

      where extract_directory specifies the directory to extract .rpm files.

      Next, install each .rpm file in order by running: 
      rpm -ivh --prefix install_location --dbpath dbpath_location rpm_file_name
      For example: 
      rpm -ivh --prefix /opt/mydir --dbpath /opt/mydir/mydb egocore-version.architecture.rpm
      Additional note for Ubuntu: To install the .rpm files on Ubuntu, you must also specify the --ignorearch parameter. For example:
      rpm -ivh --ignorearch --prefix install_location --dbpath dbpath_location rpm_file_name
    • If you want to install without user interaction, enter one of the following commands:
      Remember: Before installing with the --quiet option, ensure that you have set the IBM_SPECTRUM_SYMPHONY_LICENSE_ACCEPT environment variable to Y or y.
      sym-version_architecture.bin --quiet
      If you are using the evaluation version, run:
      symeval-version_architecture.bin --quiet

      where --quiet suppresses prompts during installation.

  9. After installation is complete, source the environment.
    • If you are running Linux by using bash, run: 
      . $EGO_TOP/profile.platform
    • If you are running Linux by using csh, run: 
      source $EGO_TOP/cshrc.platform

      where $EGO_TOP is the path to your installation directory (default is /opt/ibm/spectrumcomputing).

  10. Verify that you have installed the applicable packages:
    • For Linux and Linux on POWER hosts: Access the System & Services > Software Components page within the cluster management console to see the software components installed on each host. This page indicates the installed software component packages, and active versions on each host.
    • Query the installation packages on the host database using rpm commands:
      rpm -qa [--dbpath dbpath_location] | grep package_name
      Note that if you installed IBM Spectrum Symphony using the --dbpath option, use the same dbpath_location to verify. For example, if you installed using /data/abc/rpmdb as the dbpath_location, then run the verification commands using that database location. Here is an example: 
      rpm -qa --dbpath /data/abc/rpmdb | grep soammrmgmt

    For a list of installation package names, refer to Files within the installation packages.

What to do next

After you install IBM Spectrum Symphony, set entitlement from the primary host. See Setting entitlement on the Linux primary host and starting the cluster.

Tip: If all of the hosts in your cluster can be reached using short host names, you can configure EGO to use the short host names by specifying the portion of the domain name to remove, by configuring the EGO_STRIP_DOMAIN parameter within the ego.conf file. For full usage and syntax, see ego.conf reference.