Configuring Google Cloud Platform for LSF resource connector

Follow these steps to configure Google Cloud Platform (GCP) to create instances for LSF resource connector to make allocation requests on behalf of LSF. The instances launched from Google Cloud Platform join the LSF cluster. If instances become idle, LSF resource connector terminates them.

Before you begin

The Google Cloud Platform provider requires IBM Spectrum LSF Fix Pack 4 or newer.

Before using the LSF resource connector with the Google Cloud Platform provider, you must complete the following steps:
  1. Apply LSF Fix Pack 4 or newer.
  2. Manually move the required configuration files to the appropriate directory under LSF_TOP/conf/resource_connector/<provider_name>/conf.
  3. Change the ownership of the new files and directories to the cluster administrator.

For more information about applying Fix Packs to LSF resource connector, see Use the LSF patch installer to update resource connector.

  • You must have root access to the LSF management host.
  • Both the management host and the Google Cloud Platform instances that act as LSF execution hosts must be reachable from each other.
  • The LSF cluster must have a DNS server.
  • You must have correct permissions to update the DNS server.
  • You must be able to restart the LSF cluster.
  • You must be familiar with and have the ability to perform Google Cloud Platform administrative operations.
  • Google Cloud VPC (Google Networks) must be configured to be used by the LSF management and Google Cloud instances.

About this task

LSF resource connector has been tested on the following systems:
  • Linux x86 Kernel 2.6, glibc 2.11 RHEL 6.x
  • Linux x86 Kernel 3.10, glibc 2.17 RHEL 7.x
  • Linux x86 Kernel 3.10, glibc 2.17 CentOS 7.x
  • LSF 10.1 Standard Edition (Fix Pack 4 or newer)

In the following steps, you must perform all operations as the Google Cloud Platform administrator unless otherwise stated.

For full details on installing LSF, see Install IBM Spectrum LSF on UNIX and Linux.

To get the job submitted by a user to run on the instance, the instance must have this user prepared or LSF user mapping configured. For more information about user groups and user account mapping, see Managing LSF user groups and Between-Host User Account Mapping.

Procedure

  1. Create a Google project and create a service account authentication file.
    1. Create a project.

      Before you begin to create any type of resources on Google Cloud Platform, you must create a project. Go to the Google Cloud Platform console to create a new project.

      https://console.cloud.google.com/

    2. Create a Google service account key.
      The resource connector Google Cloud plugin uses the service account key as the default authentication method. Use the following link to create the service account key: https://console.developers.google.com/apis/credentials?pli=1
      1. Choose your project.
      2. Click Create credentials and select Service account key.
      3. On the next page, choose your service account and key type.
      4. Select JSON format as the key type. The service account key downloads automatically. Save the key to a safe location.
  2. Optional: Create a VPC network and subnetworks.

    A default VPC network is created automatically when you create a project on Google Cloud. The subnet creation mode of the default VPC is Auto subnets, which automatically creates one subnetwork named default in each Google Cloud region. All the subnets in one VPC can communicate with each other across regions. Use the default VPC as your LSF network.

    Either set the subnet creation node to Auto, which is the same as the default VPC, or alternatively. set it to Custom and create a custom VPC network.

  3. Configure firewall rules for the LSF VPC network.

    Firewall rules control incoming or outgoing traffic to an instance. By default, incoming traffic from outside your network is blocked. However, incoming traffic inside your VPC network is all opened in the default VPC network. You must add customized rules to open all LSF listening ports to the firewall rules that are used by LSF instances. The ports must match the ports from the existing LSF cluster.

    Use the following default port numbers:
    • LSF_LIM_PORT=7869 (TCP and UDP)
    • LSF_RES_PORT=6878 (TCP)
    • LSB_SBD_PORT=6882 (TCP)
    Note: If you only allow the traffic from these ports, some NIOS commands might not work because the ports are configured for them dynamically and are different from the default LSF ports.

    You can also accept all traffic from the management host (add the management host IP address).

  4. Connect Google Cloud Instances and SSH keys.

    You can use the Google Cloud gcloud command or your browser to connect to your instances. The Google Cloud Platform manages your SSH keys for you to create and apply SSH key pairs when you connect to a Linux instance from the browser (in the Google Cloud Platform Console) or by using the gcloud command.

    For more information, see the Google Cloud documentation:

    https://cloud.google.com/compute/docs/instances/connecting-to-instance

    For example, the following command connects your instance if it has a public IP.
    gcloud compute ssh [INSTANCE_NAME]
  5. Build the LSF cloud image.

    To create a Google Cloud custom image for an LSF execution host, you must first manually launch an instance and install LSF on that instance. Create a custom image from the instance using the Google Cloud Platform console or by using the gcloud command. You can launch subsequent cloud instances dynamically, using this custom image.

    1. Log in to the instance that you created.
    2. Copy the LSF packages and entitlement file to the instance.
      • lsf10.1_linux2.6-glibc2.3-x86_64.tar.Z
      • lsf10.1_lsfinstall.tar.Z
      • lsf_std_entitlement.dat
    3. Install the software.
      ed.x86_64

      If no software is installed, you can use the command yum install to install.

      # yum install ed
    4. Edit the server.config file to set the installation options.
      For example,
      LSF_TOP="/opt/lsf"
         LSF_ADMINS="lsfadmin"
         LSF_TARDIR="/opt/install/"
         LSF_LICENSE="/opt/install/lsf_std_entitlement.dat"
         LSF_SERVER_HOSTS="management.myserver.com"
         LSF_LOCAL_RESOURCES="[resource googlehost]"
         LSF_LIM_PORT="7869"
         LSF_GET_CONF=lim
      LSF_LOCAL_RESOURCES
      The new resource name googlehost is used by LSF to identify Google Cloud instances. Use the bhosts -a command to see instances that are used by LSF.
      LSF_LIM_PORT
      The port number must be the same as the one defined on the LSF management host.
      LSF_GET_CONF
      Updates the LSF configuration to synchronize the cluster configuration with the management host.
    5. Run the LSF installation script.
      ./lsfinstall -s -f server.config
      After installation, make sure the <LSF_TOP>/conf/lsf.conf configuration file contains the "googlehost" resource:
      cat /opt/lsf/conf/lsf.conf
         LSF_GET_CONF=lim
         LSF_CONFDIR=/opt/lsf/conf
         LSF_LIM_PORT=7869
         LSF_SERVER_HOSTS="management.myserver.com"
         LSF_VERSION=10.1
         LSF_LOCAL_RESOURCES="[resource googlehost]"
         LSF_TOP=/opt/lsf/
      ...
    6. If needed, configure your DNS/NIS server.

      The default DNS server for Google Cloud works with LSF.

    7. Start the LSF daemons on the instance manually and make sure that the instance can join the LSF cluster as a dynamic host.
      If the instance does not join the cluster, check the following items:
      • The instance firewall, or firewall rules of the VPC network.
      • The management host and the instance can ping each other using the private IP address.
    8. Shut down the LSF daemons and log out from the instance.
    9. Create the LSF compute image.

      You can create the LSF compute image in the Google Cloud Console or by using the gcloud command. For detailed steps about how to create a custom image, see the Google Cloud documentation:

      https://cloud.google.com/compute/docs/images/create-delete-deprecate-private-images

      For example, use these steps to create a custom image using the Google Cloud Console.
      1. Before capturing the image, stop the instance:

        In the Compute Engine Service, on the VM instances tab, select your LSF instance, and click Stop.

      2. Go to the Image tab, and click Create Image.
      3. Input your image name, select Source disk as your LSF instance, and click Create.

      The custom image is in the images list.

      Use the name of the image in the imageId parameter in the googleprov_templates.json file.