Adding a dynamic host in a non-shared file system environment

In a non-shared file system environment, you must install LSF binaries, a localized lsf.conf file, and shell environment scripts (cshrc.lsf and profile.lsf) on each dynamic host.

Specifying installation options in the server.config file

All dynamic hosts are server hosts because they cannot serve as management host candidates. The server.config file contains parameters for configuring all server hosts.

Procedure

  1. Define the required parameters:
    • LSF_SERVER_HOSTS="host_name [host_name ...]"
    • LSF_ADMINS="user_name [ user_name ... ]"
    • LSF_TOP="/path"
  2. Optionally define the LSF_LIM_PORT parameter: 
    LSF_LIM_PORT=port_number
    Important: If the management host does not use the default LSF_LIM_PORT, you must specify the same LSF_LIM_PORT defined in lsf.conf on the management host.

Adding local resources on a dynamic host to the cluster

Ensure that the resource name and type are defined in lsf.shared file, and that the ResourceMap section of the lsf.cluster.cluster_name file contains at least one resource mapped to at least one static host. LSF can add local resources as long as the ResourceMap section is defined; you do not need to map the local resources.

Procedure

In the server.config file, define the LSF_LOCAL_RESOURCES parameter.
For numeric resources, define name-value pairs:
"[resourcemap value*resource_name]"
For Boolean resources, the value is the resource name in the following format:
"[resource resource_name]"
For example:
LSF_LOCAL_RESOURCES="[resourcemap 1*verilog] [resource linux]"
Tip: If LSF_LOCAL_RESOURCES are already defined in a local lsf.conffile on the dynamic host, lsfinstall does not add resources you define in LSF_LOCAL_RESOURCES in the server.config file.

When the dynamic host sends a request to the management host to add it to the cluster, the dynamic host also reports its local resources. If the local resource is already defined in the lsf.cluster.cluster_name file as default or all, it cannot be added as a local resource.

Installing LSF on a dynamic host

Procedure

Run lsfinstall -s -f server.config.
lsfinstall creates a local lsf.conf file for the dynamic host, which sets the following parameters:
  • LSF_CONFDIR="/path"
  • LSF_GET_CONF=lim
  • LSF_LIM_PORT=port_number (same as the management host LIM port number)
  • LSF_LOCAL_RESOURCES="resource ..."
    Tip: Do not duplicate LSF_LOCAL_RESOURCES entries in the lsf.conf file. If local resources are defined more than once, only the last definition is valid.
  • LSF_SERVER_HOSTS="host_name [host_name ...]"
  • LSF_VERSION=10.1.0

Configuring dynamic host parameters

Procedure

  1. In the lsf.conf configuration file on the management host, define the LSF_DYNAMIC_HOST_WAIT_TIME parameter, in seconds, and assign a value greater than zero.

    LSF_DYNAMIC_HOST_WAIT_TIME specifies how long a dynamic host waits before sending a request to the management host LIM to add the host to the cluster.

    For example:
    LSF_DYNAMIC_HOST_WAIT_TIME=60
  2. Define the LSF_DYNAMIC_HOST_TIMEOUT parameter.
    LSF_DYNAMIC_HOST_TIMEOUT specifies how long LSF waits before the management host automatically removes unavailable dynamic host. Each time LSF removes a dynamic host, mbatchd daemon automatically reconfigures itself.
    Note: For very large clusters, defining this parameter could decrease system performance
    For example: 
    LSF_DYNAMIC_HOST_TIMEOUT=60m
  3. In the lsf.cluster.cluster_name configuration file on the management host, define the LSF_HOST_ADDR_RANGE parameter.
    LSF_HOST_ADDR_RANGE enables security by defining a list of hosts that can join the cluster. Specify IP addresses or address ranges for hosts that you want to allow in the cluster.
    Note: If you define the LSF_ENABLE_SUPPORT_IPV6 parameter in the lsf.conf file, IP addresses can have either a dotted quad notation (IPv4) or IP Next Generation (IPv6) format; you do not have to map IPv4 addresses to an IPv6 format.
    For example: 
    LSF_HOST_ADDR_RANGE=100-110.34.1-10.4-56

    All hosts belonging to a domain with an address having the first number between 100 and 110, then 34, then a number between one and ten, and then a number between four and 56, will be allowed access. In this example, no IPv6 hosts are allowed.

Starting LSF daemons

Procedure

  1. Log on as root to each host you want to join the cluster.
  2. Source the LSF environment:
    • For csh or tcsh:
      source LSF_TOP/conf/cshrc.lsf
    • For sh, ksh, or bash:
      . LSF_TOP/conf/profile.lsf
  3. Consider if you want LSF to start automatically when the host reboots.

    • If do not want automatic restarting, go to the next step.

    • If you want automatic restarting, run the hostsetup command. For example:
      cd /usr/share/lsf/10.1.0/install
./hostsetup --top="/usr/share/lsf" --boot="y"

    For complete hostsetup usage, enter hostsetup -h.

  4. Start the daemons.
    Assuming RSH (or password-less SSH) is set up, run the startup commands so that they take effect on the host being added to the cluster.
    • To run the commands on the host being added:
      bctrld start lim
bctrld start res
bctrld start sbd
    • To run the commands from another host (for example, if you want to start the daemons on hostB from hostA):
      rsh hostB bctrld start lim
rsh hostB bctrld start res
rsh hostB bctrld start sbd