Configuring LSF resource connector for IBM Cloud Virtual Servers for VPC Gen 2

Configure LSF and the LSF resource connector for IBM Cloud Gen 2.

Before you begin

Ensure that Python 3.6, or newer, is installed on the LSF master host. Python 3.6 is required for the VPC Python SDK.

Procedure

Log in to the LSF master host as the LSF administrator.
Install the required IBM VPC open source packages.
1. Install the ibm-vpc package.
```
pip install --upgrade "ibm-vpc>=0.0.3"
```
  For more details on the ibm-vpc package, refer to https://github.com/IBM/vpc-python-sdk.
2. Install the ibm-cloud-networking-services package.
```
pip install ibm-cloud-networking-services==0.0.2
```
  For more details on the ibm-cloud-networking-services package, refer to https://pypi.org/project/ibm-cloud-networking-services/#description.
Enable dynamic hosts if this is not already enabled in LSF.
1. Edit the lsf.conf file and specify a small value for the LSF_DYNAMIC_HOST_WAIT_TIME parameter.
  For example,
```
LSF_DYNAMIC_HOST_WAIT_TIME=2
```
2. Edit the lsf.cluster.cluster_name file and specify a small value for the LSF_HOST_ADDR_RANGE parameter.
  For example,
```
LSF_HOST_ADDR_RANGE=*.*.*.*
```

Edit the lsb.modules file and add the demand calculation scheduler module.

Begin PluginModule
SCH_PLUGIN          RB_PLUGIN     SCH_DISABLE_PHASES
...
schmod_demand    ()            ()
...
End PluginModule

Edit the lsf.shared file and add the IBM Cloud Gen 2 resource to the LSF resource list as a new boolean resource.

Begin Resource
RESOURCENAME   TYPE    INTERVAL INCREASING  DESCRIPTION
...
icgen2host     Boolean ()       ()          (GEN2 Hosts borrowed from IBM Cloud)
...
End Resource

Edit the lsf.conf file and define the LSB_RC_EXTERNAL_HOST_FLAG parameter with the IBM Cloud Gen 2 resource.
Defining the LSB_RC_EXTERNAL_HOST_FLAG parameter enables the LSF resource connector. If this parameter is already defined, append the IBM Cloud Gen 2 resource to the existing values in the LSB_RC_EXTERNAL_HOST_FLAG parameter.
```
LSB_RC_EXTERNAL_HOST_FLAG="... icgen2host"
```
Edit the lsb.queues file and define the RC_HOSTS parameter for any queue that needs to borrow an IBM Cloud Gen 2 resource.
If this parameter is already defined for the queue, append the IBM Cloud Gen 2 resource to the existing values in the RC_HOSTS parameter.
```
RC_HOSTS=icgen2host ...
```

Edit the hostProviders.json file and add the IBM Cloud Gen 2 resource provider to the list of defined resource providers.

The default location for this file is

<LSF_TOP>/conf/resource_connector/hostProviders.json

For example,

{
    "providers":[
        {
            "name": "ibmcloudgen2",
            "type": "ibmcloudgen2Prov",
            "confPath": "resource_connector/ibmcloudgen2",
            "scriptPath": "resource_connector/ibmcloudgen2"
        }
    ]
}

Edit the ibmcloudgen2_config.json file to configure the functions that the LSF resource connector must perform on IBM Cloud Gen 2.

The default location for the file is

<LSF_TOP>/conf/resource_connector/ibmcloudgen2/conf/ibmcloudgen2_config.json

For example:

{
  "IBMCLOUDGEN2_KEY_FILE": "/opt/lsf_rc/conf/resource_connector/ibmcloudgen2/conf/credentials",
  "IBMCLOUDGEN2_SSH_FILE": "/opt/lsf_rc/conf/resource_connector/ibmcloudgen2/conf/id_rsa",
  "IBMCLOUDGEN2_PROVISION_FILE": "/opt/lsf_rc/10.1/resource_connector/ibmcloudgen2/scripts/user_data.sh",
  "IBMCLOUDGEN2_MACHINE_PREFIX": "ibm-gen2host",
  "LogLevel": "DEBUG"
}

Edit the files for each function that you specified in the ibmcloudgen2_config.json file and configure the necessary information.
1. For the credentials file, specify your IBM Cloud API key for the VPC_APIKEY and RESOURCE_RECORDS_APIKEY parameters.
  
  For more information on creating and using IBM Cloud API keys, see Managing user API keys (https://cloud.ibm.com/docs/account?topic=account-userapikey).
  
  Use the default values for the other parameters.
2. Copy your SSH public key to the specified id_rsa file path.
3. For the user_data.sh file, use the example user_data.sh script file and modify it for your particular environment.

Edit the ibmcloudgen2_templates.json file to define the mapping between LSF resource demand requests and IBM Cloud Gen 2.

The default location for the file is

<LSF_TOP>/conf/resource_connector/ibmcloudgen2/conf/ibmcloudgen2_templates.json

For example:

{
    "templates": [
        {
            "templateId": "ibm-gen2-vm-1",
            "maxNumber": 10,
            "attributes": {
                "type": ["String", "X86_64"],
                "ncores": ["Numeric", "1"],
                "ncpus": ["Numeric", "1"],
                "mem": ["Numeric", "8192"],
                "zone": ["String", "dallas01"],
                "ibmgen2host": ["Boolean", "1"]
            },
            "imageId": "r006-3c8a935c-2622-4e68-ab59-5a031aac9d78",
            "subnetId": "0717-0ca5f395-caec-4d4c-aad6-9f5f8d493c8e",
            "vpcId": "r006-37a75f6d-36cc-4b78-aa20-c558410fc46f",
            "resourceGroupId": "0d4b98578d5a474489b928cae03767d0",
            "vmType": "bx2-2x8",
            "securityGroupIds": ["r006-1e7d19a0-c530-4e02-9b86-e1ccd1d75cee"],
            "sshkey_id": "r006-691a1dff-c977-4f96-bc63-821cea5aa0a4",
            "region": "us-south",
            "zone": "us-south-1",
            "priority": "18",
            "userData": "zone=dallas01"
        }
    ]
}

Restart the LSF daemons for the changes to take effect.
```
lsadmin limrestart
badmin mbdrestart
```
Verify that the ebrokerd process is running.
```
ps -ef | grep ebrokerd
```