Installing and configuring the IPython plug-in

The IPython plug-in for LSF allows the IPython interactive computing environment to be launched across hosts spanning the LSF cluster. LSF will pick the best suited hosts to launch the IPython worker nodes in the cluster.

Refer to https://ipyparallel.readthedocs.io/en/latest/process.html for more information on IPython’s architecture for parallel and distributed computing

Before you begin

Note: The Analytics Integration Kit is supported on Linux x86 64-bit and Linux Power 8/9 64-bit.

The following are requirements and dependencies for the IPython plug-in:

Table 1. Requirements for IPython Plug-in
Software	Version
LSF	10.1.0.7 or later
Analytics Integration Kit	10.2 or later
IPython	5.8 or later
Python	2.7, or 3.3 or greater

LSF is installed.
Python, IPython and IPython Parallel are installed on one or more LSF server host.
IPython can be installed with pip using the pip install ipython command. For details refer to the installation guide: https://ipython.readthedocs.io/en/stable/install/install.html

IPython Parallel can be installed with pip using the pip install ipyparallel command. For details refer to the installation guide: https://ipyparallel.readthedocs.io/en/latest/
Test for successful installation of Python, IPython, and IPython Parallel:
- Python: Enter the python command on the LSF server host.
- IPython: Enter the ipython command on the LSF server host.
- IPython Parallel: Enter the ipcluster and ipcontroller commands on the LSF server host.

Procedure

Install the IPython plug-in:
1. Navigate to your extracted packages.
  
  The packages are extracted when the IBM Spectrum Computing Suite for High Performance Analytics (HPA) bin is run.
  
  For example: ./ shpa10.2_lsfs_linux-x86_64.bin
2. Navigate to the Analytics Integration Kit sub-directory
  
  For example: cd ./shpa10.2_lsfs_linux-x86_64/Analytics_Integration_Kit
3. Source the LSF profile:
  For example:
  - For csh or tcsh: %source <LSF_TOP>/conf/cshrc.lsf
  - For sh, ksh, or bash: $ . <LSF_TOP>/conf/profile.lsf
4. Configure the required parameters in the install.config file:
  
  Set IPYTHONDIR to a shared directory in which to store IPython configuration files, history, commands, and extensions.
5. Execute the Analytics Integration Kit installation script.
  
  For example: integrationkit-install -f install.config
Configure the IPython plug-in:
1. Configure a static boolean resource in LSF to indicate which hosts have IPython installed.
  For example, in $LSF_ENVDIR/lsf.shared:
```
Begin Resource
RESOURCENAME  TYPE    INTERVAL INCREASING  DESCRIPTION
 IpythonHost Boolean  ()   ()       (Boolean resource to indicate IPython is installed)
End Resource
```
2. Add the resource to the hosts that have IPython installed in your LSF cluster file under $LSF_ENVDIR/lsf.clsuter.<clustername>
  For example:
```
Begin   Host
HOSTNAME  model    type        server  RESOURCES   
    host1  !        !        1    (mg IpythonHost)
    host2  !        !        1    (IpythonHost)
End     Host
```
3. Enable the interactive queue if it is not already enabled.
  
  Check the output of the bqueues command to see if the interactive queue is enabled.
4. Reconfigure LSF with the changes using the lsadmin reconfig and badmin reconfig commands.