Overview of IBM Spectrum Computing Suite for High Performance Analytics

IBM Spectrum Computing Suite for High Performance Analytics (HPA) combines the most current functionality from the IBM Spectrum LSF family. IBM Spectrum Computing Suite for HPA also enables the distribution of workloads for IBM InfoSphere DataStage and SAS Grid Manager.

IBM Spectrum Computing Suite for HPA features an integrated UI experience that can enhance the productivity of users through ease of use and simplification. Users have more ways to access grid resources, including mobile clients for job monitoring and notifications, an integrated desktop client for Microsoft Windows environments that provides seamless Linux cluster access, and a RESTful API for accessing the environment through web services. The extendable UI enables sites to include customer-specific customizations and extensions.

IBM Spectrum Computing Suite for HPA includes a powerful reporting capability that is based on Elasticsearch enabling administrators to quickly define reports and dashboards so that they can report on cluster and resource use, to both users and management.

IBM Spectrum Computing Suite for HPA includes capabilities to support hybrid cloud, enabling workloads to be forwarded to multiple clouds (that is, OpenStack, IBM Cloud, Microsoft Azure, Amazon EC2, and Google Compute). Additionally, data can be automatically staged to or from the cloud and the resources that are consumed on the cloud can be auto-scaled based on workload demands and scheduling policies.

IBM Spectrum Computing Suite for HPA offering components

IBM Spectrum Computing Suite for HPA combines the following IBM Spectrum LSF family products in one offering:

  • LSF

    LSF is a powerful workload management platform for demanding, distributed HPC environments. It provides a comprehensive set of intelligent, policy-driven scheduling features that enable you to use all of your compute infrastructure resources and ensure optimal application performance. The resource connector for LSF feature is also enabled for LSF clusters to borrow resources from supported resource providers.

  • LSF Data Manager

    When large amounts of data are required to complete computations, it is desirable that your applications access required data unhindered by the location of the data in relation to the application execution environment. LSF Data Manager solves the problem of data locality by staging the required data as closely as possible to the site of the application.

  • Application Center

    Application Center provides a flexible, easy to use interface for cluster users and administrators. Application Center is the full-featured offering, including job submission, monitoring, reporting, application templates, simplified application configuration, extended template and page customization, visualization support through VNC, configurable user access control, and integration with Process Manager. The Application Center desktop client is also included.

  • Process Manager

    Complex scripts are often used to automate lengthy computing tasks. But these scripts can be risky to modify, and can depend on the expertise of a few key individuals. Process Manager simplifies the design and automation of complex computational processes, capturing and protecting repeatable best practices.

  • Explorer

    Explorer is a lightweight data analysis solution for IBM Spectrum LSF clusters allowing business and technical users to rapidly create and view reports and dashboards. Explorer uses Elasticsearch to rapidly store, index, and query the data. With rich, interactive, and extensible visualization capabilities, you can generate reports on how the compute environment is performing, or which resources projects or lines or business are consuming.

  • RTM

    RTM is an operational dashboard for LSF environments that provides comprehensive workload monitoring, reporting, and management. It makes cluster administrators more efficient in their day-to-day activities and provides the information and tools needed to improve cluster efficiency, enable better user productivity, and contain or reduce costs. Dashboards provide comprehensive reports to support the day-to-day administrative tasks associated with managing single and multiple cluster environments. Timely information on the current status of the HPC environment helps improve decision-making, reduce costs, and increase service levels.

  • Analytics Integration Kit

    An included kit that provides integrated support for the orchestration of open source analytics tools including Python and Dask. Support for TensorFlow and Jupyter Notebooks are included out of the box with the Suite or the stand-alone (Application Center) installers.

  • Support for Apache Spark and R jobs is also provide. See IBM Spectrum LSF V10.2 documentation for more information

Installation options

A simplified Suite installation process is provided that uses Ansible to quickly deploy the IBM Spectrum Computing Suite for HPA components from a single server into an existing environment. A stand-alone installer is also provided for each individual components to let administrators choose which components to deploy in the environment.