Check the dependencies for optimized scanning

Identify the dependencies that must be met on the IBM Storage Scale system to optimize automated scan ingest.

The dependencies that must be satisfied on the IBM Storage Scale system to optimize automated scan ingest from IBM Spectrum® Discover are:
  • A librdkafka library 0.11.4 or later
  • A Python 3.0 or later with accompanying Python package installer (pip3)
  • A confluent-kafka version that is greater than or equal to the installed librdkafka version.

If these dependencies are met, the scan output is pushed to the ingest Kafka queue of IBM Spectrum Discover system directly from the IBM Storage Scale policy output.

An administrator can determine whether librdkafka is installed on the IBM Storage Scale node by running the find /usr -name "*librdkafka*" or ls /lib64/librdkafka* commands. The librdkafka package is included with newer levels of IBM Storage Scale on x86 and ppc64le platforms. However, it can also be built from the source code on older levels of IBM Storage Scale or ppc64 platforms. If the IBM Storage Scale system runs on Red Hat® Enterprise Linux® (RHEL) and is connected to a Red Hat Satellite, you can install it by running the Yellowdog Updater Modified (YUM) command yum install librdkafka as root. You can find source packages of librdkafka here: https://github.com/edenhill/librdkafka

The user ID specified in the data source connection must be able to locate the following two binaries by using the OS shell path:

  1. A Python 3 binary as either python or python3
  2. A Python package installer as pip3
    Note: Symbolic links or aliases may be used to locate the Python executables.

After you install a sufficient version of Python, you can install confluent-kafka by using pip. To get pip, you must install the python-setuptools package, which provides a binary called easy_install. For more information, see https://pypi.org/project/setuptools/#files

After easy_install is available, you can install pip by running easy_install-2.7 pip as root. After you install pip, you can install confluent-kafka by running pip install confluent-kafka as root.