Setting up Apache NiFi for Nokia NSP Technology Pack

Set up and start Apache NiFi to convert the data files that are collected from Nokia NSP servers. Convert the data to Avro format records and write them to Kafka. These records are then picked up by File Collector for processing the metrics and device inventory data.

Before you begin

The following tasks must be completed before you start the configurations on Apache NiFi for Nokia NSP Technology Pack:
  • Configure the Open VPN to connect to Nokia servers.
  • Encrypt the password that is sent to you from Nokia to Base64 format. You can use any online tool. For example, https://www.base64encode.org/.
  • Log in to the Nokia server by using the credentials.
  • Get the keystore and truststore certificates from the Nokia servers and replace them at /packs/nifi-collector/config/nokia-nsp/ssl folder.

NiFi setup tasks

Following are the major configuration tasks that needed to set up Apache NiFi to start collecting metrics and inventory metadata from NFM-P server:

Access the NiFi UI


Follow these steps to access the NiFi web interface on OpenShift® Container Platform:
  1. Log in to your cloud platform web console of your cluster.
  2. Make sure you are in tncp project or namespace.
  3. Navigate to Networking > Routes.
  4. Click the NiFi route link.

    You can see the UI that has a canvas to orchestrate a data flow for the installed File-based Technology Packs:

  1. Open a web browser and type the following URL on Kubernetes cloud platform:

    http://<node_hostname>:30026/nifi

    Where, <node_hostname> is the hostname of any node in your cluster.

    30026 is the port number of the NiFi Service on the node where it is installed.

    You can see the UI that has a canvas to orchestrate a data flow for the installed File-based Technology Packs:

NiFi flow in the Nokia NSP Technology Pack project is divided into the following sections:
  • FAILOVER FLOW
  • AUTHENTICATION
  • INVENTORY FILES CREATION
  • COLLECTING AND STORING THE INVENTORY FILES
  • SUBSCRIPTION FOR METRIC CLASSES
  • METRIC FILES CREATION
  • CONVERT TO AVRO RECORDS

Configure the variables for failover setup

Variables are created and configured within the NiFi UI. Variables are defined at the Process Group level. Variables are defined in and are available to any processor defined at that level and descendants. Follow these steps to update the global SDN and NFMP variables:
  1. In the FAILOVER FLOW section, right-click the project canvas with nothing selected and select Variables from the menu.
  2. In the Variables window, provide values for the following variables:
    Nokia NSP Pack project variable configurations
    Note: You can see all the referencing processors for each of the variable.
    Variable name Value
    NFMP IP address of the NFMP server
    SDN IP address of the SDN server
    aux IP address of the aux server
    file_path  
    secondary_NFMP IP address of the secondary NFMP server for failover
    secondary_SDN IP address of the secondary SDN server for failover
  3. Click Apply.
  4. Start all the processors under Failover flow.

Authenticate to the NFM-P servers

The InvokeHTTP processor is configured to authenticate to the NFM-P server and to obtain a response token from the server. Authentication is mandatory to fetch file-based data and subscription data from the server. The response bearer token is refreshed after every hour. The API call is as follows:
https://{SDN}:{port}/rest-gateway/rest/api/v1/auth/token

Telco Network Cloud Manager - Performance can connect to the Nokia NSP server successfully after the SSL check in controller servers and authentication.

  1. In the AUTHENTICATION section, right-click the InvokeHTTP processor and select Configure.
  2. Provide values for the following properties:
    InvokeHttp processor configuration
    Parameter Value
    Authorization The password that is sent to you from Nokia that is encrypted to Base64 format.
  3. Right-click on the project canvas with nothing selected and select Configure.
  4. Click the View Configuration (View Configuration icon) icon on the StandardSSLContextService controller service.
  5. In the Controller Service Details window, provide values for the following properties:
    Authentication configurations
    Parameter Value
    Keystore Filename Location where the keystore file is available. By default, it is /packs/nifi-collector/config/nokia-nsp/ssl.
    Keystore password Keystore password.
    Truststore Filename Location where the truststore file is available. By default, it is /packs/nifi-collector/config/nokia-nsp/ssl.
    Truststore Password Truststore password.
  6. Click OK.
  7. Click the Enable () icon on the StandardSSLContextService controller service.
  8. Disable the EmbeddedDbService controller service and enable again.
  9. Start all the processors under AUTHENTICTION flow.

Create the inventory files

Inventory classes request is sent with the bearer token to the server. A non-readable 0-bytes response is sent from the server. The fileAvailable event notification is received from Kafka. Then, the inventory files are created at the specified location in file name attribute in request. You can fetch these XML files through FTP from that location. By default, the inventory files are fetched once a day. The files are stored in a temporary database in NiFi for further processing. The API call is as follows:
https://${NFMP}:{port}/nfm-p/rest/api/v2/general/findToFile/
  1. In the INVENTORY FILES CREATION flow, right-click the GetInventoryFile processor and select Configure.
  2. Provide values for the following property:
    Property Value
    Input Directory Location where the Inventory_Class_Request.json file is available. By default, it is ${content_dir}/nifi_collector}/config/${pack_name}/inventory
  3. Click Apply.
  4. Start all the processors in the INVENTORY FILES CREATION flow.

Collect and store the inventory files

  1. In the COLLECTING AND STORING THE INVENTORY FILES section, right-click the ListSFTP and FetchSFTP processors and select Configure.
  2. Provide values for the following properties:
    Property Value
    Username ossuser
    Password Specify the password that you received from Nokia for the ossuser user.
  3. Click Apply.
  4. Start all the processors in the COLLECTING AND STORING THE INVENTORY FILES flow.

Create the subscription ID for metric classes

A subscription is created for all the metric classes and the subscription is bearer token to the InvokeHttp processor. Subscription ID and topic ID are returned in the response. The API call is as follows:
https://{SDN}:{port}/nbi-notification/api/v1/notifications/subscriptions
  1. In the SUBSCRIPTION FOR METRIC CLASSES section, right-click the GetEventSubscription_RequestFile processor and select Configure.
  2. Provide values for the following properties:
    Property Value
    Input Directory Location where the Event_Subscription_Format.json file is available. By default, it is ${content_dir}/nifi_collector}/config/${pack_name}/metric
  3. Click Apply.
  4. Start all the processors in the SUBSCRIPTION FOR METRIC CLASSES flow.

Create metric files

Metric classes request is sent with the bearer token and the subscription ID to the server. A non-readable 0-bytes response is sent from the server. The logfileAvailable event notification is received from Kafka. Then, the metric files are created at the specified location in file name attribute in request.
Note: OAM metric files are created with findToFile event notifications from Kafka.

The metric files are received in XML format. The metric classes are then processed and converted to Avro Records and passed as input to the File Collector Service.

  1. In the METRIC FILES CREATION section, right-click the GetOAM_MetricRequestFile and GetMetricRequestFile processors.
  2. Select Configure and provide values for the following properties:
    Processor Property Value
    GetEventSubscription_RequestFile Input Directory Location where the OAM_Metric_Class_Request.json file is available. By default, it is ${content_dir}/nifi_collector}/config/${pack_name}/metric
    GetMetricRequestFile Input Directory Location where the Metric_Class_Request.json file is available. By default, it is ${content_dir}/nifi_collector}/config/${pack_name}/metric
  3. Click Apply.
  4. Start all the processors in the METRIC FILES CREATION flow.

Convert to AVRO record format

  1. In the CONVERT TO AVRO RECORDS section, right-click the FetchSFTMMetricFiles processor and select Configure.
  2. Provide values for the following properties:
    Property Value
    Username ossuser
    Password Specify the password that you received from Nokia for the ossuser user.
  3. Click Apply.
  4. Start all the processors in the CONVERT TO AVRO RECORDS flow.