Defining Network Discovery Observer jobs

You use the Network Discovery Observer to retrieve network topology data from the Network Discovery database via REST API and use this data to create topologies within the topology service.

Before you begin

The Network Discovery Services are installed as part of the core installation procedure. This includes the Network Discovery Observer, Network Discovery Collector and Network Discovery Engine, which have been installed and should be running, as well as the required scripts to manage jobs.

Before you configure a network discovery job, you must configure the discovery details. This procedure is described here: Configuring the network discovery services

When successful, the configuration details will be saved in topology management artifacts.

All observers have scripts to start and stop all available jobs, to list the status of a current job, to set its logging levels, and to configure its job parameters.

Network Discovery Sizing: The Agile Service Manager network discovery service supports a maximum number of discovered entities.
For size0
10,000 entities
For size01
1,000,000 entities
The number of entities created by a discovery depend on factors such as the number of devices in scope, and how many interfaces, cards, VLANs and so on are present on each device.

Verify the Network Discovery deployment before configuring the observer jobs.
  1. Use the following command to verify that the docker containers are running:
    ASM_HOME/bin/docker-compose ps
    The system should return output indicating that the following containers have a state of Up and therefore are running:
    asm_netdisco-observer_1       sh -c /opt/ibm/$APPNAME/st    Up   9152/tcp, 9153/tcp
    nasm-net-disco-collector      ./entrypoint.sh               Up   8443/tcp, 8445/tcp
    nasm-net-disco-config         ./entrypoint.sh               Up   8443/tcp
    nasm-net-disco-consul         /etc/init.sh agent -server    Up   0.0.0.0:8501->8501/tcp
    nasm-net-disco-control        ./entrypoint.sh               Up   8443/tcp
    nasm-net-disco-schema-        /etc/init.sh                  Up   8081/tcp,
    nasm-net-disco-security       ./entrypoint.sh               Up   8443/tcp
    nasm-net-disco-sftp           ./entrypoint.sh               Up   0.0.0.0:9022->9022/tcp
    nasm-net-disco-sidecar        ./entrypoint.sh               Up   0.0.0.0:8765->8443/tcp
    nasm-net-disco-status         ./entrypoint.sh               Up   8443/tcp
    nasm-net-disco-swagger-       /etc/init.sh                  Up   8443/tcp,
    nasm-net-disco-swagger-ui     /etc/init.sh                  Up   8443/tcp
    nasm-net-disco-topogram       ./entrypoint.sh               Up   8443/tcp
    nasm-net-disco-worker         /bin/sh -c /opt/$SERVICE_N    Up
  2. Check the status of the storage and discovery services. The status of the network discovery services can be checked in the Consul UI on port 8501, for example:
    https://localhost:8501/ui/nasm_net_disco/services
Known issue and workaround: The Kafka service may not yet be fully initialized when the nasm-net-disco-schema-registry is started during an Agile Service Manager start up or restart, and it may subsequently fail a consul health check.
Workaround
  1. Verify via consul that the nasm-net-disco-schema-registry has failed its healthcheck:
    https://localhost:8501/ui/nasm_net_disco/services
  2. Restart the nasm-net-disco-schema-registry:
    docker restart nasm-net-disco-schema-registry
  3. Wait until all Agile Service Manager services have registered with consul, and only then run the network discovery.
Remember: Swagger documentation for the observer is available at the following default location: https://<your host>/1.0/netdisco-observer/swagger
Important:
OCP requirement: On the OCP hosts, network discovery requires that pids_limit be set at least to 44406 inside the crio.conf file.
For information about changing the values in the crio.conf file using Machine Configs, see Creating a ContainerRuntimeConfig CR to edit CRI-O parameters
For reference information about Machine Configs, see Red Hat Enterprise Linux CoreOS (RHCOS)

About this task

To use the Network Discovery component, you must complete two steps:
Configure the discovery process
Configure Network Discovery details (upon success, configuration details will be saved in the topology).
This procedure is described here: Configuring the network discovery services
Configure the full load job
Triggers Network Discovery
Loads Network Discovery data
By default, these jobs are one-off, transient jobs that perform a full upload of all requested topology data as soon as they are triggered.
You can also run these jobs (again) manually from the Observer UI, or schedule them to run at set times when configuring them.

The Network Discovery Observer receives network data from the network services. Once the discovery configuration has successfully been sent to the topology management artifact and Network Discovery engine, you can run a full load job to discover Network Discovery data.

Procedure

To edit the parameters in the configuration file

  1. Open the netdisco_observer_common.sh configuration file and edit the following parameters:
    Table 1. Network Discovery Observer job parameters
    Parameter Action Details
    Unique ID Enter a unique name for the job Required
    Config ID Specify the Network Discovery configuration ID Required
    Job schedule

    Specify when the job should run, and whether it should run at regular intervals.

    By default the job runs immediately, and only once.

    Optionally you can specify a future date and time for the job to run, and then set it to run at regular intervals after that.

    Optional. Transient (one-off) jobs only.

    If you set a job schedule, the run intervals must be at least 90 seconds apart, and if you set them at less than 15 minutes, a warning is displayed, as the frequency can impact system performance.

    Observer job description Enter additional information to describe the job. Optional
  2. To start the Network Discovery Observer Load job, use one of the following commands:
    • This command triggers the job to load all requested topology data.
      $ASM_HOME/bin/netdisco_observer_load_start.sh
    • Alternatively, you can start the load job using the following cURL command:
      curl –insecure -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'Authorization: Basic YXNtOmFzbQ==' -d '{
        "unique_id": "my job",
        "type": "string",
        "description": "job description",
        "parameters": {
          "configId": "default-netdisco"
        }
      }' 'https://localhost/1.0/netdisco-observer/jobs/load'

Results

The Network Discovery Observer job discovers and then uploads the network services data to the Agile Service Manager topology service.

What to do next

You can also use the following scripts:
netdisco_observer_load_stop.sh
Stops the load job
netdisco_observer_job_list.sh
Lists the current job status
netdisco_observer_log_level.sh
Sets the log level
netdisco_observer_discovery_status.sh
Checks the discovery status
netdisco_observer_discovery_stop.sh
Stops the discovery
Important: Do not run the netdisco_observer_discovery_stop.sh script as a standalone script. It is run by the netdisco_observer_load_stop script as part of a specific sequence to stop a network discovery before stopping an observer job.
Remember: As an alternative to being configured using the Observer Configuration UI, observer jobs have scripts to start and stop all available jobs, to list the status of a current job, and to set its logging levels. These scripts can be run with -h or --help to display help information, and with -v or --verbose to print out the details of the actions performed by the script, including the full cURL command. For the on-prem version of Agile Service Manager, observer scripts are configured for specific jobs by editing the script configuration files.
Tip: Network discovery log are described here.