Defining OpenStack Observer jobs

The OpenStack Observer is installed as part of the core installation procedure. Using the OpenStack Observer, you can define jobs that dynamically load OpenStack data for analysis by Netcool Agile Service Manager.

Before you begin

Important: The OpenStack Observer supports the on-premise OpenStack versions Ocata, Pike, Rocky, Queens, Stein, Train, Zed and Antelope.

Ensure you have the OpenStack service details to hand, such as username, password, and URL.

Remember: Swagger documentation for the observer is available at the following default location: https://<your host>/1.0/openstack-observer/swagger
Note: OpenStack uses RBAC-based protection of its API by defining policy rules based on an RBAC approach. Availability of resources retrieved by the observer is also governed by the same policy. For example, a VM created in project A by users with the admin role may only be available to other users with the same admin role. This can be configured or modified according to user requirements in the OpenStack's policy configuration.

About this task

The OpenStack Observer jobs extract OpenStack resources via REST or RabbitMQ. The Observer loads and updates the resources and their relationships within the Netcool Agile Service Manager core topology service.
openstack_observer_common.sh
The config file you use to customize OpenStack Observer settings.
The parameters defined here are then used by the openstack_observer_query_start.sh and the openstack_observer_listen_start.sh scripts to trigger the OpenStack Observer jobs.
You define and start the following two jobs. You must edit the parameters in the config file before running these jobs.
Restapi Load job
A transient (one-off) job that loads all requested topology data.
This job is started by the openstack_observer_query_start.sh script.
Restriction: An OpenStack environment that has a list of endpoints whereby the heat-cfn service comes first before the heat service, will encounter a JSON parsing error in the logs due to a known issue in the openstack4j library. When this happens, the full load for the heat service will be skipped entirely. The other service will run as normal.
Rabbitmq Listen job
A long-running job that monitors its source for updates and runs until it is explicitly stopped, or until the Observer is stopped.
This job is started by the openstack_observer_listen_start.sh script.
Restriction: Only one listening job should be listening to one queue (or sets of queues) at any one time. If you need to listen to multiple projects, then separate queues must be set up in OpenStack, with appropriate names, before separate listening jobs are submitted for each. For example, for Nova via the rmq_nova_notify attribute, for Neutron via the rmq_neutron_notify attribute.
Tip: Alternatively, you can set the appropriate environment variables. If an environment variable is set, it takes precedence over the config file settings.

Procedure

To edit the parameters in the configuration file

  1. Open the openstack_observer_common.sh configuration file and edit the following parameters:
    Table 1. OpenStack Observer restapi job parameters
    Parameter Action Details
    Job type Select either restapi load or rabbitmq listen. Required
    Unique ID Enter a unique name for the job Required
    OpenStack authentication type Specify the OpenStack connection authentication technique to use. Required. Choose either v2_Tenant, v3_Unscoped, v3_Project, or v3_Domain.
    OpenStack password Specify the OpenStack password with which to authenticate. Required. Must be encrypted.
    OpenStack identity endpoint Specify the authentication URL. Required. Must include the port and version.
    Data center name Specify the name of the data center in which the OpenStack instance is running. Required. If more than one OpenStack instance is run, and duplicate project or tenant names exist, you must disambiguate them here.
    OpenStack username Specify the OpenStack user name to connect as (or to). Required
    OpenStack project name Specify the OpenStack project name, for example the value of OS_PROJECT_NAME. Multi project names must be separated by comma. Required
    OpenStack project domain name Specify the project domain name to use, for example the value of OS_PROJECT_DOMAIN_NAME Optional
    OpenStack user domain name Specify the user domain name to use, for example the value of OS_USER_DOMAIN_NAME Optional
    OpenStack region name Specify the OpenStack region, for example the value of OS_REGION_NAME. The value is case-sensitive. If it is not set, the default value is regionOne. Required for v3 authentication
    OpenStack perspective Select the URL perspective the API accesses data from. Optional. Choose from Admin, Public, and Internal.
    Connection and read timeout (ms) Choose the timeout setting for the connection and read actions. Optional. The default is 5000 (5 seconds).
    Optionally load hypervisors Choose whether to turn the loading of hypervisors on or off. This option requires the os_compute_api:os-hypervisors privilege granted by the OpenStack administrator. Optional. The default is false (off).
    Optionally load Host Aggregates Choose whether to turn the loading of Host Aggregates on or off. This option requires the os_compute_api:os-aggregates:index privilege granted by the OpenStack administrator. Optional. The default is false (off).
    SSL Verification Choose whether to use SSL verification (true or false). If false, HTTPS is used, but without hostname validation. Optional
    OpenStack host certificate Specify a certificate name to load into the trust store. Optional. For details, see Configuring observer job security
    SSL truststore file name Specify a truststore file name. Provide the JKS name, and the observer will then create the JKS file accordingly.
    SSL truststore file password Specify a truststore file password.

    Required. Must be encrypted.
    Proxy username Specify the proxy username Optional
    Proxy password Specify the proxy password Optional
    Proxy host Specify the proxy host to connect via Optional
    Proxy port Specify the proxy port to connect via Optional. Default is 8080.
    Job schedule

    Specify when the job should run, and whether it should run at regular intervals.

    By default the job runs immediately, and only once.

    Optionally you can specify a future date and time for the job to run, and then set it to run at regular intervals after that.

    Optional. Transient (one-off) jobs only.

    If you set a job schedule, the run intervals must be at least 90 seconds apart, and if you set them at less than 15 minutes, a warning is displayed, as the frequency can impact system performance.
    Observer job description Enter additional information to describe the job. Optional
    For encryption: Run the encrypt_password.sh script in the ASM_HOME/bin directory: 
    ./bin/encrypt_password.sh
    Enter and then confirm the password. The encryption utility will return an encrypted version.
    Table 2. OpenStack Observer rabbitmq job parameters
    Parameter Action Details
    Unique ID Enter a unique name for the job Required
    RabbitMQ username Specify the AMQP user name to connect to the broker. Required
    RabbitMQ password Specify the password to use to connect to the broker. Required. Must be encrypted.
    RabbitMQ hosts Enter a (comma-separated) list of hosts in the RabbitMQ cluster. Required. Add a comma-separated list of host(s) (for example, host1:5672,host2:5672) to connect to in the RabbitMQ cluster. The first host connected to successfully is used.
    Note: Use IP addresses instead of hostnames.
    Data center name Specify the name of the data center in which the OpenStack instance is running. Required. If more than one OpenStack instance is run, and duplicate project or tenant names exist, you must disambiguate them here.
    OpenStack username Specify the OpenStack user name to connect as (or to). Required
    OpenStack project name Specify the OpenStack project. Optional
    RabbitMQ virtual host name Specify the virtual host to connect to the broker. Optional
    Use SSL? Choose whether to use an SSL connection. Optional. Choose true or false. For RabbitMQ, you must choose true.
    Nova v2 Oslo message queue Specify the Nova v2 Oslo message queue. Optional
    Neutron v2 Oslo message queue Specify the Neutron v2 Oslo message queue. Optional
    Cinder v2 Oslo message queue Specify the Cinder v2 Oslo message queue. Optional
    Heat v2 Oslo message queue Specify the Heat v2 Oslo message queue. Optional
    Number of consumer instances Specify the number of consumer instances to create for each API queue type. Optional
    Observer job description Enter additional information to describe the job. Optional

To configure the OpenStack installation method

  1. Do one of the following depending on whether you have used, or are planning to use, DevStack or another method to install OpenStack.
    • DevStack installation
      If you have already installed OpenStack using DevStack
      Add the following code to the end of the local.conf file, and then reinstall OpenStack.
      If you are planning to install OpenStack using DevStack
      Add the following code to the end of the local.conf file before installation.
      [[post-config|$NOVA_CONF]] 
[DEFAULT]
notification_topics = notifications,com.ibm.asm.obs.nova.notify
notification_driver=messagingv2 
notify_on_state_change=vm_and_task_state 
notify_on_any_change=True
    • Other installation
      For standard (or any other) OpenStack installations
      Add the following code under the [DEFAULT] section of the nova.conf file, and then restart the nova compute service.
      notification_topics = notifications,com.ibm.asm.obs.nova.notify
notification_driver=messagingv2 
notify_on_state_change=vm_and_task_state 
notify_on_any_change=True
      To enable port numbers for standard OpenStack installations
      OpenStack assigns specific port numbers to each of its service. For Agile Service Manager to communicate with the OpenStack instance, you must enable the following default ports.
      Tip: Check with your OpenStack administrator if custom port numbers have been configured.
      Load jobs
      Default ports:
      • KeyStone: 5000
      • Nova: 8774
      • Neutron: 9696
      • Glance: 9292
      • Cinder: 8776
      • Heat: 8004
      Listen jobs
      Default port:
      • RabbitMQ: 5672

To start the Load and Listener jobs

  1. To start the OpenStack Observer Full Topology Upload job, use the following command: 
    $ASM_HOME/bin/openstack_observer_query_start.sh
    The Full Topology Upload job loads all requested topology data. This job runs only once.
  2. To start the OpenStack Observer listener job, use the following command: 
    $ASM_HOME/bin/openstack_observer_listen_start.sh
    The Listener job monitors its source for updates and runs until it is explicitly stopped, or until the Observer is stopped.

What to do next

You can also use the following scripts:
openstack_observer_query_stop.sh
Stops the Full Topology Upload job
openstack_observer_listen_stop.sh
Stops the Listener job
openstack_observer_job_list.sh
Lists the status of current jobs
openstack_observer_log_level.sh
Sets the log level
Remember: As an alternative to being configured using the Observer Configuration UI, observer jobs have scripts to start and stop all available jobs, to list the status of a current job, and to set its logging levels. These scripts can be run with -h or --help to display help information, and with -v or --verbose to print out the details of the actions performed by the script, including the full cURL command. For the on-prem version of Agile Service Manager, observer scripts are configured for specific jobs by editing the script configuration files.