Configuring OpenStack Observer jobs
Using the OpenStack Observer, you can configure jobs that dynamically load OpenStack data for analysis.
Before you begin
Important: The OpenStack Observer supports the on-premises OpenStack up to versions Ocata, Pike, Rocky, Queens, Stein, Train, Zed and Antelope.
Ensure you have the OpenStack service details to hand, such as the parameters for its APIs or RabbitMQ message bus. If you are configuring a query job, have OpenStack location and authorization details to hand. If you are configuring a rabbitmq job, you must also identify and provide access to the RabbitMQ message bus.
OpenStack installation requirements
If you previously installed OpenStack with DevStack, you must add the configuration that is specified here to the end of the local.conf file, and reinstall it. If you previously installed OpenStack with another installation method, you must add the configuration that is specified here to the nova.conf file, and then restart the Nova (compute) service.
- If you have already installed OpenStack using DevStack, add the following configuration to the end of the local.conf file, and then reinstall OpenStack.
- If you are planning to install OpenStack using DevStack, add the following configuration to the end of the local.conf file before installation.
[[post-config|$NOVA_CONF]]
[DEFAULT]
notification_topics = notifications,com.ibm.asm.obs.nova.notify
notification_driver=messagingv2
notify_on_state_change=vm_and_task_state
notify_on_any_change=True
For standard (or any other) OpenStack installations, add the following configuration under the [DEFAULT] section of the nova.conf file, and then you restart the Nova (compute) service.
notification_topics = notifications,com.ibm.asm.obs.nova.notify
notification_driver=messagingv2
notify_on_state_change=vm_and_task_state
notify_on_any_change=True
Note: OpenStack uses RBAC-based protection of its API by defining policy rules based on an RBAC approach. Availability of resources retrieved by the observer is also governed by this same policy. For example, a VM created in project A by users with the admin role may be available to other users with the same admin role. This can be configured or modified according to user requirements in the OpenStack's policy configuration.
OpenStack port numbers
OpenStack assigns standard port numbers to each of its service. To communicate with the OpenStack instance, you must enable the following default ports.
Tip: Check with your OpenStack administrator if custom port numbers have been configured.
Load jobs default ports:
- KeyStone: 5000
- Nova: 8774
- Neutron: 9696
- Glance: 9292
- Cinder: 8776
- Heat: 8004
Listen jobs default port:
- RabbitMQ: 5672
Troubleshooting
A Certificate Chaining Error can occur when launching an OpenStack Observer job.
Enabling access to the URL routes
To access the URL routes for the topology Swagger documentation, see the Enabling access to URL routes topic.
About this task
The OpenStack Observer jobs extract OpenStack resources through REST or RabbitMQ, and the observer loads and updates the resources and their relationships within the topology service.
You configure and run the following two jobs.
Restapi Load job
A transient (one-off) job that loads all requested topology data from the OpenStack instance by REST API.
By default, Load jobs are one-off, transient jobs that do a full upload of all requested topology data as soon as they are triggered.
You can also run these jobs (again) from the Observer UI, or schedule them to run at set times when configuring them.
The job loads baseline topology data through the following OpenStack's APIs:
- Keystone (identity)
- Cinder (block storage)
- Glance (image)
- Heat (orchestration)
- Neutron (network)
- Nova (compute)
Restriction: An OpenStack environment that has a list of endpoints whereby the 'heat-cfn' service comes first (before the 'heat' service) will encounter a JSON parsing error recorded in the logs due to a known issue in the openstack4j library. When this happens, the full load for the heat service will be skipped entirely. Other services will run as normal.
Rabbitmq Listen job A long-running job that reads messages on OpenStack's RabbitMQ message bus for activity from the Cinder (block storage), Heat (orchestration), Neutron (network) and Nova (compute) components continually, until it is explicitly stopped, or until the Observer is stopped.
Note: The rabbitmq job should only be run after an initial restapi job has been completed.
Restriction: Only one rabbitmq job should be listening to one queue (or sets of queues) at any one time. If you need to listen to multiple projects, then separate queues must be set up in OpenStack, with appropriate names, before separate listening jobs are submitted for each. For example, for Nova through the rmq_nova_notify attribute, for Neutron through the rmq_neutron_notify attribute.
Procedure
Define or edit the following parameters, then click Run job to save and run the job.
Encryption requirement: See the Configuring observer jobs security topic for more information.
Restapi Load job parameters
| Parameter | Action | Details |
|---|---|---|
| Unique ID | Enter a unique name for the job | Required |
| Data center name | Specify the name of the data center in which the OpenStack instance is running | Required. If more than one OpenStack instance is run, and duplicate project or tenant names exist, you must disambiguate them here. |
| OpenStack Project/Tenant name | Specify the OpenStack project name(s) to use, for example the value of OS_PROJECT_NAME. Multi project names should be separated by comma. |
Required |
| OpenStack authentication type | Specify the OpenStack connection authentication technique to use | Required. Choose either v2_Tenant, v3_Unscoped, v3_Project, v3_Domain, or v3_ProjectDomain`. |
| OpenStack identity endpoint | Specify the OpenStack Identity Endpoint to connect to inclusive of port and version, e.g. the value of OS_AUTH_URL. |
Required |
| OpenStack username | Specify the OpenStack username to connect as (or to) | Required |
| OpenStack password | Specify the OpenStack password with which to authenticate | Required |
| OpenStack host certificate | Specify a certificate name to load into the truststore | Required |
| SSL truststore file name | Specify the truststore file name. The observer generates the trust store file based on the file name provided. Tip: You can use the observer name as file name (<observer>.jks), for example OpenStack.jks. |
Required. The supported format is JKS and the file is relative to $ASM_HOME/security. |
| SSL truststore file password | Specify the truststore password the observer will use to decrypt the HTTPS truststore file. Use a password that conforms to your internal security requirements. | Required |
| OpenStack user domain name | Specify the user domain name to use, for example the value of OS_USER_DOMAIN_NAME. |
Optional |
| OpenStack project domain name | Specify the project domain name to use, for example the value of OS_PROJECT_DOMAIN_NAME. |
Optional |
| OpenStack region name | Specify the OpenStack region to use, for example the value of OS_REGION_NAME The value is case-sensitive. If it is not set, the default value is regionOne. |
Required for v3 authentication. |
| OpenStack perspective | Select the URL perspective the API accesses data from | Optional. Choose from Admin, Public, and Internal. |
| SSL Verification | Choose whether to use SSL verification (true or false). If false, HTTPS is used, but without hostname validation. | Optional |
| Optionally load hypervisors | Choose whether to turn the loading of hypervisors on or off. Note: To load hypervisors, admin privileges must be assigned to the user. | Optional. The default is false (off). |
| Optionally load Host Aggregates | Choose whether to turn the loading of Host Aggregates on or off. Note: To load host aggregates data, admin privileges must be assigned to the user. | Optional. The default is false (off). |
| Connection timeout (ms) | Choose the timeout setting for the connection actions | Optional. The default is 5000 (5 seconds). |
| Read timeout (ms) | Choose the timeout setting for the read actions | Optional. The default is 5000 (5 seconds). |
| Proxy host | Specify the proxy host to connect via | Optional |
| Proxy username | Specify the proxy username. | Optional |
| Proxy password | Specify the proxy password | Optional |
| Proxy port | Specify the proxy port to connect via | Optional. Default is 8080. |
| Trust all certificates | Set to true to connect to target environment without verification |
Optional |
| Access scope | Enter text to provide a scope for the resources. Access scope can help map alerts to resources when resources in different scopes share the same parameters, such as matchTokens. | Optional. Tip: You can define access scope for locations, project names, namespaces, etc. |
| Generate debug support file | Set this parameter to 'True' in order to capture the output of the next scheduled job run as a file. This file will be stored with an observer's log files and can be used to debug observer issues, for example at the request of your designated support team, or while using a test environment. For one-off jobs (that is, Load jobs), this parameter reverts to 'False' after the next completed run. To examine the output produced, you can load the generated debug file using the File Observer. | Optional |
| Observer job description | Enter additional information to describe the job | Optional |
| Job schedule | Specify when the job runs | Optional. Load jobs only. |
Rabbitmq Listen job parameters
| Parameter | Action | Details |
|---|---|---|
| Unique ID | Enter a unique name for the job | Required |
| Data center name | Specify the name of the data center in which the OpenStack instance is running | Required. If more than one OpenStack instance is run, and duplicate project or tenant names exist, you must disambiguate them here. |
| OpenStack Project/Tenant name | Specify the project or tenant name(s) to use, such as the value of `OS_PROJECT_NAME. Multi-project or tenant names should be separated by comma. | Optional |
| OpenStack username | Specify the OpenStack username to connect as (or to) | Required |
| RabbitMQ username | Specify the AMQP username to connect to the broker | Required |
| RabbitMQ hosts | Enter a (comma-separated) list of hosts in the RabbitMQ cluster | Required. Add a comma-separated host, or list of hosts, (for example, host1:5672,host2:5672) to connect to in the RabbitMQ cluster. The first host connected to successfully is used. Note: Use IP addresses
instead of hostnames. |
| RabbitMQ password | Specify the password to use to connect to the broker | Required |
| RabbitMQ virtual host name | Specify the virtual host to connect to the broker | Optional |
| Use SSL | Choose whether to use an SSL connection | Optional. Choose true or false. For RabbitMQ, you must choose true. |
| Nova v2 Oslo message queue | Specify the Nova v2 Oslo message queue | Optional |
| Neutron v2 Oslo message queue | Specify the Neutron v2 Oslo message queue | Optional |
| Cinder v2 Oslo message queue | Specify the Cinder v2 Oslo message queue | Optional |
| Heat v2 Oslo message queue | Specify the Heat v2 Oslo message queue | Optional |
| Number of consumer instances | Specify the number of consumer instances to create for each API queue type | Optional |
| Access scope | Enter text to provide a scope for the resources. Access scope can help map alerts to resources when resources in different scopes share the same parameters, such as matchTokens. | Optional. Tip: You can define access scope for locations, project names, namespaces, etc. |
| Generate debug support file | Set this parameter to 'True' in order to capture the output of the next scheduled job run as a file. This file will be stored with an observer's log files and can be used to debug observer issues, for example at the request of your designated support team, or while using a test environment. For one-off jobs (that is, Load jobs), this parameter reverts to 'False' after the next completed run. To examine the output produced, you can load the generated debug file using the File Observer. | Optional |
| Observer job description | Enter additional information to describe the job | Optional |
| Job schedule | Specify when the job runs | Optional. Load jobs only. |
Important: You must specify the following properties consistently for both the restapi and rabbitmq jobs:
- Data center name
- OpenStack project name
- OpenStack username