Defining REST Observer jobs

The REST (or RESTful) Observer is installed as part of the core installation procedure. Use the REST Observer for obtaining topology data via REST endpoints. This observer is a counterpart to the File Observer.

Remember: Swagger documentation for the observer is available at the following default location: https://<your host>/1.0/rest-observer/swagger

The REST Observer passes topology data to Agile Service Manager using a RESTful set of interfaces, which provide REST APIs that enable the following functionality:

Management of Listen and bulk-replace job types.
The insert-update (HTTP POST) of resources.
The insert-update (HTTP POST) of relationships.
The insert-replacement (HTTP PUT) of resources.
The deletion (HTTP DELETE) of resources.
A REST API that supports the deletion (HTTP DELETE) of all relationships of a given type from a specified resource.
A REST API that supports the deletion (HTTP DELETE) of a specific relationship.
Restriction: Resources created via REST can have a provider, but not an observer.

Benefits

Using the REST Observer rather than the File Observer or Topology Service APIs includes the following benefits:

The ability to provide data to Agile Service Manager via HTTP REST Endpoints instead of files.
The processing performed by all observers in their framework ensures that meta-data about observations from observers is managed correctly.
A simple way of deleting all edges of a given type on a resource or a specific edge instance.

To use the REST Observer, a job request must be issued (HTTP POST) to the Observer instance job management APIs before sending data to the Resource and Relationship APIs.

Listen

A long-running listen job capable of consuming topology data over a long period of time.

A listen job is designed to support scenarios where the input data stream is unpredictable, or there is little or no consistency or versioning of resources within the data stream.

Note: These examples assume that the environment variables have been set in rest_observer_common.sh

start

$ASM_HOME/bin/rest_observer_listen_start.sh

stop

Default job

./bin/rest_observer_listen_stop.sh

Named job

env unique_id='My job name' $ASM_HOME/bin/rest_observer_listen_stop.sh

Bulk replace

A long-running job with the same resource replace semantics as the File Observer.

Bulk-replace jobs are designed to support scenarios where a known set of resources are subject to updates or versioning, and a prior observation about resources is to be replaced with a new one.

This job can provide a new set of resources and relationships and synchronize them to Agile Service Manager, thereby causing any previous data provided by the Observer to be deleted and replaced with the new data.

Note: These examples assume that the environment variables have been set in rest_observer_common.sh

start

Default job:

./bin/rest_observer_bulk_replace_start.sh

Job with bulk_replace_unique_id and provider given:

env bulk_replace_unique_id=manageDataCenter provider=MyJavaProgram $ASM_HOME/bin/rest_observer_bulk_replace_start.sh

synchronize

Default job

./bin/rest_observer_bulk_replace_synchronize.sh

Named job

env unique_id='My job name' $ASM_HOME/bin/rest_observer_bulk_replace_synchronize.sh

stop

Default job

$ASM_HOME/bin/rest_observer_bulk_replace_stop.sh

Named job

env unique_id='My job name' $ASM_HOME/bin/rest_observer_bulk_replace_stop.sh

Once a job request has been successfully submitted, you can start to provide data to the Resource and Relationship APIs on behalf of a given job instance.

The Resource and Relationship APIs may respond with an HTTPS 503 Service Unavailable response with a Retry-After: 10 seconds in the header. This indicates that even though the request against those APIs is valid, the observer has not been able to ascertain that meta-data about the job is held in Agile Service Manager yet; this may be due to, for example, any prevailing conditions in the network that support the Agile Service Manager micro-services.

Tip: If such a response is received, try the request again later.

Important: Ensure that the body is structured correctly. When posting the body, information included in the body after the closing } that matches an opening { is ignored, and no error is recorded.

Listen job process and examples

The following procedure (steps one to ten) includes examples that show how to use the REST Observer listen job to create and adjust a small topology.

Start the Listen job.

Use the following example as a model.

curl -u PROXY_USER[:PROXY_PASSWORD] -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' -d '{
  "unique_id": "my job",
  "type": "listen",
  "parameters": {
    "provider": "MyListenJob"
  }
}' 'https://localhost/1.0/rest-observer/jobs/listen'

Verify that the job is running.

curl -u PROXY_USER[:PROXY_PASSWORD] -X GET --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' 'https://localhost/1.0/rest-observer/jobs/my%20job'

Create a 'person' resource.

This example creates a person resource called 'Thomas Watson'.

curl -u PROXY_USER[:PROXY_PASSWORD] -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: my job' -d '{
  "name": "Thomas Watson",
  "uniqueId": "Thomas Watson",
  "entityTypes": [
    "person"
  ]
}' 'https://localhost/1.0/rest-observer/rest/resources'

Create an 'organization' resource.

This example creates an 'organization' resource of 'IBM'.

curl -u PROXY_USER[:PROXY_PASSWORD] -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: my job' -d '{
  "name": "IBM",
  "uniqueId": "IBM",
  "entityTypes": [
    "organization"
  ]
}' 'https://localhost/1.0/rest-observer/rest/resources'

Create a 'manages' relationship between Thomas Watson and IBM.

curl -u PROXY_USER[:PROXY_PASSWORD] -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: my job' -d '{
  "_fromUniqueId": "Thomas Watson",
  "_edgeType": "manages",
  "_toUniqueId": "IBM"
}' 'https://localhost/1.0/rest-observer/rest/references'

Create a new 'location' resource and relate it to Thomas Watson.

This example creates a 'location' resource of 'Campbell, New York' for Thomas Watson, and a location relationship (an edgeType of locatedAt).

curl -u PROXY_USER[:PROXY_PASSWORD] -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: my job' -d '{
  "name": "Campbell, New York",
  "uniqueId": "Campbell, New York",
  "entityTypes": [
    "location"
  ],
  "_references": [
    {
      "_fromUniqueId": "Thomas Watson",
      "_edgeType": "locatedAt"
    }
  ]
}' 'https://localhost/1.0/rest-observer/rest/resources'

Replace the location resource with one having latitude and longitude properties.

curl -u PROXY_USER[:PROXY_PASSWORD] -X PUT --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: my job' -d '{
  "name": "Campbell, New York",
  "uniqueId": "Campbell, New York",
  "entityTypes": [
    "location"
  ],
  "latitude": 42.232909,
  "longitude": -77.196918
}' 'https://localhost/1.0/rest-observer/rest/resources'

Delete all locatedAt relationships from Thomas Watson.

curl -u PROXY_USER[:PROXY_PASSWORD] -X DELETE --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: my job' 'https://localhost/1.0/rest-observer/rest/resources/Thomas%20Watson/references/both/locatedAt'

Delete the Campbell, New York location.

curl -u PROXY_USER[:PROXY_PASSWORD] -X DELETE --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: my job' 'https://localhost/1.0/rest-observer/rest/resources/Campbell%2C%20New%20York'

Delete the manages relationship between Thomas Watson and IBM.

curl -u PROXY_USER[:PROXY_PASSWORD] -X DELETE --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: my job' 'https://localhost/1.0/rest-observer/rest/resources/Thomas%20Watson/references/both/manages/IBM'

Bulk Replace job process and examples

The following procedure (steps 11 - 21) includes examples that show how to use the REST Observer bulk-replace job to create and adjust a small topology.

Note: These examples use a mixture of the provided scripts in $ASM_HOME/bin and the cURL command.

Submit a bulk-replace job request with the unique ID of 'my bulk replace'.

[root@asm-backend asm]# env bulk_replace_unique_id="my bulk replace" provider="Me" bin/rest_observer_bulk_replace_start.sh

Verify that the job is running.

curl -u PROXY_USER[:PROXY_PASSWORD] -X GET --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' 'https://localhost/1.0/rest-observer/jobs/my%20job'

Submit a location resource for the city of Coventry.

curl -u PROXY_USER[:PROXY_PASSWORD] -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: my bulk replace' -d '{
  "name": "Coventry",
  "uniqueId": "Coventry",
  "entityTypes": [
    "location"
  ]
}' 'https://localhost/1.0/rest-observer/rest/resources'

Submit a location resource for the town of Rugby

curl -u PROXY_USER[:PROXY_PASSWORD] -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: my bulk replace' -d '{
  "name": "Rugby",
  "uniqueId": "Rugby",
  "entityTypes": [
    "location"
  ]
}' 'https://localhost/1.0/rest-observer/rest/resources'

Submit a location resource for the Town of Leamington Spa with relationships to the existing resources of Coventry and Rugby.

curl -u PROXY_USER[:PROXY_PASSWORD] -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: my bulk replace' -d '{
  "name": "Leamington Spa",
  "uniqueId": "Leamington Spa",
  "entityTypes": [
    "location"
  ],
  "_references": [
     {
        "_toUniqueId": "Coventry",
        "_edgeType": "routesVia"
     },
     {
        "_toUniqueId": "Rugby",
        "_edgeType": "routesVia"
     }
  ]
}' 'https://localhost/1.0/rest-observer/rest/resources'

Check your progress by rendering the topology.

Having completed this set of observations, initiate a synchronize request for 'my bulk replace' job.

[root@asm-backend asm]# env bulk_replace_unique_id="my bulk replace" ./bin/rest_observer_bulk_replace_synchronize.sh

Provide a new topology for the 'my bulk replace' job.

This example submits a location resource for the town of Leamington Spa with relationships to the towns of Warwick and Stratford-Upon-Avon (resource placeholders).

curl -u PROXY_USER[:PROXY_PASSWORD] -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: my bulk replace' -d '{
  "name": "Leamington Spa",
  "uniqueId": "Leamington Spa",
  "entityTypes": [
    "location"
  ],
  "_references": [
     {
        "_toUniqueId": "Warwick",
        "_edgeType": "routesVia"
     },
     {
        "_toUniqueId": "Stratford-upon-Avon",
        "_edgeType": "routesVia"
     }
  ]
}' 'https://localhost/1.0/rest-observer/rest/resources'

Provide the resource data for the Town of Warwick resource placeholder, thus fully creating the resource.

curl -u PROXY_USER[:PROXY_PASSWORD] -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: my bulk replace' -d '{
  "name": "Warwick",
  "uniqueId": "Warwick",
  "entityTypes": [
    "location"
  ]
}' 'https://localhost/1.0/rest-observer/rest/resources'

Provide the resource data for the town of Stratford-upon-Avon resource placeholder, thus fully creating the resource.

curl -u PROXY_USER[:PROXY_PASSWORD] -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: my bulk replace' -d '{
  "name": "Stratford-upon-Avon",
  "uniqueId": "Stratford-upon-Avon",
  "entityTypes": [
    "location"
  ]
}' 'https://asm-backend.rtp.raleigh.ibm.com/1.0/rest-observer/rest/resources'

Initiate a synchronize request for the 'my bulk replace' job. This signifies to the Observer that it should instruct ASM to replace the previous set of observations with the new ones.
Note: The new data is available immediately as it is provided to ASM. The synchronize request simply deletes any resources previously observed that were not observed this time. In the current example, Coventry and Rugby were not observed, and therefore they are deleted.
```
[root@asm-backend asm]# env bulk_replace_unique_id="my bulk replace" ./bin/rest_observer_bulk_replace_synchronize.sh 
```

Remember: As an alternative to being configured using the Observer Configuration UI, observer jobs have scripts to start and stop all available jobs, to list the status of a current job, and to set its logging levels. These scripts can be run with -h or --help to display help information, and with -v or --verbose to print out the details of the actions performed by the script, including the full cURL command. For the on-prem version of Agile Service Manager, observer scripts are configured for specific jobs by editing the script configuration files.