Defining GitLab Observer jobs

Using the GitLab Observer, you can define a full load job that will dynamically load GitLab projects and their child resources, and visualize this data as a topology view in the Agile Service Manager UI.

Before you begin

Important: The GitLab Observer supports the API version 15.2.

Ensure you have the GitLab server details to hand, such as hostname, private token, datacenter and certificate.

The observer is installed as part of the core installation procedure.

Remember: Swagger documentation for the observer is available at the following default location: https://<your host>/1.0/gitlab-observer/swagger

About this task

The GitLab Observer job extracts GitLab resources information via REST API calls. The observer loads and updates the resources and their relationships within the Agile Service Manager core topology service.

gitlab_observer_common.sh
The configuration file you use to customize GitLab Observer settings.
The parameters defined here are then used by the gitlab_observer_load_start.sh script to trigger the GitLab Observer job.
Tip: Alternatively, you can set the appropriate environment variables. If an environment variable is set, it takes precedence over the configuration file settings.
You define and start the following jobs.
Load job
Run this job whenever you need GitLab topology data refreshed.
By default, these jobs are one-off, transient jobs that carry out a full upload of all requested topology data when they are triggered.
You can also run these jobs (again) manually from the Observer UI, or schedule them to run at set times when you configure them.
This job is started by the gitlab_observer_load_start.sh script
The GitLab Observer loads the following resources and their relationship into the Agile Service Manager core topology service:
Table 1. GitLab resources and relationships loaded by the GitLab Observer job
Vertex Description
Project Elements used to host codebase
IssuesManager Manager of issues – Dummy vertex
MergeRequestsManager Manager of Merge requests – Dummy vertex
PackagesBuilder Manager of packages – Dummy vertex
Pipeline Component that can trigger sequences and jobs for CI, delivery and deployment
ReleasesManager Manager of release – Dummy vertex
MergeRequests Use to check source code changes into a branch (PR)
Issues Use issues to collaborate on ideas, solve problems, and plan work
Jobs Smallest unit to run in GitLab CI/CD, can be a build or compilation task
Releases A checkpoint on the source code history
Packages Repository for a variety of common package managers

Procedure

To edit the parameters in the configuration file

  1. Open the gitlab_observer_common.sh configuration file and edit the following parameters:
    For encryption: Run the encrypt_password.sh script in the ASM_HOME/bin directory: 
    ./bin/encrypt_password.sh
    Enter and then confirm the password. The encryption utility returns an encrypted version.
    Table 2. Load job parameters
    Parameter Action Details
    Unique ID Enter a unique name for the job. Required
    GitLab Hostname Enter the GitLab server name. Required.
    Repository Visibility

    Choose a visibility for the repository.

    Options are 'private', 'public' or 'both'.

    		Required. Default is 'both'.
    Project Access

    Choose a project access type.

    Options are 'membership' or 'owned'.

    		 Required. Default is 'membership'.
    No Of Projects Enter the number of projects to be displayed in the topology. Required. Default is '50'. Must be a value greater than 0 (zero)
    Private Token Provide the private token for the GitLab account. Required. Must be encrypted.
    Data tenant The data tenant is used to track your data. It must be unique for each job or else it will override the previously discovered data. Required
    Target system certificate Specify a certificate by name to load into the trustStore. Required. To obtain authentication certificates using OpenSSL,see Configuring observer job security. Create and store the certificate in the $ASM_HOME/security directory.
    GitLab Port Enter the GitLab port number for the REST API. Optional. Default is 443.
    SSL Host Name Verification Choose whether SSL validation is true or false. When SSL validation is set to false, the observer will make HTTPS connection without validating the hostname of the provided certificate. Optional. Default is False.
    Connection timeout Specify the connection timeout in ms. Optional. Must be a value greater than 0 (zero), and the default is 5000 (5 seconds).
    Read timeout Specify the read timeout in ms. Optional. Must be a value greater than 0 (zero), and the default is 5000 (5 seconds).
    State for the issue Return all issues or just those that are opened or closed issues to be shown in topology. Optional. Default is 'opened'.
    Status of the pipeline Define a status. Options are:
    • created
    • waiting_for_resource
    • preparing
    • pending
    • running
    • success
    • failed
    • canceled
    • skipped
    • manual
    • scheduled
    		 Optional. Default is 'created'.

To start a job

  1. To start the GitLab job, use the following commands: 
    $ASM_HOME/bin/gitlab_observer_load_start.sh

Results

This job loads all requested data, and runs only once. Run this job whenever you need the topology data refreshed.

What to do next

You can also use the following scripts:
gitlab_observer_load_stop.sh
Stops the Load job
gitlab_observer_job_list.sh
Lists the status of current jobs
gitlab_observer_log_level.sh
Sets the log level
Remember: As an alternative to being configured using the Observer Configuration UI, observer jobs have scripts to start and stop all available jobs, to list the status of a current job, and to set its logging levels. These scripts can be run with -h or --help to display help information, and with -v or --verbose to print out the details of the actions performed by the script, including the full cURL command. For the on-prem version of Agile Service Manager, observer scripts are configured for specific jobs by editing the script configuration files.