Configuring Kubernetes Observer jobs

Using this observer, you can configure jobs that discover the structure of your Kubernetes clusters, including pods, worker nodes and containers.

Before you begin

Important: The Kubernetes Observer supports Kubernetes version 1.14.

For Kubernetes load jobs, ensure you have the Kubernetes service details to hand, such as the Kubernetes host IP and SSL Certificate details. For Weave Scope listen jobs, first install Weave Scope, and then configure a job using the Weave Scope IP and port parameters.

Existing Load job functionality has been divided into two separate jobs, Load and Local. Local observations now run on the jobs/local endpoint. Existing scripts that use jobs/local to trigger a local observation of the Kubernetes environment will need to change the endpoint to jobs/local. With the introduction of new local job, the system-health job has been automatically migrated to use the same endpoint by the Kubernetes cron job. Any existing local jobs using the load API that are not automatically migrated, can be seen in the UI, deleted, and created using the correct job type.

The Kubernetes Observer is installed as part of the core installation procedure.

About this task

The observer reads topology data from Kubernetes through its REST APIs, or Weave Scope.

You can run the following jobs:
Load
A transient (one-off) job that loads all requested topology data from a Kubernetes environment.
By default, Load jobs are one-off, transient jobs that perform a full upload of all requested topology data as soon as they are triggered.
You can also run these jobs (again) manually from the Observer UI, or schedule them to run at set times when configuring them.
Local
Performs a local observation of the Kubernetes REST API for available resources, and loads them in the topology service.
Not supported for on-premise installation.
Weave_scope Listen
A standalone job that listens to the Weave Scope agent and continues to stream topology and state data to Agile Service Manager.
The Weave Scope listen job provides visibility of your Kubernetes services, pods, containers, deployments, stateful sets, Cron Jobs and processes for a specified namespace.
A long-running job that monitors its source for updates and runs until it is explicitly stopped, or until the Observer is stopped
You must install Weave Scope and then use the Weave Scope Master IP and Node port parameters. For more information on Weave Scope, see the following location: https://www.weave.works/docs/scope/latest/introducing/
For OCP
  1. Create Namespace 'weave' with 'ibm-privileged-psp'. 
    kubectl create namespace weave 
kubectl -n weave create rolebinding weave-clusterrole-rolebinding --clusterrole=ibm-privileged-clusterrole --group=system:serviceaccounts:weave
  2. Install Weave Scope using the following command:
    kubectl apply -f "https://cloud.weave.works/k8s/scope.yaml?k8s-service-type=NodePort&k8s-version=$(kubectl version | base64 | tr -d '\n')"
    This will result in a port being opened that the Observer can use.
  3. You can discover the NodePort using the following command:
    kubectl -n weave describe service weave-scope-app
  4. Launch the Weave Scope UI using the following URL: 
    https://<master ip>:<NodePort>
Table 1. Kubernetes Observer load job parameters
Parameter Action Details
Unique ID Enter a unique name for the job Required
Encrypted Kubernetes token The service account token for kubernetes. Required. Must be encrypted.
Kubernetes Master IP address Enter the Kubernetes Master IP address. Required
Kubernetes API port Enter the Kubernetes API port number. Required
Trust all certificate by bypassing certificate verification Enter true if you want to connect to Kubernetes without certificate Required
Exact HTTPS certificate file name Enter the exact name of the SSL/HTTPS certificate. Optional. If 'Trust all certificate' is set to false, then this parameter is Required.
data_center Specify the name of the data center in which the Kubernetes instance is running. Required
API query timeout (ms) Specify the Kubernetes REST API query timeout. Optional. The default is 5000 ms (that is, 5 seconds)
Correlate If 'true', enables the Event Analytics correlation on the namespace groups. Optional
Namespace Specify the Kubernetes namespace. Optional. If left empty, all namespaces are observed.
Terminated pods Choose whether terminated pods should be hidden (true or false). Optional. The default is false.
Job schedule Specify when the job runs. Optional. Load jobs only.
Observer job description Enter additional information to describe the job. Optional
Encryption requirement: The Load job requires the token to be encrypted. You encrypt the Kubernetes token using the encrypt_password.sh script in the $ASM_HOME/bin directory: 
./bin/encrypt_password.sh
Enter and then confirm the password. The encryption utility will return an encrypted password, which you enter in the Encrypted Kubernetes token field when configuring the Load job.
SSL certificate requirement: The Load job requires an SSL Certificate, and for it to be in a specific location:
  1. Get the kubernetes master IP and its API port using:
    kubectl cluster-info
  2. Run the following OpenSSL command:
    echo -n | openssl s_client -connect {master ip}:{api} | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > ./certificate_file_name.crt

    The certificate is saved as certificate_file_name.crt

  3. Copy the certificate file to the $ASM_HOME/security directory.
  4. When configuring the Load job, enter the certificate file name in the Exact HTTPS certificate file name field.
Table 2. Kubernetes Observer weave_scope job parameters
Parameter Action Details
Unique ID Enter a unique name for the job Required
Host Enter the Weave Scope host name (or IP address) of the web socket to be observed. Required
Port Enter the Weave Scope port number of the web socket to be observed. Required
Cluster Name Enter the name of the cluster or data center to be observed. Required
Namespaces Enter a list of namespaces to be observed. Optional. If left empty, all namespaces will be observed.
Resource types Select the Weave Scope resource types to observe. Optional.
Resources to exclude List resources to be excluded by ID, label, rank or namespace. Optional. Containers named 'pod' are excluded by default.
Observer job description Enter additional information to describe the job. Optional

Procedure

  1. From the Observer Configuration UI, click Configure under the Kubernetes icon, or select an existing Kubernetes job to be edited.
  2. Choose either load, local or weave_scope from the job type drop-down.

Configure the load or local jobs

  1. Enter or edit the following required parameters:
    • Unique ID
    • Encrypted Kubernetes token
    • Kubernetes Master IP address
    • Kubernetes API port
    • Exact HTTPS certificate file name
    • data_center
  2. Enter or edit the following optional parameters:
    • Namespaces
    • API query timeout (ms)
    • Terminated pods
  3. Optional: Define a Job schedule (for Load jobs only) by setting the time when the job should run, and whether it should run at regular intervals. By default, the job runs immediately, and only once. Optionally, you can specify a future date and time for the job to run, and then set it to run at regular intervals after that, if required. The run intervals must be at least 90 seconds apart, and if you set them at less than 15 minutes, a warning is displayed, as the frequency can impact system performance.
  4. Optional: Enter an Observer job description to explain the purpose of the job in more detail.

Configure the weave_scope job

  1. Enter the following required parameters:
    • Unique ID
    • Host
    • Port
      Tip: The NodePort can be obtained using the following command: 
      kubectl -n weave describe service weave-scope-app
    • Cluster Name
    Note: The host and port parameter fields must be empty.
  2. Enter the following optional parameters:
    • Namespaces
      Tip: Run the following command in the Kubernetes environment to get a list of namespaces: 
      kubectl get namespaces
    • Resource types
    • Resources to exclude
    • Observer job description
  3. Click Run job to save your job and begin retrieving information.