Managing Runtimes

Important: This content is a technical preview, and should not be relied on in a production environment.

• Overview
• Prerequisites
• Installing and configuring
• Understanding and viewing the data
• VM Enablement
• Advanced: Defining your own CR
• Limitations
• Troubleshooting

Overview

IBM Cloud Pak® for Multicloud Management Manage Runtimes provides a customizable and central management pane (bridges cross IBM Cloud Pak capabilities) for workloads that run across VMs and Clusters. By default, the Manage Runtimes UI is populated with all resource views available in the master branch of our public GitHub repo. The current behavior of the operator is to pull in updates from the repo every 12 hours.

For each specified runtime type (through custom resource instances), the following elements are provided through the IBM Cloud Pak console:

• High-Level visibility
  • View all runtimes that run across all infrastructures (VMs and Clusters)
  • View runtime infrastructure type, name, provider, region, and purpose
  • View runtime version and status
  • View related applications (where applicable, see limitations)
• Dynamic details and metric dropdowns (custom)
  • KPIs, if appropriate dynamic details are defined and workload is instrumented with data collector
  • Links to related applications
• Links and actions
  • Add workload to new application (for cluster-based workloads)
  • External Links or Consoles, where defined and applicable (for example, change Queue manager settings, add Queues, and other actions)
  • View and set thresholds (where applicable)
  • View Monitoring Dashboards (where applicable)
  • View and edit cluster compliance (all)

Prerequisites

1. Install Runtimes that you want to monitor.
2. If you want to show metrics in the dropdown portion of the Manage Runtimes table, do the following steps:
   1. After you install Monitoring as instructed in Installing the IBM Cloud Pak for Multicloud Management, you need to do extra configuration and installation steps to enable full monitoring capabilities. For more information, see Enabling monitoring capabilities.
   2. Enable and configure the data collector or agent for the runtime that you want to monitor. In the IBM MQ case, for cluster-based workloads, you must configure and enable the IBM MQ plug-in as outlined in Configuring IBM MQ monitoring in Unified Agent.

Installing and configuring the IBM Cloud Pak® for Multicloud Management Manage Runtimes

To learn how to install the IBM Cloud Pak® for Multicloud Management Manage Runtimes, see Installing the IBM Cloud Pak for Multicloud Management.

Note:

• Manage runtime operator is technology preview and by default, install configuration is false. Update the enabled section to true to enable the manage runtimes function.
• You must install the Manage Runtime operator with the default configuration. You cannot update any parameters.

Understanding and viewing the data

1. In the IBM Cloud Pak console, click Manage Applications > Manage Runtimes.
2. Click the Runtime that you want to view to open the details, for example, IBM MQ.
3. In the beginning of the Runtime dashboard, it lists the total number of resources and also the respective number of Container-based and VM-based resources.
4. In the table, you can see the Name, Infrastructure type, Infrastructure name, Provider, Purpose, Region, Namespace, Version, and Status for each resource. Note: Data that is shown in the table for all runtimes is defined in spec.search by the CR instance.
   • Version
     • Derived from metadata.labels.version for cluster-based resources
     • Not currently available for VM-based resources
   • Purpose
     • Derived from metadata.labels.environment (of cluster) for cluster-based resources
     • Derived from metadata.labels for label prefixed with environment.management.ibm.com/ (of VM CR) for VM-based resources, for example, the label environment.management.ibm.com/development=true would have Purpose set to development
   • Region
     • Derived from metadata.labels.region (of cluster) for cluster-based resources
     • Derived from spec.vminfo.availabilityzone (of VM CR) for VM-based resources
   • Status
     • For cluster, status is mapped for Kubernetes StatefulSet, deployment, or DaemonSet associated with runtime instance.
       • Status is indeterminate if status information is unavailable, or the Kubernetes pod controller is not one of the previous types.
       • Status is available if it meets the following parameters:
         • For StatefulSet: currentReplicas == desiredReplicas
         • For deployment: currentReplicas == availableReplicas
         • For DaemonSet: readyReplicas == desiredReplicas
       • Status is critical otherwise
       • For VMs, status is reported by agent (if it is being monitored) or indeterminate (See Limitations)
   • Dropdown items Click the name of the Resource in the table, you can see dropdown items for custom data. The custom data depends on CR instance spec.dynamicdetail. To get data, the data collector must be properly installed and configured.
   • Links Hover over list item in the table, if applicable to that runtime instance, you can select from the dropdown options by clicking the icon on the end of each row.
     • Add to Application: Add your workloads to new application
     • View Compliance: Opens the Governance and risk dashboard where you can view and manage the number of security risks and policy violations in your clusters and applications. For more information, see Governance and compliance.
     • Launch Console: Opens the console page where you can change Queue manager settings, add Queues, and other actions. Notes: It is only available for cluster-based workloads. And it is only applicable where dynamic search links are defined. For more information, see public GitHub repo.
     • Monitoring: Opens the Monitoring Resources dashboard where applicable. For more information, see Resource dashboard.
     • View thresholds: Open the Thresholds dashboard where you can create, edit, delete, enable, or disable thresholds. For more information, see Managing thresholds.

VM Enablement

To enable VM-based runtimes to show on the Manage Runtimes page, you currently must annotate VM custom resources on your hub to designate which type of runtimes and what instances are running on them. The general process is outlined in the following example scenario.

Example Scenario

Say that you have an example VM, ExampleVM. It is running two instances of IBM MQ and one instance of IBM App Connect. Get the yaml output of the VM from the hub by using kubectl get virtualmachine example-vm -n default -o yaml.

You see the following output:

apiVersion: infra.management.ibm.com/v1alpha1
kind: VirtualMachine
metadata:
  name: example-vm
  namespace: default
spec:
  vmid: "1234"
status:
  vminfo:
    vmName: ExampleVM
    availabilityZone: RTP
    provider: AWS
    hostnames:
    - example.ec2.internal

Navigate to Monitoring > Infrastructure Monitoring > MQ Queue Manager, you can see that you have two VM-based MQ Instances by the names mqInstance1 and mqInstance2.
Navigate to Monitoring > Infrastructure Monitoring > IBM ACE, you can see that you have one instance of VM-based ACE, aceInstance1. Ensure that the hostname that is noted in the Resource properties of each resource dashboard matches status.vminfo.hostnames of the VM instance.

Now, reference the runtime custom resource for your runtimes and note the spec.search.vm-based.label (if there is one). For IBM MQ, it is app.management.ibm.com/ibm-mq=true. For IBM ACE, it is app.management.ibm.com/ibm-ace=true.

Now, edit the VM CR, copy the status.vminfo content into spec.vminfo, and append the following information:

• metadata.labels:
  • Corresponding labels from your manage runtimes custom resource spec.search.X.label where X is cluster-based or vm-based.
• spec.vminfo.runtimeInstanceNames:
  • Formatted list of strings that designate resource type and instance
  • String format is runtimeLabelKey=runtimeLabelValue:runtimeInstanceName

The resulting virtual machine custom resource looks like the following format:

apiVersion: infra.management.ibm.com/v1alpha1
kind: VirtualMachine
metadata:
  name: example-vm
  namespace: default
  labels:
    app.management.ibm.com/ibm-mq: true  # NOTE
    app.management.ibm.com/ibm-ace: true # NOTE
spec:
  vmid: "1234"
  vminfo:
    vmName: ExampleVM
    availabilityZone: RTP
    provider: AWS
    hostnames:
    - example.ec2.internal
    runtimeInstanceNames: # NOTE
    - app.management.ibm.com/ibm-mq=true:mqInstance1
    - app.management.ibm.com/ibm-mq=true:mqInstance2
    - app.management.ibm.com/ibm-ace=true:aceInstance1
status:
  vminfo:
    vmName: ExampleVM
    availabilityZone: RTP
    provider: AWS
    hostnames:
    - example.ec2.internal

Advanced: defining and publishing your own CR

• For more information about how to create or publish your own custom resources for the Manage Runtimes console, see the public GitHub repo. Custom resources might be created on your hub cluster, or added to the public repo after review.

Limitations

• Related applications only shown for cluster-based workloads
• Add to application capability only available for cluster-based workloads
• Version information not available in overview for VM-based resources. If your runtime is instrumented with a data collector, you might be able to view it in the Monitoring dashboard or dropdown detail (if configured properly).
• Dynamic links only available for cluster-based workloads
• Status is only shown for VM-based resources if runtime instance is monitored by data collector.

Troubleshooting

Agent not installed

Your runtime is not instrumented or configured for monitoring. Go to Installing and configuring agents for installation, and validate that data is flowing.

Navigate to Monitor Health > Infrastructure Monitoring. Choose the resource that represents the resourceType in your Runtime CR, and identify that a dashboard is present and data is flowing on your specific instance. If you do not see data, see agent troubleshooting.

Improper User Account Access

Symptoms

• You cannot see any listed runtimes in overview menu but the CRs are installed (ie you see objects listed when running kubectl get runtimes -A from your hub)
• You can see runtimes in the overview and details pages, but no metric data is shown in the dropdowns (Manage Applications > Manage Runtimes > click on runtime type > click dropdown on resource instance), and you have verified that data is flowing (by navigating to resource monitoring dashboard directly through Monitoring > Infrastructure Monitoring > Resource Type of Interest > Resource Instance of Interest)

To view monitoring data from Manage Runtimes and navigate smoothly to all links, one of the following scenarios must be true:

• You are submitting agent data to the default tenant (id-mycluster-account)
• You are submitting agent data to a non-default tenant and your user is assigned the ClusterAdmin role

Note: To see which tenant agent data is being submitting to, visit Monitoring > Infrastructure Monitoring and note the value X in "/apmui/icam/X/resources" of the URL

Landing page is stuck in spinning

• Check health of search services in hub
• Try restarting search pods in hub

Disconnected network or airgap environment

Symptom

• No runtime custom resources are listed in UI or hub environment (kubectl get runtimes -A), but the manage runtime operator is installed and running

Cause

• You disconnect from your network or are in an airgap environment, so the manage runtime operator cannot automatically pull in the custom resources from the public GitHub repo

Resolution

• In your hub cluster, manually apply the CRs from public GitHub repo with the following content:

  kubectl/oc apply -f <CR template file> -n <namespace of operator>