Integration with OpenShift Container Platform

Overview

This guide walks you through adding a Red Hat OpenShift Container Platform cluster to a Infrastructure Automation container provider catalog. This deployment focuses on enabling the OpenShift Container Platform cluster metrics plug-in, so that Infrastructure Automation can collect information from an OpenShift Container Platform cluster upon integration.

Each procedure in this guide is covered in greater detail in the Infrastructure Automation and OpenShift Container Platform product documentation. However, links to the corresponding sections are provided for more detail.

The following sections will describe the required configuration for both products prior to integration.

Prerequisites

This guide assumes that you have:

When enabling metrics on OpenShift Container Platform, you can store your metrics data on persistent or non-persistent storage. To use persistent storage, you need to provision a persistent volume specifically for this purpose before configuring the metrics components. See Persistent Volumes in the OpenShift Container Platform Architecture documentation for more information.

Retrieving the OpenShift Container Platform Management Token

After enabling cluster metrics on your OpenShift Container Platform deployment, retrieve the management token while you are still logged in to the OpenShift Container Platform host. This will be required later in Adding OpenShift Container Platform as a Container Provider.

Run the following to obtain the token needed to add a Red Hat OpenShift provider:

  1. Obtain the management service account token name:

    # oc describe sa -n $project_name $service_account_name
...
Tokens:  management-admin-token-0f3fh
         management-admin-token-q7a87

  2. Select and describe one of the tokens to retrieve the full token output, replacing management-admin-token-0f3fh with the name of your token:

    # oc describe secret -n $project_name management-admin-token-0f3fh
...
Data
====
token:  eyJhbGciOiJSUzI1NiI...

Configuring Infrastructure Automation

Configuring Infrastructure Automation involves two steps:

  1. Configuring Infrastructure Automation Capacity and Utilization, and

  2. Enabling SmartState Analysis

These steps are required to allow Infrastructure Automation to collect metrics from OpenShift Container Platform Enabling OpenShift Container Platform Metrics and use them to perform a SmartState analysis. You can choose different servers to perform either function; the following sections assume that you will.

Configuring Infrastructure Automation Capacity and Utilization

For metrics collection to work properly, you also need to configure Infrastructure Automation to allow for all three Capacity & Utilization server roles, which are available from the settings menu under Configuration > Server > Server Control.

To enable these server roles:

  1. Click Configuration, then select the server to configure from menu: Settings > Zone in the accordion menu on the left.

  2. Browse to the Server Roles list in the menu: Server > Server Control section. From there, set the required capacity and utilization roles to ON, namely:

    1. Capacity & Utilization Coordinator

    2. Capacity & Utilization Data Collector

    3. Capacity & Utilization Data Processor

  3. Click Save.

Data collection is enabled immediately. However, the first collection begins 5 minutes after the server is started, and every 10 minutes after that. Therefore, the longest the collection takes after enabling the Capacity & Utilization Collector role is 10 minutes. The first collection from a particular provider may take a few minutes since Infrastructure Automation is gathering data points going one month back in time.

For more information, see Capacity and Utilization Collection in the Deployment Planning Guide.

Enabling SmartState Analysis

After enabling the required server roles, enable SmartState analysis. See Smart State Analysis Support from the Support Matrix.

Enabling SmartState analysis is similar to Assigning the Capacity and Utilization Server Roles, in that the procedure also involves enabling server roles on a specific server. To do so:

  1. Click Configuration.

  2. Select the server to configure from menu: Settings[Zone] in the left pane of the appliance.

  3. Browse to the Server Roles list in the menu: Server > Server Control section. From there, set the appropriate SmartState roles to ON. Namely:

    1. SmartProxy

    2. SmartState Analysis

  4. Click Save.

Adding OpenShift Container Platform as a Container Provider

At this point, you should now be ready to add your OpenShift cluster to Infrastructure Automation as a container provider. To do so, prepare the token you retrieved in Retrieving the OpenShift Container Platform Management Token and follow the procedure below:

  1. Browse to menu: Compute > Containers > Providers.

  2. Click Configuration, then click Add a New Containers Provider (Add a New Containers Provider).

  3. Enter a Name for the provider.

  4. From the Type list, select OpenShift Container Platform.

  5. Enter the appropriate Zone for the provider. If you do not specify a zone, it is set to default.

  6. In the Default tab, configure the following for the OpenShift provider:

    1. Select a Security Protocol method to specify how to authenticate the provider:

      • SSL: Authenticate the provider securely using a trusted Certificate Authority. Select this option if the provider has a valid SSL certificate and it is signed by a trusted Certificate Authority. No further configuration is required for this option.

      • SSL trusting custom CA: Authenticate the provider with a self-signed certificate. For this option, copy your provider’s CA certificate to the Trusted CA Certificates box in PEM format.

        Note:

        You can obtain your OpenShift Container Platform provider’s CA certificate for all endpoints (default, metrics, alerts) from /etc/origin/master/ca.crt. Paste the output (a block of text starting with -----BEGIN CERTIFICATE-----) into the Trusted CA Certificates field.

      • SSL without validation: Authenticate the provider insecurely (not recommended).

    2. Enter the Hostname (or IPv4 or IPv6 address) of the provider.

      Important:

      The Hostname must use a unique fully qualified domain name.

    3. Enter the API Port of the provider. The default port is 6443.

    4. Enter a token for your provider in the Token box.

      Note:

      To obtain a token for your provider, run the oc get secret command on your provider; see Obtaining an OpenShift Container Platform Management Token.

      For example:

      # oc get secret –namespace management-infra management-admin-token-8ixxs –template=’’ | base64 –decode

    5. Click Validate to confirm that Infrastructure Automation can connect to the OpenShift Container Platform provider.

  7. In the Metrics endpoint tab optionally configure the metrics service details:

    1. Select a service type (e.g.: Prometheus) or leave disabled

    2. Select a Security Protocol method to specify how to authenticate the service:

      • SSL: Authenticate the provider securely using a trusted Certificate Authority. Select this option if the provider has a valid SSL certificate and it is signed by a trusted Certificate Authority. No further configuration is required for this option.

      • SSL trusting custom CA: Authenticate the provider with a self-signed certificate. For this option, copy your provider’s CA certificate to the Trusted CA Certificates box in PEM format.

      • SSL without validation: Authenticate the provider insecurely using SSL. (Not recommended)

    3. Enter the Hostname (or IPv4 or IPv6 address) of the metrics service.

      In order to find the hostname you can use the oc get route command.

      You can retrieve the Prometheus route by running:

      oc get route prometheus-k8s -n openshift-monitoring

      Example:

      oc get route prometheus-k8s -n openshift-monitoring
NAME             HOST/PORT                                            PATH             SERVICES  PORT      TERMINATION          WILDCARD
prometheus-k8s   prometheus-k8s-openshift-monitoring.ocp.example.com  prometheus-k8s   web       reencrypt/Redirect   None

      In this example you would use prometheus-k8s-openshift-monitoring.ocp.example.com as the metrics hostname.

    4. Enter the API Port of your metrics endpoint uses a non-standard port for access. The default port is 443.

    5. Click Validate to confirm that Infrastructure Automation can connect to the metrics endpoint.

  8. In the Virtualization endpoint tab optionally configure the virtualization service details:

    1. Select a service type (OpenShift Virtualization) or leave disabled

    2. Select a Security Protocol method to specify how to authenticate the service:

      • SSL: Authenticate the provider securely using a trusted Certificate Authority. Select this option if the provider has a valid SSL certificate and it is signed by a trusted Certificate Authority. No further configuration is required for this option.

      • SSL trusting custom CA: Authenticate the provider with a self-signed certificate. For this option, copy your provider’s CA certificate to the Trusted CA Certificates box in PEM format.

      • SSL without validation: Authenticate the provider insecurely using SSL. (Not recommended)

    3. Enter the Hostname (or IPv4 or IPv6 address) of the virtualization service.

    4. Enter the API Port of your virtualization service. The default port is 443.

    5. Enter a token for your provider in the Token box.

  9. Click the Advanced tab to add image inspector settings for scanning container images on your provider using OpenSCAP.

    Note:

    • These settings control downloading the image inspector container image from the registry and obtaining the Common Vulnerabilities and Exposures (CVE) information (for effective scanning) via a proxy.

    • CVE URL that Infrastructure Automation requires to be open for OpenSCAP scanning: https://www.redhat.com/security/data/metrics/ds/. This information is based on the source code of OpenSCAP.

    1. Enter the proxy information for the provider in either HTTP, HTTPS, or NO Proxy depending on your environment.

    2. Enter the Image-Inspector Repository information. For example, openshift3/image-inspector.

    3. Enter the Image-Inspector Registry information. For example, registry.access.redhat.com.

    4. Enter the Image-Inspector Tag value. A tag is a mark used to differentiate images in a repository, typically by the application version stored in the image.

    5. Enter https://www.redhat.com/security/data/metrics/ds/ in CVE location.

  10. Click Add.

Note:

You can also set global default image-inspector settings for all OpenShift providers in the advanced settings menu by editing the values under ems_kubernetes, instead of setting this for each provider.

For example:

:image_inspector_registry: registry.access.redhat.com
:image_inspector_repository: openshift3/image-inspector

Container Image Scanning

Configuring Image Scanning

Infrastructure Automation manages vulnerability scanning of container images. When an OpenShift provider is added, OpenShift images from the internal registry are discovered. To enable image scanning, perform the following configuration steps:

  1. Browse to menu: Compute > Containers > Providers.

  2. Select the checkboxes of the OpenShift providers on which to enable scanning.

  3. From the Policy pull-down menu, click Manage Policies.

  4. Select the OpenSCAP profile checkbox.

  5. Click Save.

This action will trigger a SmartState analysis, or scan, of all images referenced by the OpenShift provider. The initial scan may take several hours to complete, depending on the number and size of images. The scan occurs in the OpenShift provider, which Infrastructure Automation receives and records in the database. OpenShift limits the number of scanning pods; only three images can be scanned simultaneously.

Scheduling A Recurring Scan

Software vulnerability databases are updated frequently. To apply these updates, a rescan is required. To schedule a recurring scan of container images:

:schedule openscap scan

  1. Click Configuration.

  2. From Settings > Zones in the left pane of the appliance, select Schedules.

  3. From the drop-down menu, click Configuration > Add a new Schedule.

  4. Type an arbitrary Name.

  5. Type an arbitrary Description.

  6. Ensure the Active checkbox is selected.

  7. In Action, select Container Image Analysis.

  8. In Filter, select All Container Images for Containers Provider, OpenShift.

  9. In Run, set the schedule as desired.

  10. Set the Time Zone, Starting Date, and Starting Time.

  11. Click Add.

Working with Images

Viewing Results

Image scanning results are displayed in each image summary page.

  1. Select menu: Compute > Containers > Container Images.

  2. Click the desired image.

For an OpenSCAP HTML report, locate the Configuration section and select OpenSCAP HTML.

:container configuration

For compliance and scanning history information, locate the Compliance section and note the Status field or select Available from the History field.

:container scan history

Manual Scanning

SmartState analysis scanning may be initiated manually for images. From an image summary page, select Configuration > Perform SmartState Analysis. Refreshing the image page will reflect the latest scan results and compliance history.

Evaluating Compliance

If the image scan policy has been updated since the last scan, compliance conditions may be re-evaluated. From an image summary page, select Policy > Check Compliance of Last Known Configuration. Refreshing the image page will reflect the latest compliance history.

Generating a Report on Images

You can output the results of an OpenSCAP scan of images to a report for an overview of the security risk level of images. The **Images by Failed OpenSCAP Rule Results are included with Infrastructure Automation and shows whether the image passed or failed OpenSCAP policy criteria, and the security risk.

Note: You can also create a copy of this report and edit it to contain additional information, such as the project name where the image is used to produce more useful results. For more information, see Editing a Report and Reportable Fields in Infrastructure Automation in Monitoring, Alerts, and Reporting for instructions on customizing reports.

Steps to create a report to show image compliance:

  1. Browse to menu: Overview > Reports.

  2. Click the menu: Reports > All Reports accordion.

  3. Browse to menu: Configuration Management > Containers > Images by Failed OpenSCAP Rule Results to run a report that shows which images failed the OpenSCAP compliance.

  4. Click play arrow Queue.

  5. The report generation is placed in the queue and its status shows in the reports page.

    failedimagescan

  6. Click reload (Refresh this page) to update the status.

  7. Browse to the Saved Reports accordion, and click the report when it completes.

  8. Click the report download buttons for the type of export you want. The report is automatically named with the type of report and date.

    • Click textimage Download this report in text format to download as text.

    • Click textimage Download this report in CSV format to download as a comma-separated file.

    • Click 2134 Download this report in PDF format to download as PDF.

OpenSCAP Policy Profile

Infrastructure Automation is pre-configured with a default scanning policy profile. This includes conditions to scan and identify compliance, as well as annotate compliance failure. SmartState analysis is performed when new images are added to OpenShift.

Customizing the Scanning Policy Profile

The built-in OpenSCAP policy profile cannot be edited. However, you can assign edited copies of these policies to a new policy profile. This provides you the ability to create a customized version of the built-in OpenSCAP policy profile.

To do so, you must first copy the policy that you want to customize:

  1. Browse to menu: Control > Explorer.

  2. Click the Policies accordion, select Container Image Compliance Policies, and click OpenSCAP.

  3. Click imageConfiguration, and an option to copy the policy appears; for example, imageCopy this Container Image Policy.

  4. Click OK to confirm.

The new policy is created with a prefix of Copy of in its description, and it can be viewed in the Policies accordion.

image

You can now edit the copied policy. After editing copied policies, you can add them to a new policy profile. For instructions on how to edit policies, create a new policy profile, and add policies to it, see the Policies and Profiles guide. Once you have a customized policy profile, you can assign it to a containers provider.

Controlling OpenShift Pod Execution

Through the default policy profile, non-compliant images receive the control policy action Mark as Non-Compliant. This action annotates the image object (not to be confused with the imagestream object) with images.openshift.io/deny-execution=true. This annotation may be used to prevent nodes from running non-compliant images.

Reference

More information about OpenSCAP, see visit the OpenSCAP web site.