HAproxy configuration

Learn how to configure the HAproxy for cross region failover.

The necessary components for this task:
  • Two Red Hat® OpenShift® clusters that are running on a hybrid architecture.
  • Two WebGUI deployments.
  • Two HAproxys.
The WebGUI and HAproxy deployments must be colocated with access to Netcool® Operations Insight® through the colocated HAproxy. The HAproxy directs traffic to the currently active Netcool Operations Insight application. The application authenticates the user through the WebGUI that is colocated with the HAproxy.
Example values:
  • https://webgui.east.example.com as the URL for WebGUI normally accessed by users in the east.
  • https://webgui.west.example.com as URL for WebGUI normally accessed by users in the west.
  • https://netcool.east.example.com as URL for HAproxy associated with the WebGUI normally accessed by users in the east.
  • https://netcool.west.example.com as URL for HAproxy associated with the WebGUI normally accessed by users in the west.
  • https://netcool.primary.example.com as URL for the Netcool Operations Insight application that is running in the primary Red Hat OpenShift cluster.
  • https://netcool.secondary.example.com as URL for the Netcool Operations Insight application that is running in the secondary Red Hat OpenShift cluster.
Note: netcool.primary.example.com and netcool.secondary.example.com must resolve to the IP addresses of the respective Red Hat OpenShift ingress routers. However, netcool.east.example.com and netcool.west.example.com resolve to the IP addresses of HAProxies configured to forward requests to the currently active Red Hat OpenShift.

WebGUI configuration

  1. Ensure that each WebGUI is set up to use a valid TLS certificate with the correct DNS name in the subject alternative name extension. Also, ensure that the web browser doesn't show any certificate errors.
  2. Obtain a copy of the root certificate authority (CA) certificate used to sign each of the WebGUI certificates. You can skip this step if all the certificates are signed by a well-known public CA.
  3. Update the jazz/profile/config/cells/JazzSMNode01Cell/oauth20/base.clients.xml by creating a <client> element for each client. You need a different client ID for each HAproxy, and for each Red Hat OpenShift cluster too if users can access Netcool Operations Insight directly on Red Hat OpenShift. For more information, see OAuth. The following table shows the properties of each element according to the example.
    Table 1. Example of properties for WebGUI configuration.
    ID Secret Redirect
    east secret1 https://netcool.east.example.com/
    west secret2 https://netcool.west.example.com/
    primary secret3 https://netcool.primary.example.com/
    secondary secret4 https://netcool.secondary.example.com/
  4. Stop and restart the server.

HAproxy configuration steps

To install HAproxy, run the yum install haproxy command. Install version 2.3 or later. A version of HAproxy is installed on the proxy host and the haproxy.cfg file is added to the /etc/haproxy directory. Repeat this step to install HAproxy on each proxy host.

The HAproxy configuration file is usually called /usr/local/etc/haproxy/haproxy.cfg and is usually in a directory similar to /usr/local/etc/haproxy. The configuration file shows a single backend to explain the changes to requests and responses that the HAproxy needs to make. For more information about setting up multiple backends with rules for monitoring and switching, see HAproxy external icon in the HAproxy documentation.

Configure the haproxy.cfg file and update it to perform dashboard switching for failover and failback of the IBM® Netcool Operations Insight on Red Hat OpenShift dashboard. Configure the haproxy.cfg file on each proxy host, as in the following example.
Note: Replace example.com domain names with the domain names for your environment.
global
    log stdout local0
    # The following is the location for valid certificates for HA Proxy endpoint and the value is given by the PROXY_HOST.
    ca-base <path_to_certs_dir>
    # For example: ca-base /usr/local/etc/ca-certs 

    # PROXY_HOST – will only work with port 443
    # This file configures the east HAProxy 
    presetenv PROXY_HOST "<primary_proxy_host>"
    # Use the following line on the west HAProxy 
    #presetenv PROXY_HOST "netcool.west.example.com"


    # This is the NOI OCP route endpoint for the PRIMARY Deployment
    setenv NETCOOL_OCP_HOST_PRIMARY "<primary_route_name>"

    # This is the NOI OCP route endpoint for the BACKUP Deployment
    setenv NETCOOL_OCP_HOST_BACKUP "<backup_route_name>"
    
defaults
    mode http
    log global
    timeout connect 5000ms
    timeout client 50000ms
    timeout server 50000ms
    option httplog

frontend http-in
# This is the server certificate for the proxy frontend hostname, PROXY_HOST. This is usually generated by the Corporation Signing Certificate authority (CA). proxy.pem is the file we created in step 7
    bind *:443 ssl crt <path_to_certs_dir>/proxy1.pem
    # For example: bind *:443 ssl crt /usr/local/etc/keys/proxy.pem
    use_backend http-out-primary  if { srv_is_up(http-out-primary/noi-primary) }
    use_backend http-out-backup  if { srv_is_up(http-out-backup/noi-backup) }

backend http-out-primary
    option httpchk
    http-check send meth GET uri /coordinator/operationalState hdr Host <primary_route_name> hdr Authorization  'Basic YXBpX3VzZXJuYW1lOmFwaV9wYXNzd29yZA=='
#    default-server inter 3s fall 3 rise 2
    # The ca-file refers to the signing certificate of the NETCOOL_OCP_HOST_PRIMARY route endpoint. 
    server noi-primary "${NETCOOL_OCP_HOST_PRIMARY}":443 ssl verify none check 
    http-request set-header X-NOI-HAProxy-Host %[req.hdr(Host)]
    http-request set-header x-noi-haproxy-host %[req.hdr(Host)]

    acl auth1 res.hdr(Location) -m beg https://<proxy_hostname>/users/api/authprovider/v1/authorize
    # If redirecting to NOI, change base to point to the proxy
    http-request set-header Host "${NETCOOL_OCP_HOST_PRIMARY}"

    http-response replace-value location ^([^:]*://)"${NETCOOL_OCP_HOST_PRIMARY}"(.*)$ \1"${PROXY_HOST}"\2
    # If redirect has a return URI within NOI, change that as well
    http-response replace-value location ^(.*redirect_uri=[^&]*)"${NETCOOL_OCP_HOST_PRIMARY}"(.*)$ \1"${PROXY_HOST}"\2 

backend http-out-backup
    option httpchk
    http-check send meth GET uri /coordinator/operationalState hdr Host <backup_route_name> hdr Authorization  'Basic <dr_coordinator_username:password>'
#    default-server inter 3s fall 3 rise 2
    server noi-backup "${NETCOOL_OCP_HOST_BACKUP}":443 ssl verify none check
#    server noi-backup "${NETCOOL_OCP_HOST_BACKUP}" addr "${NETCOOL_OCP_HOST_BACKUP}" port 443 ssl verify none check
    acl auth1 res.hdr(Location) -m beg https://<primary_proxy_host>/users/api/authprovider/v1/authorize

    http-request set-header X-NOI-HAProxy-Host %[req.hdr(Host)]
    http-request set-header Host "${NETCOOL_OCP_HOST_BACKUP}"

    # If redirecting to NOI, change base to point to the proxy
    http-response replace-value location ^([^:]*://)"${NETCOOL_OCP_HOST_BACKUP}"(.*)$ \1"${PROXY_HOST}"\2
    # If redirect has a return URI within NOI, change that as well
    http-response replace-value location ^(.*redirect_uri=[^&]*[^\.])"${NETCOOL_OCP_HOST_BACKUP}"(.*)$ \1"${PROXY_HOST}"\2
Where:
  • <path_to_certs_dir> is the path to the directory that contains the necessary certificates, generated on the proxy server host.
  • <primary_proxy_host> is the full DNS or the proxy host server, for example, proxy1.test.xyz.com. When you set up a HAproxy on your backup, ensure that the backup proxy host is referenced here.
  • <primary_route_name> full name of the Netcool route on the primary server, for example netcool-east.apps.east.cp.xyz.com.
  • <backup_route_name> full name of the Netcool route on the backup server, for example netcool-west.apps.west.cp.xyz.com.
Where:
  • <dr_coordinator_username:password> is the base64 encoded output from the following api username and password command:
    echo -n api_username:api_password |base64
  • http-request set-header X-NOI-HAProxy-Host
    This command is used by Netcool Operations Insight to identify the HAproxy that the request passed through. This command is used to identify the correct WebGUI.
  • http-request set-header Host
    This command is used to update the Host header so that the Red Hat OpenShift ingress router can direct the request to the correct pod.
  • http-response replace-value location
    This command updates any redirect responses so that the client continues to use the HAproxy and does not get redirected to the Red Hat OpenShift router.
Start the HAproxy instance. The example uses podman to run HAproxy version 2.3. It is also possible to run the HAproxy outside of podman.
podman run --name <unique_container_name> -p 443:443/tcp -v /root/new_certs:/root/new_certs -v /etc/haproxy:/usr/local/etc/haproxy:ro haproxy:2.3
Where /root/new_certs is the directory where you placed the certificates, for example /usr/local/etc/ca-certs, as specified by <path_to_certs_dir>.

Netcool Operations Insight hybrid configuration

  1. Use the following command, based on the example in Table 1, to create a <RELEASE>-was-oauth-cnea-secrets secret that contains the necessary client ID and secret values.
    kubectl create secret generic <RELEASE>-was-oauth-cnea-secrets \
        --from-literal=client-id=primary \
        --from-literal=client-secret=secret3 \
        --from-literal=netcool.east.example.com.id=east \
        --from-literal=netcool.east.example.com.secret=secret1 \
        --from-literal=netcool.west.example.com.id=west \
        --from-literal=netcool.west.example.com.secret=secret2

    Where <RELEASE> is the name of the Netcool Operations Insight hybrid deployment.

    Note: The client-id and client-secret values must contain the values for the local instance. When you run this command for the secondary instance of your Netcool Operations Insight hybrid deployment, the values change to secondary and secret4. The other values are named after the FQDN of the HAProxies. Do not include the port numbers even if they are not default. Each proxy has a .id and a .secret value.
    Note: You can either create the secret before the hybrid installation or patch the secret after the hybrid installation. For more information, see Install kubectl external icon and kubectl CLI external icon in the kubectl documentation.
  2. When you create your hybrid deployment, include the following parameters in the noihybrid YAML file. For more information, see Installing cloud native components on hybrid.
    • dash.url as the URL of the default WebGUI.
    • dash.username as the username for connecting to on-premises WebGUI.
    • dash.trustedCAConfigMapName as the name of a configuration map that contains the CA certificates that are used to sign all the WebGUI certificates. If all certificates are signed by well-known public CAs, this value can be omitted.
    • dash.crossRegionUrls as an array of the HAproxy and WebGUI pairs in the following format. Remember to include port numbers if they are not the default (443) port number.
    dash:
    
      ....
    
      crossRegionUrls:
      - proxy: https://netcool.east.example.com
        dash: https://webgui.east.example.com
      - proxy: https://netcool.west.example.com
        dash: https://webgui.west.example.com