Example: Configuring Native HA using the IBM MQ Operator

This example deploys a queue manager using the native high availability feature by using the IBM® MQ Operator. Mutual TLS is used for authentication, to map from a TLS certificate to an identity in the queue manager.

Before you begin

To complete this example, you must first have completed the following prerequisites:

Create a project/namespace for this example on your cluster.
On the command line, log into your cluster, then switch to the above namespace.
Ensure the IBM MQ Operator is installed and available in the above namespace.

About this task

This example provides a custom resource YAML defining a queue manager to be deployed. It also details the additional steps required to deploy the queue manager with TLS enabled.

Procedure

Create a pair of certificates as described in Creating a self-signed PKI using OpenSSL.
Create a config map containing MQSC commands and an INI file
Create a Kubernetes ConfigMap containing the MQSC commands to create a new queue and a SVRCONN channel, and to add a channel authentication record that allows access to the channel.
Ensure you are in the namespace you created earlier (see Before you begin), then enter the following YAML in the OCP web console, or using the command line.
```
apiVersion: v1
kind: ConfigMap
metadata:
  name: example-nativeha-configmap
data:
  example-tls.mqsc: |
    DEFINE CHANNEL('MTLS.SVRCONN') CHLTYPE(SVRCONN) SSLCAUTH(REQUIRED) SSLCIPH('ANY_TLS13_OR_HIGHER') REPLACE
    SET CHLAUTH('MTLS.SVRCONN') TYPE(SSLPEERMAP) SSLPEER('CN=*') USERSRC(NOACCESS) ACTION(REPLACE)
    SET CHLAUTH('MTLS.SVRCONN') TYPE(SSLPEERMAP) SSLPEER('CN=example-app1') USERSRC(MAP) MCAUSER('app1') ACTION(REPLACE)
    SET AUTHREC PRINCIPAL('app1') OBJTYPE(QMGR) AUTHADD(CONNECT,INQ)
    DEFINE QLOCAL('EXAMPLE.QUEUE') REPLACE 
    SET AUTHREC PROFILE('EXAMPLE.QUEUE') PRINCIPAL('app1') OBJTYPE(QUEUE) AUTHADD(BROWSE,PUT,GET,INQ)
  example-tls.ini: |
    Service:
        Name=AuthorizationService
        EntryPoints=14
        SecurityPolicy=UserExternal
```
The MQSC defines a channel called MTLS.SVRCONN and a queue called EXAMPLE.QUEUE. The channel is configured to allow access only to clients which present a certificate with a "common name" of example-app1. This is the common name used in one of the certificates created in Step 1. Connections on this channel with this common name are mapped to a user ID of app1, which is authorized to connect to the queue manager, and to access the example queue. The INI file enables a security policy which means that the app1 user ID does not need to exist in an external user registry — it exists only as a name in this configuration.
Deploy the queue manager
Create a new queue manager by using the following custom resource YAML. Ensure you are in the namespace that you created before you begin this task.
The availability type is set to NativeHA and persistent storage is selected. The default storage class configured in your Kubernetes cluster will be used. If you do not have a storage class configured as default, or you want to use a different storage class, add defaultClass: storage_class_name under spec.queueManager.storage.

The three pods in a Native HA queue manager replicate data over the network. This link is not encrypted by default, but this example uses the queue manager's certificate for encrypting traffic. You can specify a different certificate for additional security. The Native HA TLS Secret must be a Kubernetes TLS Secret, which has a particular structure (for example, the private key must be called tls.key).

Note that the example-qm-tls Secret was created in Step 1, and the example-nativeha-configmap ConfigMap was created in Step 2

Note:
The following YAML uses an IBM MQ Advanced license, though you can also use an IBM MQ license. For more information, see Configuring Native HA and Cross-Region Replication on IBM MQ licensed Queue Managers by using the IBM MQ Operator.

Important: If you accept the IBM MQ Advanced license agreement, change accept: false to accept: true. See mq.ibm.com/v1beta1: Current license versions for details on the license.
Example QueueManager for Red Hat® OpenShift®
Deploy your queue manager with the following configuration, using the command line or using the Red Hat OpenShift Container Platform web console.
```
apiVersion: mq.ibm.com/v1beta1
kind: QueueManager
metadata:
  name: exampleqm
spec:
  license:
    accept: false
    license: L-CXDB-XTZ9J8
    use: Production
  queueManager:
    name: EXAMPLEQM
    availability:
      type: NativeHA
      tls:
        secretName: example-qm-tls
    mqsc:
    - configMap:
        name: example-nativeha-configmap
        items:
        - example-tls.mqsc
    ini:
    - configMap:
        name: example-nativeha-configmap
        items:
        - example-tls.ini
    storage:
      queueManager:
        type: persistent-claim
  version: 10.0.0.0-r1
  pki:
    keys:
      - name: default
        secret:
          secretName: example-qm-tls
          items:
            - tls.key
            - tls.crt
            - ca.crt
```
Example QueueManager for Amazon EKS
Deploy your queue manager with the following configuration on an Amazon EKS cluster.
```
apiVersion: mq.ibm.com/v1beta1
kind: QueueManager
metadata:
  name: exampleqm
spec:
  license:
    accept: false
    license: L-CXDB-XTZ9J8
    use: Production
  queueManager:
    name: EXAMPLEQM
    availability:
      type: NativeHA
      tls:
        secretName: example-qm-tls
    mqsc:
    - configMap:
        name: example-nativeha-configmap
        items:
        - example-tls.mqsc
    ini:
    - configMap:
        name: example-nativeha-configmap
        items:
        - example-tls.ini
    storage:
      queueManager:
        type: persistent-claim
    route:
      enabled: false
    metrics:
      serviceMonitor:
        enabled: false
  version: 10.0.0.0-r1
  pki:
    keys:
      - name: default
        secret:
          secretName: example-qm-tls
          items:
            - tls.key
            - tls.crt
            - ca.crt
  web:
    enabled: false
    route:
      enabled: false
```
spec.queueManager.route.enabled, spec.queueManager.metrics.serviceMonitor.enabled, and spec.web.route.enabled are set to false because these are Red Hat OpenShift specific features that are enabled by default. These features must be disabled explicitly on Amazon EKS.
Confirm that the queue manager is running
The queue manager is now being deployed. Confirm it is in Running state before proceeding. For example:
- For deployments on the Red Hat OpenShift Container Platform:
```
oc get qmgr exampleqm
```
- For deployments on Amazon EKS:
```
kubectl get qmgr exampleqm
```
Test the connection to the queue manager
To confirm the queue manager is configured for mutual TLS communication, follow the steps in the appropriate link for your platform:
- Red Hat OpenShift: Testing a mutual TLS connection to a queue manager from your laptop for Red Hat OpenShift
- Amazon EKS: Testing a mutual TLS connection to a queue manager from a client application on your laptop for Amazon EKS
  Important: Amazon EKS users must complete the steps to Configuring a LoadBalancer to connect to a queue manager for Amazon EKS deployments before continuing on this page.
Force the active pod to fail
To validate the automatic recovery of the queue manager, simulate a pod failure:
1. View the active and standby pods
  Run the following command:
  - For deployments on the Red Hat OpenShift Container Platform:
```
oc get pods --selector app.kubernetes.io/instance=exampleqm
```
  - For deployments on Amazon EKS:
```
kubectl get pods --selector app.kubernetes.io/instance=exampleqm
```
  Note that, in the READY field, the active pod returns the value 1/1, whereas the standby pod returns the value 0/1.
2. Delete the active pod
  Run the following command, specifying the full name of the active pod:
  - For deployments on the Red Hat OpenShift Container Platform:
```
oc delete pod exampleqm-ibm-mq-<VALUE>
```
  - For deployments on Amazon EKS:
```
kubectl delete pod exampleqm-ibm-mq-<VALUE>
```
3. View the pod status again (see step 6.a).
4. View the queue manager status
  Run the following command, specifying the full name of the <ACTIVE_POD>. The active pod should be in ready state (this is the pod with the READY value of 1/1 shown by the pod status command in the previous step):
  - For deployments on the Red Hat OpenShift Container Platform:
```
oc exec -t <ACTIVE_POD> -- dspmq -o nativeha -x -m EXAMPLEQM
```
  - For deployments on Amazon EKS:
```
kubectl exec -t <ACTIVE_POD> -- dspmq -o nativeha -x -m EXAMPLEQM
```
  You should see the status shows that the active instance has changed, for example:
```
QMNAME(EXAMPLEQM) ROLE(Active) INSTANCE(inst1) INSYNC(Yes) QUORUM(3/3)
INSTANCE(inst1) ROLE(Active) REPLADDR(9.20.123.45) CONNACTV(Yes) INSYNC(Yes) BACKLOG(0) CONNINST(Yes) ALTDATE(2022-01-12)       ALTTIME(12.03.44)
INSTANCE(inst2) ROLE(Replica) REPLADDR(9.20.123.46) CONNACTV(Yes) INSYNC(Yes) BACKLOG(0) CONNINST(Yes) ALTDATE(2022-01-12)       ALTTIME(12.03.44)
INSTANCE(inst3) ROLE(Replica) REPLADDR(9.20.123.47) CONNACTV(Yes) INSYNC(Yes) BACKLOG(0) CONNINST(Yes) ALTDATE(2022-01-12)       ALTTIME(12.03.44)
```
5. Test the connection to the queue manager again
  Note: You do not need to repeat the steps to configure the connection, only those to put and receive messages to or from the queue
  To confirm the queue manager has recovered, follow the steps in the appropriate link for your platform:
  - Red Hat OpenShift: Testing a mutual TLS connection to a queue manager from your laptop for Red Hat OpenShift
  - Amazon EKS: Testing a mutual TLS connection to a queue manager from a client application on your laptop for Amazon EKS

Results

You deployed a queue manager with native high availability and mutual TLS authentication, and verified that it automatically recovers when the active Pod fails.