Synchronizing the ingress-ca certificates between sites

The X.509 CA certificate that is used for TLS communication between the API Connect deployments in each data center is referred to as the ingress-ca X.509 certificate. This certificate is stored in the ingress-ca secret in both data centers, and it must be the same in each data center for 2DCDR to function.

The cert-manager has a separate object that is referred to as the ingress-ca Kubernetes cert-manager certificate object. Cloud Pak for Integration and OpenShift top-level CR deployments has the ingress-ca Kubernetes cert-manager certificate object in both data centers. Kubernetes and OpenShift individual subsystem installations, has the ingress-ca Kubernetes cert-manager certificate object in only one of the data centers, the data center that was the original active data center when 2DCDR was first configured.

Ingress-ca on Kubernetes and OpenShift individual subsystem installations

In Kubernetes and OpenShift individual subsystem CR installations, the ingress-ca Kubernetes cert-manager certificate object exists on only one of the data centers in a 2DCDR deployment. The ingress-ca secret object exists in both data centers and must contain the same certificate string:
kubectl -n <namespace> get secret ingress-ca -o yaml

apiVersion: v1
data:
  ca.crt: <long cert string>
  tls.crt: <long cert string>
  tls.key: <long cert string>
kind: Secret
...
To identify the data center that has the ingress-ca Kubernetes cert-manager certificate object, run:
kubectl -n <namespace> get cert ingress-ca

   NAME         READY   SECRET       AGE
   ingress-ca   True    ingress-ca   30h
Note: If the data center that has the ingress-ca Kubernetes cert-manager certificate object is in a failed state and is unrecoverable, a new ingress-ca Kubernetes cert-manager certificate object can be created on the active data center by using the ingress-ca certificate section of the installation YAML file: helper_files/ingress-issuer-v1.yaml:
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: ingress-ca
  labels: {
    app.kubernetes.io/instance: "management",
    app.kubernetes.io/managed-by: "ibm-apiconnect",
    app.kubernetes.io/name: "ingress-ca"
  }
spec:
  secretName: ingress-ca
  commonName: "ingress-ca"
  usages:
  - digital signature
  - key encipherment
  - cert sign
  isCA: true
  duration: 87600h # 10 years
  renewBefore: 720h # 30 days
  privateKey:
    rotationPolicy: Always
  issuerRef:
    name: selfsigning-issuer
    kind: Issuer

Synchronizing the ingress-ca X.509 certificate across data centers

Follow these steps to extract your ingress-ca X.509 certificate from your source data center and prepare it for application on your target data center:
  1. Extract the ingress-ca secret from your source data center to a file called new-ca-issuer-secret.yaml:
    kubectl get secret ingress-ca -o yaml -n <namespace>  > new-ca-issuer-secret.yaml
  2. Edit the new-ca-issuer-secret.yaml file and remove the creationTimestamp, resourceVersion, uid, namespace, and managedFields. Remove the labels and annotations sections completely. The resulting contents should include the ingress-ca X.509 certificate, and the secret name:
    apiVersion: v1
data:
  ca.crt: <long cert string>
  tls.crt: <long cert string>
  tls.key: <long cert string>
kind: Secret
metadata:
  name: ingress-ca
type: kubernetes.io/tls
  3. Copy the new-ca-issuer-secret.yaml to the target data center.
Follow these steps to apply the extracted ingress-ca X.509 certificate on your target data center:
  1. To apply the new-ca-issuer-secret.yaml file, run:
    kubectl apply -f new-ca-issuer-secret.yaml -n <namespace>
  2. Regenerate all ingress-ca end-entity certificates:
    kubectl get secrets -n <namespace> -o custom-columns='NAME:.metadata.name,ISSUER:.metadata.annotations.cert-manager\.io/issuer-name' --no-headers=true | grep ingress-issuer | awk '{ print $1 }' | xargs kubectl delete secret -n <namespace>
    All affected pods should automatically restart. For more information about regenerating certificates, see: Renewing cert-manager controlled certificates.