Certificates in a Kubernetes environment
Use of cert-manager is recommended for managing certificates in a Kubernetes environment.
When deploying on Kubernetes, choose one of the following methods for creating internal certificates:
- Do not specify any internal certificates. In this case, the operator generates all of the
internal certificates without using a certificate manager. The CR definition to enable this option
is:
microServiceSecurity: custom
- Specify some, or all, of the internal certificates yourself without using a certificate manager.
In this case, any certificates that you do not create are created by the operator. You retain
control over the certificates that you create. The CR definition to enable this option
is:
microServiceSecurity: custom
- Configure cert-manager to generate the internal certificates. In this case, cert-manager creates
and manages the certificates. The CR definition to enable this option
is:
microServiceSecurity: certManager
To use cert-manager for generating the certificates, download and install cert-manager v1.12.13 from https://github.com/cert-manager/cert-manager/releases/tag/v1.12.13 and create an Issuer CR in the same namespace where you will install API Connect; for example:
apiVersion: cert-manager.io/v1 kind: Issuer metadata: name: selfsigning-issuer spec: selfSigned: {}
In the CR for the API Connect Management subsystem, specify a
certManagerIssuer
. ThecertManagerIssuer
can be configured to be a self-signing issuer, as in the following example:microServiceSecurity: certManager certManagerIssuer: name: selfsigning-issuer kind: Issuer
API Connect supports any other kind of Issuer supported by cert-manager, in addition to the self signed type.
In all the CRs, certs are expected at the level of any endpoint. For example, for the cloud admin endpoint of the Management subsystem you could have:
cloudManagerEndpoint:
annotations:
cert-manager.io/issuer: ingress-issuer
hosts:
- name: admin.example.com
secretName: cm-endpoint
The previous entry lets cert-manager automatically generate a cm-endpoint
TLS
secret using the ingress-issuer
. One way to define the
ingress-issuer
is as follows:
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: selfsigning-issuer
spec:
selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: ingress-ca
spec:
secretName: ingress-ca
commonName: "ingress-ca"
usages:
- digital signature
- key encipherment
- cert sign
isCA: true
duration: 87600h # 10 years
renewBefore: 720h # 30 days
privateKey:
rotationPolicy: Always
issuerRef:
name: selfsigning-issuer
kind: Issuer
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: ingress-issuer
spec:
ca:
secretName: ingress-ca
Another way to configure the cloud admin endpoint would be with:
cloudManagerEndpoint:
hosts:
- name: admin.example.com
secretName: my-cm-endpoint
Where my-cm-endpoint
is the name of a secret that you previously created, and is
present in the same namespace as the subsystem, and contains TLS certificate files for termination
of the endpoint.
Creating secrets and generating certificates
There is a number of "secrets" for which you can provide a name in the CRs:
- Portal admin secret.
portal: admin: secretName:
- Analytics ingestion secret.
analytics: ingestion: secretName: analytics-ingestion-client
- APIC Gateway Service TLS secret. Template entry from
gateway_cr_v6.yaml (in the file name, "v6" refers to the API Gateway service):
apicGatewayServiceTLS: secretName:
- APIC Gateway Peering TLS secret. Template entry from gateway_cr_v6.yaml (in the file name, "v6" refers to the API Gateway service):
apicGatewayPeeringTLS: secretName:
You can create manually create secrets that contain the tls.crt, tls.key, and tls.ca files for a certificate. You can then refer to them by name in their subsystem CR. However, a simpler way to have those secrets created is by using cert-manager. The following example shows how to generate those certificates with cert-manager:
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: selfsigning-issuer
spec:
selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: ingress-ca
spec:
secretName: ingress-ca
commonName: "ingress-ca"
usages:
- digital signature
- key encipherment
- cert sign
isCA: true
duration: 87600h # 10 years
renewBefore: 720h # 30 days
privateKey:
rotationPolicy: Always
issuerRef:
name: selfsigning-issuer
kind: Issuer
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: ingress-issuer
spec:
ca:
secretName: ingress-ca
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: portal-admin-client
spec:
commonName: portal-admin-client
secretName: portal-admin-client
issuerRef:
name: ingress-issuer
usages:
- "client auth"
- "signing"
- "key encipherment"
duration: 17520h # 2 years
renewBefore: 720h # 30 days
privateKey:
rotationPolicy: Always
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: analytics-ingestion-client
spec:
commonName: analytics-ingestion-client
secretName: analytics-ingestion-client
issuerRef:
name: ingress-issuer
usages:
- "client auth"
- "signing"
- "key encipherment"
duration: 17520h # 2 years
renewBefore: 720h # 30 days
privateKey:
rotationPolicy: Always
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: gateway-service
spec:
commonName: gateway-service
secretName: gateway-service
issuerRef:
name: ingress-issuer
usages:
- "client auth"
- "signing"
- "key encipherment"
duration: 17520h # 2 years
renewBefore: 720h # 30 days
privateKey:
rotationPolicy: Always
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: gateway-peering
spec:
commonName: gateway-peering
secretName: gateway-peering
issuerRef:
name: ingress-issuer
usages:
- "client auth"
- "signing"
- "key encipherment"
duration: 17520h # 2 years
renewBefore: 720h # 30 days
privateKey:
rotationPolicy: Always
So with the previous example, in the subsystem CRs (for Management and Gateway) the following values could be set:
-
portal: admin: secretName: portal-admin-client
-
analytics: ingestion: secretName: analytics-ingestion-client
-
apicGatewayServiceTLS: secretName: gateway-service
-
apicGatewayPeeringTLS: secretName: gateway-peering
Also, the Client certificate should be signed by same CA as the one for the endpoint certificates, and the server side should be configured with the expected Subject DN for the client. For example:
- The
portalAdminSecret
certificate and the certificate used to terminateportalAdminEndpoint
must have the same CA (in the example the CA isingress-ca
for both). Also, theadminClientSubjectDN
in the Portal subsystem CR should be set to the Subject DN ofportal.admin.secretName
. For example:adminClientSubjectDN: CN=portal-admin-client,O=cert-manager
- Similarly, the
analyticsIngestionSecret
and the Analytics ingestion endpoint certificate should share the same CA. Theingestion.clientSubjectDN
should be set to the Subject DN of theanalytics.ingestion.secretName
certificate.ingestion: endpoint: annotations: cert-manager.io/issuer: ingress-issuer hosts: - name: ai.example.com secretName: analytics-ai-endpoint clientSubjectDN: CN=analytics-ingestion-client,O=cert-manager
Creating TLS secret using openssl
Use of the cert-manager is recommended for managing certificates. However, you can choose not to use cert-manager, and instead create certificates manually. Requirements for manually created certificates:
- Extended Key Usage (EKU), either serverAuth or clientAuth depending upon the type of certificate. Certificates of type Server must have an Extended Key Usage with serverAuth purpose. Certificates of type Client must have an Extended Key Usage with clientAuth purpose.
- Subject Alternative Name (SAN) for the required hosts
- If a certificate is signed by an internal or custom CA, include the full chain in the end
certificate. If you omit the full chain, then a user who uses openSSL to access the endpoint will
see the following error:
error:num=20:unable to get local issuer certificate
The following example shows the full chain for an end certificate (signed by a custom CA):-------BEGIN CERTIFICATE ----- Cert contents for end-cert --------END CERTIFICATE ---------- -------BEGIN CERTIFICATE ----- Cert contents for Intermediate-CA --------END CERTIFICATE ---------- -------BEGIN CERTIFICATE ----- Cert contents for Root CA --------END CERTIFICATE ----------
The following steps can be used as an alternative to using cert-manager. If you choose to create your certificates this way, you lose some of the management features of cert-manager. For example, you must manually update your certificates when their expiration date gets near.
- Generate the custom certificate with the appropriate EKU and SAN. You will need to obtain the
private key, public certificate, and CA certificates in non-password-protected PEM format for the
custom certificate. Following is an example for how to generate a certificate (platform-api-example)
with an EKU serverAuth and SAN using openssl:
openssl x509 -req -days 360 -in platform-api-example.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -out platform-api-cert -sha256 -extfile <(cat /etc/ssl/openssl.cnf <(printf "\n[SAN]\nsubjectAltName=DNS:fqdn.myserver.com\nextendedKeyUsage=serverAuth")) -extensions SAN
whereDNS:fqdn.myserver.com
is the fully qualified domain name of the endpoint the certificate applies to.platform-api-example.csr
is the file name for the certificate signing request
Following is an example for how to generate a certificate (portal-client) with an EKU clientAuth and SAN using openssl:
openssl x509 -req -days 360 -in portal-client-example.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -out portal-client-cert -sha256 -extfile <(cat /etc/ssl/openssl.cnf <(printf "\n[SAN]\nkeyUsage=critical, digitalSignature,keyEncipherment\nextendedKeyUsage = clientAuth\nbasicConstraints=critical, CA:FALSE\nsubjectKeyIdentifier=hash\n")) -extensions SAN
- Create a Kubernetes secret containing tls.cert, tls.key, and ca.cert.
Setting an encryption-secret for the management database using openssl
You can optionally set the encryption-secret for the management database. If you do not set one, one is automatically generated for you.
- The encryption-secret is a secure random bytes password used for field level encryption in the
management database. You can generate 128 random bytes using the following command in
openssl:
openssl rand -out /path/to/secret/encryption-secret.bin 128
- Use kubectl to put the generated secret into
management-encryption-key
:kubectl create secret generic management-encryption-key -n <namespace> --from-file=/path/to/secret/encryption_secret.bin
where
<namespace>
is the namespace where the Management CR is going to be created) - Make sure that in management_cr.yaml, encryptionSecret.secretName points to the name of the secret created in the previous step.