Renewing IPA certificates

The IPA certificates expire every two years, starting with system provisioning date. The IPA certificates need to be renewed before expiry, otherwise they might cause user login failures. See Renewing IPA certificates before expiry to renew IPA certificates before expiry. Learn how to renew IPA certificates post expiry by using the automated utility. Alternatively, in case the automated utility fails, you can use the manual steps. For more information, see Renewing IPA certificates (manual steps).

Before you begin

The following procedure can be applied on 1.0.7.0 systems, other 1.0.7.x versions, and 1.0.8.x versions with expired certificates.

It is recommended to verify that the certificates did not expire before any upgrade from versions 1.0.7.0 and above.

The IPA certificate renewal takes approximately 30 minutes.

If any issues occur during the renewal of certificates, contact IBM Support.

To verify when the IPA certificates expire on your system, run the following command on e1n1:
getcert list | grep -i expi
root@e1n1:Node01~# getcert list | grep -i expi
        expires: 2022-02-02 00:03:58 UTC
        expires: 2022-02-02 00:03:33 UTC
        expires: 2022-02-02 00:03:32 UTC
        expires: 2022-02-02 00:03:33 UTC
        expires: 2040-02-13 00:03:32 UTC
        expires: 2022-02-02 00:03:32 UTC
        expires: 2022-02-13 00:04:35 UTC
        expires: 2022-02-13 00:05:19 UTC
        expires: 2022-02-13 00:05:30 UTC
root@e1n1:Node01~#

If the certificates expired, run the following steps on all control nodes in the system.

To get the list of all control nodes:
/opt/ibm/appliance/platform/xcat/scripts/xcat/display_nodes.py --control

Run the following procedure on control nodes only.

Procedure

  1. Use ssh to log in to the e1n1 control node as root user.
  2. Run:
    cd  /opt/ibm/appliance/platform/userauth/bin
  3. Download 1.0.0.0.apipacert-WS-ICPDS-fpxxx from Fix Central.
  4. Copy apipacert.py to the e1n1 control node location: /opt/ibm/appliance/platform/userauth/bin
  5. Check certificates expiry status by running:
    python   /opt/ibm/appliance/platform/userauth/bin/apipacert.py status
  6. Update expired certificates by running:
    python  /opt/ibm/appliance/platform/userauth/bin/apipacert.py update
  7. If required, verify the logs in the following location: /var/log/appliance/platform/userauth/apipacert.log.tracelog

Renewing IPA certificates before expiry

Procedure

  1. Collect current hostnames for the control nodes by using the hostname command.
    ssh e1n1 'hostname'
    ssh e1n2 'hostname'

    ssh e1n3 'hostname'
  2. Collect request IDs in e1n1, e1n2, and e1n3 by running the getcert list command.
    ssh control node 'getcert list'
    For example, run:
    ssh e1n1 'getcert list'
    The expected output:
    # ssh e1n1 'getcert list'
    Number of certificates and requests being tracked: 9.
    Request ID '20200706032214':
    	status: MONITORING
    	stuck: no
    	key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key'
    	certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem'
    	CA: dogtag-ipa-ca-renew-agent
    	issuer: CN=Certificate Authority,O=APDOMAIN.IBM.COM
    	subject: CN=IPA RA,O=APDOMAIN.IBM.COM
    	expires: 2024-04-09 18:32:28 UTC
    	key usage: digitalSignature,keyEncipherment,dataEncipherment
    	eku: id-kp-serverAuth,id-kp-clientAuth
    	pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
    	post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
    	track: yes
    	auto-renew: yes
    ...
    ...
    
  3. Set the hostnames to fbond hostname format on the control nodes:
    ssh e1n1 'hostnamectl  set-hostname e1n1.fbond' 
          ssh e1n2 'hostnamectl  set-hostname e1n2.fbond' 
          ssh e1n3 'hostnamectl  set-hostname e1n3.fbond'
  4. Submit the request for certificate renewal by using the command:
    getcert resubmit -i <Request ID>
    Example:
    getcert resubmit -i 20200706032214
  5. Set hostnames back to the initial hostnames collected in step 1.

Renewing IPA certificates (manual steps)

Procedure

  1. Use ssh to log in to the control nodes (e1n1, e1n2, e1n3) as root user and add NSSEnforceValidCerts off to /etc/httpd/conf.d/nss.conf on all control nodes:
    grep -ri "NSSEnforceValidCerts off" /etc/httpd/conf.d/nss.conf
    echo "NSSEnforceValidCerts off" >> /etc/httpd/conf.d/nss.conf
    grep -ri "NSSEnforceValidCerts off" /etc/httpd/conf.d/nss.conf
  2. Collect current hostnames for the control nodes by using the hostname command:
     ssh e1n1 'hostname'
    ssh e1n2 'hostname'

    ssh e1n3 'hostname' 
    (to be used later in step 6)
  3. Set the hostnames to fbond hostname format on the control nodes:
      ssh e1n1 'hostnamectl  set-hostname e1n1.fbond' 
          ssh e1n2 'hostnamectl  set-hostname e1n2.fbond' 
          ssh e1n3 'hostnamectl  set-hostname e1n3.fbond'
  4. On e1n1, update certificates with the following commands:
      systemctl restart apipactl
      systemctl start httpd
      wait 120 sec   
      ipa-cert-fix   (give yes if prompted )
      systemctl restart apipactl
      kinit admin   (password is passw0rd)
      ipa-certupdate 
      getcert refresh -a 
      getcert refresh-ca -a 
      getcert list | grep -i status
          ---->wait until  all showing "MONITORING" if any under process 
      getcert list | grep -i expi  
         ---->this give latest dates
      systemctl restart apipactl
    
  5. Update the certificates on the remaining nodes (e1n2 and e1n3) by running the following command on these nodes:
     getcert list | grep -i status
            --->wait until  all showing "MONITORING" if any under process  approx  wait time 10 mins 
      getcert refresh -a 
      getcert refresh-ca -a 
      getcert list | grep -i expi
             --->this give latest dates 
      kinit admin   (password is passw0rd)
      ipa-certupdate  (ignore if failed )
      systemctl restart apipactl
    
  6. Set hostnames back to the initial hostnames collected in step 2 for the control nodes: e1n1/e1n2/e1n3. Use the correct hostname if custom FQDN was set for control nodes and replace <hostname> in the following commands:
    ssh e1n1 'hostnamectl set-hostname  <hostname>'
    ssh e1n2 'hostnamectl set-hostname  <hostname>'
    ssh e1n3 'hostnamectl set-hostname  <hostname>'
    Example:
    ssh e1n1 'hostnamectl set-hostname  e1n1'
    ssh e1n2 'hostnamectl set-hostname  e1n2'
    ssh e1n3 'hostnamectl set-hostname  e1n3'
Troubleshooting steps 4 and 5 for any errors with getcert list -i | grep status command
If certificate renewal failed and getcert list -i | grep status command shows status different than Monitoring (for example, NEED_GUIDANCE), submit a manual request by collecting Request ID from getcert list command:
getcert resubmit -i <Request ID>
Example:
getcert resubmit -i 20200213000358
If the above command returns an error, check the status of certmonger and start, then resubmit request:
service certmonger status
service certmonger start