Configuring the server to accept SSL connections

Configure the server to accept SSL connections before you enable SSL communication from the server to a client, a storage agent, Operations Center, or another server.

1. Specify the port on which the server waits for client communications that are enabled for SSL or accept the default port number. By default, the server is configured to accept Transport Layer Security (TLS) connections by specifying the TCPPORT or TCPADMINPORT options. To update TCPPORT or TCPADMINPORT or both options, update the dsmserv.opt file in the server instance directory. You can also configure the SSLTCPPORT and SSLTCPADMINPORT options as SSL-only connections.
2. Create the server key database by starting the server. The server key database file, cert.kdb, is stored in the server instance directory, and the default certificate label is automatically set as Tivoli Storage Manager Server SelfSigned SHA Key. The certificate is exported to the cert256.arm file.
3. If you are using the default self-signed certificate, the default self-signed certificate (cert256.arm) file is needed when you connect to the server by using TLS. After you use the cert256.arm file to import the self-signed certificate to the key database, the file is no longer needed.
4. If you are using a CA-signed certificate, each IBM Storage Protect server must send a unique server certificate to a CA to be signed. The CA returns a signed server certificate. You can use the same CA certificate to connect to multiple servers. You can also update the server certificates without needing to redistribute them to clients. To configure CA certificates, complete the following steps for each IBM Storage Protect server:
   Note: If you are using a CA-signed certificate and want to use multiple IP addresses on the same server, you must work with your certificate authority vendor to either configure your CA-signed certificate to use multiple IP addresses or to use a wildcard SSL certificate. The configuration steps vary depending on your CA vendor.
   1. Import the root CA certificate for each IBM Storage Protect server that enables SSL.
      Log on to the IBM Storage Protect server system with the instance user ID and issue the following example command from the instance directory:

      gsk8capicmd_64 -cert -add -db cert.kdb -stashed -label "CA cert" -file ca.crt

   2. Import one or more intermediate CA certificates by issuing the following example command for each intermediate certificate:

      gsk8capicmd_64 -cert -add -db cert.kdb -stashed -label "Intermediate CA cert" -file intca.crt

   3. The CA root and intermediate certificates (ca.crt and intca.crt) are used to verify the CA-signed server certificate. The CA root and intermediate certificates must be installed in the key database of all clients, storage agents, and servers that use TLS to communicate with the server.
   4. On the server, create a certificate request for the CA to sign by issuing a command that is similar to the following example:

      gsk8capicmd_64 -certreq -create -db cert.kdb -stashed -label "CA signed cert" 
      -sigalg sha256 -size 2048 -ku "digitalSignature,keyEncipherment,keyAgreement" 
      -eku "clientAuth,serverAuth" -dn "CN=tucson.example.com,OU=Spectrum Protect,O=IBM" 
      -san_dnsname tucson.example.com -san_ipaddr 9.11.0.0 -file cert_request.csr

   5. To receive the signed certificate and make it the default for communicating with clients, issue the following example command:

      gsk8capicmd_64 -cert -receive -db cert.kdb -stashed -file cert_signed.crt 
      -default_cert yes

      The CA-signed server certificate does not need to be distributed to clients.
      Note:

      • If you are using CA signed certificate, the SERVER_ADDRESS in <<install_dir>>/ui/LIberty/usr/servers/guiServer/serverConnection.properties should be having the complete hostname as the value. It should not be localhost.

      • If you have the value as localhost, change it to the server hostname, save the file, and restart Operations Center service.

      • Complete this step only on the hub server where the Operations Center is configured.
   6. Add the CA certificate to the IBM Storage Protect server key database. From the IBM Storage Protect server, as the instance user from the instance directory, issue the appropriate command.
      1. List all the certificates in cert.kdb using the following command:

         gsk8capicmd_64 -cert -list -db cert.kdb -stashed

         Example output:

         Certificates found
         * default, - personal, ! trusted, # secret key
         !    "SP Root CA Cert"
         !    "SP Intermediate CA Cert"
         !    vcloud688.storage.tucson.ibm.com:1500:0
         *-   “MyServerCertificate”

         where MyServerCertificate is the host certificate.

      2. Export the signed certificate to PKCS12 format to enable communication with the Operations Center.
         Note:

         • Complete this step only on the hub server where the Operations Center is configured.

         • If the files certp12.p12, certp12.pwd, and certp12.sth already exist in the instance directory, delete the files and continue with the next steps.

           For example, if the instance directory is /home/tsminst1, remove the files from that path by issuing the following command:

           rm -rf <<filename» e.g. : rm -rf certp12.p12

         gsk8capicmd_64 -cert -export -db cert.kdb -stashed -label "MyServerCertificate" -type pkcs12 -target certp12.p12 -target_pw "truststore_password"

         gsk8capicmd_64 -cert -rename -db certp12.p12 -pw truststore_password -label "MyServerCertificate" -new_label "TSMServerSelfSignedSHAKey"

         where truststore_password is the password provided for the PKCS12 certificate.

         Note: The new_label parameter should have the value TSMServerSelfSignedSHAKey. It should not be changed.

      To encode the password and store it on your local system for secure reuse in future operations, run the following commands.

      • Linux:

        echo "truststore_password" | base64 > certp12.pwd

      • AIX

        echo "truststore_password" | openssl base64 > certp12.pwd

      • Windows:

        echo|set /p=truststore_password > input.txt
        certutil -encode input.txt tmp.txt
        findstr /v CERTIFICATE tmp.txt > certp12.pwd
        del input.txt tmp.txt

        where truststore_password is the password of the PKCS12 truststore (certp12.p12).

5. Set File permissions for the newly created PKCS12 truststore by issuing the following command:

   chmod 777 certp12.p12 certp12.pwd

6. If you made any changes, restart the server.