Configuring IBM Spectrum Protect client/server communication with Secure Sockets Layer

Secure Sockets Layer (SSL) allows industry standard SSL-based secure communications between the IBM Spectrum Protect client and server.

About this task

The following client components support SSL:

Command-line client
Administrative command-line client
Client GUI
Client API

Only outgoing client/server connections support SSL. A V8.1.2 client communicating with a down-level servers supports SSL. A V8.1.2 client communicating with a V8.1.2 server must use SSL. Incoming connections (for example, client acceptor, server-initiated schedule connections) do not support SSL. Client-to-client communications do support SSL. Web GUI does not support SSL. The Web GUI is no longer supported when communicating with a V8.1.2 server.

Each IBM Spectrum Protect server that is enabled for SSL must have a unique certificate. The certificate can be one of the following types:

A certificate that is self-signed by IBM Spectrum Protect.
A certificate that is issued by a certificate authority (CA). The CA can be from a company such as VeriSign or Thawte, or an internal CA, maintained within your company.

Follow these steps to enable SSL communication with a self-signed certificate:

Obtain the IBM Spectrum Protect server self-signed certificate (cert256.arm) Use the cert.arm certificate file when the server is not setup to use Transport Layer Security (TLS) 1.2; otherwise, use the cert256.arm file. The client certificate file must be the same as the certificate file that the server uses.
Configure the clients. To use SSL, each client must import the self-signed server certificate.
Use the dsmcert utility to import the certificate.
For a disaster recovery of the IBM Spectrum Protect server, if the certificate has been lost, a new one is automatically generated by the server. Each client must obtain and import the new certificate.

For fast path details for communication between a V8.1.2 client and a V8.1.2 server, you can use the SSLACCEPTCERTFROMSERV option to automatically accept a self-signed certificate. See Configuring by using the default security settings (fast path) for details.

Follow these steps to enable SSL communication with a CA-signed certificate:

Obtain the CA root certificate.
Configure the clients. To use SSL, each client must import the self-signed server certificate.
Use the dsmcert utility to import the certificate.

Tip: After you complete this step, if the server gets a new certificate that is signed by the same CA, the client does not need to import the root certificate again.
If you are recovering the backup-archive client as part of disaster recovery, you must install the SSL certificate on the server again. If the certificate was lost, you must get a new one. You do not need to reconfigure the client if the new certificate has been signed by a CA.

Windows operating systems The dsmcert utility is provided by the backup-archive client and automatically installs it in C:\Program Files\Tivoli\TSM\baclient.

Before you set up the server certificate on the client, follow these steps:

Open a command prompt and change the directory to the backup-archive client directory, for example: cd "C:\Program Files\Tivoli\TSM\baclient"
Append the GSKit binary path and library path to the PATH environment variable, for example:
```
set PATH=C:\Program Files\Common Files\Tivoli\TSM\api64\gsk8\bin\;
 C:\Program Files\Common Files\Tivoli\TSM\api64\gsk8\lib64;%PATH%
```
For more information, see Creating a symbolic link to access the latest GSKit library and IBM Global Security Kit return codes for details on GSkit libraries.

Next, you must import the server certificate, or the CA root certificate.

If you use a self-signed certificate

Each IBM Spectrum Protect server generates its own certificate. The certificate has a fixed file name of either cert.arm or cert256.arm. The certificate file is stored on the server workstation in the server instance directory, for example, /opt/tivoli/tsm/server/bin/cert256.arm. If the certificate file does not exist and you specify the SSLTCPPORT or SSLTCPADMINPORT server option, the certificate file is created when you restart the server with these options set. IBM Spectrum Protect V6.3 servers (and newer versions) generate files named cert256.arm and cert.arm. IBM Spectrum Protect servers older than V6.3 generate only certificate files named cert.arm. You must choose the certificate that is set as the default on the server.

Each IBM Spectrum Protect server generates its own certificate. The certificate has a fixed file name of either cert.arm or cert256.arm. The certificate file is stored on the server workstation in the server instance directory, for example, C:\Program Files\tivoli\tsm\server1\cert256.arm. If the certificate file does not exist and you specify the SSLTCPPORT or SSLTCPADMINPORT server option, the certificate file is created when you restart the server with these options set. IBM Spectrum Protect V6.3 servers (and newer versions) generate files named cert256.arm and cert.arm. IBM Spectrum Protect servers older than V6.3 generate only certificate files named cert.arm. You must choose the certificate that is set as the default on the server.

Follow these steps to set up the SSL connection to a server:

Obtain the certificate from the server administrator.
Import the certificate into the client key database by using the following command:
```
dsmcert -add -server <servername> -file <path_to_cert256.arm>
```

If you use a certificate from a certificate authority

If the certificate was issued by a certificate authority (CA) such as VeriSign or Thawte, the client is ready for SSL and you can skip the following steps.

For the list of preinstalled root certificates from external certificate authorities, see Certificate Authorities root certificates.

If the certificate was not issued by one of the well-known certificate authorities, follow these steps:

Obtain the root certificate of the signing CA.
Import the certificate into the client key database by using the following command:
```
dsmcert -add -server <servername> -file <path_to_cert256.arm>
```

Important:

A pseudo random password is used to encrypt the key database. The password is automatically stored encrypted in the stash file (dsmcert.sth). The stash file is used by the backup-archive client to retrieve the key database password.
More than one server certificate can be added to the client key database file so that the client can connect to different servers. Also, more than one CA root certificate can be added to the client key database.
If you do not run the preceding commands from the backup-archive client directory, you must copy dsmcert.kdb and dsmcert.sth into that directory.
By default, local key database files have root ownership and permissions and cannot be read by other users. If you plan to run the client as a non-root user, you must update the permissions. For example, to grant read access to all users and groups, run the following command:
```
# chmod go+r dsmcert.*
```
If you do not run the preceding commands from the backup-archive client directory, you must copy dsmcert.kdb and dsmcert.sth into that directory.
For performance reasons, use SSL only for sessions where it is needed. A V8.1.2 client communicating with a V8.1.2 server must use SSL. SSL No (the default value) indicates that encryption is not used when data is transferred between the client and a server earlier than V8.1.2. When the client connects to a V8.1.2 or later server, the default value No indicates that object data is not encrypted. All other information is encrypted, when the client communicates with the server. When the client connects to a V8.1.2 or later server, the value Yes indicates that SSL is used to encrypt all information, including object data, when the client communicates with the server. Consider adding more processor resources on the IBM Spectrum Protect server system to manage the increased requirements.
In order for a client to connect to a server that is using Transport Layer Security (TLS) Version 1.2, the certificate's signature algorithm must be SHA-1 or stronger. If you are using a self-signed certificate, you must use the cert256.arm certificate. Your IBM Spectrum Protect administrator might need to change the default certificate on the IBM Spectrum Protect server.

Additional details for a V8.1.2 client communicating with a server V8.1.1 and earlier V8 levels, and V7.1.7 and earlier levels.

After the server certificate is added to the client key database, add the SSL Yes option to the client options file, and update the value of the TCPPORT option. It is important to understand that the server is normally set up for SSL connections on a different port. In other words, two ports are opened on the server:

One port accepts regular non-SSL client connections
Another port accepts SSL connections only

You cannot connect to a non-SSL port with an SSL-enabled client, and vice versa.

If the value of tcpport is incorrect, the client cannot connect to the server. Specify the correct port number on the tcpport option.

To disable security protocols that are less secure than TLS 1.2, add the SSLDISABLELEGACYtls yes option to the client options file, or within the Java™ GUI select the Require TLS 1.2 checkbox on the Communication tab of the Preferences editor. Requiring TLS 1.2 helps prevent attacks by malicious programs.