Configuring TLS or HTTPS for ITM V6 Integration (public preview)

Edit online

Learn how to configure secure TLS or HTTPS communication between IBM Tivoli Monitoring V6 agents and the Instana host agent.

Overview

By default, ITM agents communicate with the Instana host agent over HTTP on port 42699 (the default port for ITM agent communication with Instana). For secure communication, you can configure TLS/HTTPS connections between ITM agents and the Instana host agent.

Setting up HTTPS requires configuring certificates on both sides:

Instana host agent side: Configure TLS certificates (.crt and .key files) to enable HTTPS on port 42699.
ITM agent side: Configure IBM GSK (Global Security Kit) keyring files that trust the Instana host agent's certificate.

The Instana host agent uses standard text-based certificate files (.crt and .key), while ITM agents require IBM GSK keyring database files (.kdb and .sth). This guide provides step-by-step instructions for creating and configuring both certificate formats.

Prerequisites

Before you start, make sure that you meet the following requirements:

Install and run the Instana host agent.
ITM V6 agents patched with 6.3.0.7-TIV-ITM_TEMA-IF0008 or later. This version is required for HTTPS support.
Install OpenSSL on the system where you generate certificates.
Make sure that IBM GSKit commands are available (typically included with ITM installations).
Download and extract the ITM V6 configuration pack.

Step 1: Configure TLS on the Instana host agent

First, enable TLS on the Instana host agent to accept HTTPS connections on port 42699.

Creating TLS certificates

You can either create a self-signed certificate or obtain a certificate from your Certificate Authority (CA).

Option A: Create a self-signed certificate

Use the following commands to create a self-signed certificate:

# Create a private key
openssl genrsa -out tls.key 2048

# Create a certificate signing request (CSR)
# Replace 'instana.example.com' with your Instana host agent's hostname
openssl req -new -key tls.key -out tls.csr -subj "/CN=instana.example.com"

# Create a self-signed certificate valid for 365 days
openssl x509 -req -days 365 -in tls.csr -signkey tls.key -out tls.crt

Important: The Common Name (CN) in the certificate must match the hostname that ITM agents will use to connect to the Instana host agent.

Note: Before certificate expiration, generate a new certificate and update both the Instana host agent and ITM agent keyring files to maintain secure communication.

Option B: Use a certificate from your CA

If you have an existing certificate from your Certificate Authority:

Ensure you have both the certificate file (.crt or .pem) and the private key file (.key). Both .crt and .pem files are text-based certificate formats and can be used interchangeably.
Verify the certificate's Common Name (CN) matches your Instana host agent's hostname:
```
openssl x509 -in your-cert.crt -noout -subject
```

Installing certificates on the Instana host agent

Copy the certificate files to the Instana host agent:
- <agent_installation>/etc/certs/tls.crt
- <agent_installation>/etc/certs/tls.key
After you add the certificates, restart the agent to initialize the network connections.
Verify that the agent is listening on HTTPS:
```
# This should show a TLS handshake
openssl s_client -connect localhost:42699 -showcerts
```
The output shows the certificate chain and a successful connection.

For more information about TLS configuration on the Instana host agent, see Setting up TLS encryption for agent endpoint.

Step 2: Create IBM GSK keyring files for ITM agents

ITM agents use IBM Global Security Kit (GSK) format for certificates, which requires creating a keyring database (.kdb) and stash file (.sth).

Locating IBM GSKit commands

IBM GSKit is typically included with ITM installations. The following examples use Linux paths. For other Unix-like systems (AIX), adjust the paths according to your ITM installation.

Common GSKit command locations:

Linux: /opt/IBM/ITM/<arch>/gs/bin/gsk8capicmd_64
Windows: C:\IBM\ITM\GSK8_x64\bin\gsk8capicmd_64.exe (for 32-bit systems, use GSK8 instead of GSK8_x64)

Before using GSKit commands on Linux, set the library path:

export LD_LIBRARY_PATH=/opt/IBM/ITM/<arch>/gs/lib64:$LD_LIBRARY_PATH

Note: For AIX systems, use LIBPATH instead of LD_LIBRARY_PATH.

Verify the GSKit command is available:

/opt/IBM/ITM/<arch>/gs/bin/gsk8capicmd_64 -help

Note: If you encounter a "cannot open shared object file" error, ensure the library path is set correctly for your ITM installation.

Creating the keyring files

Create a keyfiles directory in your configuration pack:

cd /path/to/instana-v6-configuration-pack
mkdir keyfiles
cd keyfiles

Set the path to the GSKit command (adjust based on your ITM installation):
```
GSK_CMD=/opt/IBM/ITM/<arch>/gs/bin/gsk8capicmd_64
```
Create a new keyring database with a stash file:
```
$GSK_CMD -keydb -create -db keyfile.kdb -pw password -type cms -stash
```
Important: Replace "password" with a strong password. The stash file encrypts this password, but choose a secure password following your organization's security policies.

Note: The -stash option creates the keyfile.sth stash file that stores the password in encrypted form. This allows ITM agents to access the keyring without requiring a password prompt.
Copy the Instana host agent's certificate (created in Step 1) to the current directory. You will need to transfer the tls.crt file from the Instana host agent system to the system where you're creating the keyring files:
```
# Example: Copy the certificate from the Instana host agent
# name the file instana-host-agent.crt
```
Import the Instana host agent's certificate into the keyring as a trusted CA certificate:
```
$GSK_CMD -cert -add -db keyfile.kdb -pw password -label "Instana_Host_Agent_Cert" -file instana-host-agent.crt -format ascii -trust enable
```
Note: If the import fails, verify the certificate file format is ASCII/PEM and the file path is correct.

Note: The -trust enable flag marks this certificate as trusted, allowing ITM agents to accept connections from the Instana host agent.
Verify the keyring contents:
```
$GSK_CMD -cert -list -db keyfile.kdb -pw password
```
The output lists the certificates in the keyring, including "Instana_Host_Agent_Cert".
(Optional) Remove the certificate file after import:
```
rm instana-host-agent.crt
```
Note: Once the certificate is imported into the keyring, the original certificate file is no longer needed and can be removed for security purposes.
Verify the keyfiles directory contains at least the following files:
- keyfile.kdb - The keyring database
- keyfile.sth - The stash file with encrypted password
- keyfile.rdb - Request database (created automatically)
Note: You may also see a keyfile.crl file (Certificate Revocation List), which is created automatically by GSKit but is not required for basic TLS functionality.

Step 3: Configure ITM agents to use HTTPS

Now that you have created the keyring files, configure the ITM agents to use HTTPS.

Update the env.properties file in the configuration pack:
```
protocol=https
hostname=instana.example.com
tenantid=id-mycluster-account
port=42699
```
Important: Ensure the hostname matches the Common Name (CN) in the Instana host agent's certificate. If there's a mismatch, the ITM agents will fail to establish a secure connection.
Verify the keyfiles directory structure:
```
ls -la keyfiles/
```
The directory contains:
```
keyfile.kdb
keyfile.sth
keyfile.rdb
```

Run the configuration script as described in Integrating with IBM ITM V6:

Local Configuration:

./agent2server_itm.sh -i /opt/IBM/ITM -c dual -e env.properties

Remote Configuration:

# First, transfer the keyfiles directory to the remote system
tacmd putfile -m kvm-011235:LZ -s /path/to/configpack/keyfiles/keyfile.kdb -d /tmp/configpack/keyfiles/keyfile.kdb
tacmd putfile -m kvm-011235:LZ -s /path/to/configpack/keyfiles/keyfile.sth -d /tmp/configpack/keyfiles/keyfile.sth
tacmd putfile -m kvm-011235:LZ -s /path/to/configpack/keyfiles/keyfile.rdb -d /tmp/configpack/keyfiles/keyfile.rdb

# Then run the configuration script
tacmd executecommand -m kvm-011235:LZ -c "nohup /bin/sh -c 'sleep 10; /tmp/configpack/agent2server_itm.sh -i /opt/IBM/ITM -c dual -e /tmp/configpack/env.properties > /tmp/output' &" -w /tmp/configpack

The script will automatically deploy the keyring files from the keyfiles directory to $CANDLEHOME/keyfiles_icam/ on the ITM agent systems.

Verifying that ITM agent use HTTPS

After you configure ITM agents for HTTPS, verify that they are actually using the secure connection:

Check the ITM agent configuration files to verify the protocol setting:

# Check the agent's environment file for the protocol setting
grep -i "https" $CANDLEHOME/config/*.environment

The output shows entries indicating HTTPS protocol. Additionally, check the ITM agent logs for SSL or TLS-related messages during connection establishment.

Important: The Instana host agent accepts both HTTP and HTTPS connections on port 42699 when TLS is enabled. To verify ITM agents are using HTTPS, you must check the ITM agent configuration and logs, not the Instana host agent logs.

Troubleshooting HTTPS connections

If ITM agents fail to connect over HTTPS, use the following troubleshooting steps:

Certificate hostname mismatch

Symptom: ITM agents fail to connect with SSL/TLS errors in the logs.

Solution: Ensure the hostname in env.properties matches the Common Name (CN) in the Instana host agent's certificate:

# Check the CN in the certificate
openssl x509 -in /opt/instana/agent/etc/certs/tls.crt -noout -subject

# Compare with the hostname in env.properties
grep hostname env.properties

Certificate trust issues

Symptom: ITM agents report certificate validation errors.

Solution: Verify the Instana host agent's certificate is properly imported into the ITM keyring:

$GSK_CMD -cert -list -db keyfile.kdb -pw password

The list includes "Instana_Host_Agent_Cert". If not, re-import the certificate following Step 2.

ITM agent log errors

Symptom: Connection failures with no clear error message.

Solution: Check the ITM agent logs for SSL/TLS errors:

# Linux/AIX
tail -f $CANDLEHOME/logs/<short_hostname>_<product_code>_*.log

# Windows
type  %CANDLEHOME%\TMAITM6_x64\logs\<short_hostname>_<product_code>_*.log

Look for messages containing "SSL", "TLS", "certificate", or "handshake".

Test the Instana host agent TLS configuration

Symptom: Unsure if the Instana host agent is properly configured for TLS.

Solution: Use OpenSSL to verify the Instana host agent's TLS configuration:

openssl s_client -connect instana.example.com:42699 -showcerts

A successful connection will show:

The certificate chain
"Verify return code: 0 (ok)" or "Verify return code: 18 (self signed certificate)" for self-signed certs
"SSL handshake has read" messages

Keyring file permissions

Symptom: ITM agents cannot read the keyring files.

Solution: Verify the keyring files are readable by the ITM agent user:

ls -l $CANDLEHOME/keyfiles_icam/

The files must have at least read permissions for the ITM agent user. If not, adjust permissions:

chmod 644 $CANDLEHOME/keyfiles_icam/keyfile.kdb
chmod 644 $CANDLEHOME/keyfiles_icam/keyfile.sth
chmod 644 $CANDLEHOME/keyfiles_icam/keyfile.rdb

Network connectivity

Symptom: Connection timeouts or "connection refused" errors.

Solution: Verify network connectivity and firewall rules:

# Test connectivity to the Instana host agent
telnet instana.example.com 42699

# Or use nc (netcat)
nc -zv instana.example.com 42699

Ensure that:

Port 42699 is open on the Instana host agent system.
Firewall rules allow traffic from ITM agent systems to the Instana host agent on port 42699.
DNS resolution is working correctly for the hostname.

Related information

Integrating with IBM ITM V6 - Main integration documentation.
Setting up TLS encryption for agent endpoint - Instana host agent TLS configuration.