Configuring external logging server

This task describes logging configurations.

Before you begin

OSO Appliance installed on all LPARs is a locked down environment where one cannot log in and view the logs. OSO Appliance forwards logs to an rsyslog instance that should be provided to integrate the OSO appliance.

The activity owner of this task is sscadm2. The sscadm2 works with the external logging server owner to perform this integration.

About this task

The details of the external logging server must be provided to sscadm2.

The following information describes the responsibilities of the external logging server owner and sscadm2:

The owner of the external logging server shares the CA certificate configured on the logging server with sscadm2.
The sscadm2 configures logging in all the LPARs, such as LPAR1, LPAR2, and LPAR3. LPAR1 is also configured with log forwarding to an external logging server.
The sscadm2 gets the logging CA certificate of LPAR1 and shares it with the external logging server owner.
The external logging server owner reconfigures the logging server including the CA certificate of LPAR1.

Note: When configuring external logging, ensure that the external logging server is on the same network as the LPAR1 management network and is reachable through the LPAR1 management interface.

External logging server configuration

Edit online

This section describes the procedure for integrating the OSO appliance with an external rsyslog server. The activity owner for this procedure is External logging server owner.

Procedure

Install the rsyslog and rsyslog-gnutls packages, where the rsyslog server will be executed.
To configure rsyslog logging with TLS, you have to generate CA certificates.
Create a directory if it does not exist. Change to the rsyslog-certs directory:
```
mkdir -p /etc/rsyslog-certs
cd /etc/rsyslog-certs
mkdir ca
```
Generate a self-signed CA key:
```
openssl genrsa -out ca/ca.key 4096
```

Generate a self-signed CA certificate:

openssl req -x509 -new -nodes \
-key ca/ca.key \
-sha256 -days 365 \
-out ca/ca.crt \
-subj "/C=US/ST=NA/L=NA/O=RSyslog-CA/OU=Logging/CN=rsyslog-ca"

Generate ca.crt and add the value of the below command in the rsyslog section of the bootstrap _config.yaml file.
```
cat ca/ca.crt | base64 -w0
```
The generated ca.crt must be provided in LPAR1 log forwarding configurations as described in the Table of For global fields under Configuring OSO Appliance.
Once bootstrap completes, acquire the lpar1_logging_ca.pem from sscadm2 after the bootstrap process:
```
cat lpar1_logging_ca.pem | base64 -d > /etc/rsyslog-certs/ca/lpar1-ca.crt
```
To configure rsyslog logging with TLS, you have to generate server certificates.

Create a rsyslog certificate directory:

mkdir -p /etc/rsyslog-certs
cd /etc/rsyslog-certs
mkdir server

Generate a server key:

openssl genrsa -out server/server.key 4096

Prepare the SAN config for the server certificate.

Create a file server/server.cnf with the following content:

[ req ]
default_bits = 4096
default_md = sha256
prompt = no
encrypt_key = no
distinguished_name = dn
req_extensions = req_ext
[ dn ]
C = US
O = RsyslogTestServer
CN = <logging server ip>
[ req_ext ] 
subjectAltName = IP:<logging server ip>
extendedKeyUsage = serverAuth

Generate server.csr using server.cnf:

openssl req -config server/server.cnf -key server/server.key -new -out server/server.csr

openssl x509 -req -in server/server.csr -days 365 \
-CA ca/ca.crt -CAkey ca/ca.key -CAcreateserial \
-extfile server/server.cnf -extensions req_ext \
-out server/server.crt

Rsyslog server configuration

Edit online

Procedure

Create ca-bundle.crt with ca.crt and lpar1-ca.crt that was obtained during the External logging server configuration procedure:
```
cat ca/ca.crt ca/lpar1-ca.crt > ca/ca-bundle.crt
```

Copy hpvs-environment/rsyslogd_server_conf.template to /etc/rsyslog.d/server.conf and update the TLS certificate file path for ca-bundle.crt, server.crt, server-key.pem.

Example configuration:

# ==============================
# Rsyslog server TLS configuration
# Receives logs from clients
# ==============================

# Load modules
module(load="imtcp")             # TCP input
module(load="imjournal")         # Read from systemd journal
module(load="omjournal")         # Output to local journal

# TLS default for incoming connections
$DefaultNetstreamDriver gtls
$DefaultNetstreamDriverCAFile /etc/rsyslog-certs/ca/ca-bundle.crt
$DefaultNetstreamDriverCertFile /etc/rsyslog-certs/server/server.crt
$DefaultNetstreamDriverKeyFile /etc/rsyslog-certs/server/server.key

# Template for logging to journal
template(name="journal" type="list") {
    property(outname="SYSLOG_FACILITY" name="syslogfacility")
    property(outname="SYSLOG_IDENTIFIER" name="app-name")
    property(outname="HOSTNAME" name="hostname")
    property(outname="MESSAGE"  name="msg")
}

# Ruleset to log incoming messages to local journal
ruleset(name="journal-output") {
    action(type="omjournal" template="journal")
}

# TCP Listener with TLS
input(
    type="imtcp"
    port="6514"
    ruleset="journal-output"
    StreamDriver.Name="gtls"
    StreamDriver.Mode="1"
    StreamDriver.Authmode="x509/certvalid"
)

Manage the rsyslog server port.
- By default, rsyslog runs on port 6514 when TLS is configured. You must open port 6514 in the firewall to allow incoming logging traffic from the OSO LPAR1. Alternatively, if appropriate for your environment, you can disable the firewalld service.
Restart the rsyslog server with the latest configuration:
```
systemctl restart rsyslog
```
Check the status of the Rsyslog server using the command:
```
systemctl status rsyslog
```