Installing the host agent on IBM z/OS - UNIX System Services (USS)

To monitor IBM WebSphere, IBM WebSphere Liberty, and JVM-based applications on z/OS locally, install the Instana host agent on the z/OS-UNIX System Services (USS) layer.

To install and manage the Instana agent on z/OS-USS layer, see the following sections:

Before you install

Before you install the Instana agent on z/OS-USS layer, make sure to check the supported sensors and complete the prerequisite steps:

Supported sensors

See the following list of supported sensors, which can be automatically discovered on z/OS:

Prerequisites

You must complete the following steps before you install the Instana agent:

Download and run the `zOS-Agent-Prereq.sh` script

The zOS-Agent-Prereq.sh script verifies the prerequisites for the Instana agent on z/OS and installs missing prerequisites. To run the script, complete the following steps:

Download the following files and move them to the z/OS-USS layer by using SCP, SFTP, or by copying and pasting.
- zOS-Agent-Prereq.sh script file from the Instana GitHub repository.
- meta-main.XXX.zos.pax.Z file from the metaport GitHub repository.
Switch to bash shell and run the zOS-Agent-Prereq.sh script with a user which has root privilege by using the following commands:
```
bash
```
- if you moved the script file to the z/OS-USS layer by using SCP or SFTP, then run the following command:
```
chmod +x ./zOS-Agent-Prereq.sh && chtag -R -tc 819 ./zOS-Agent-Prereq.sh && ./zOS-Agent-Prereq.sh
```
- If you copied and pasted the script file to the z/OS-USS layer instead of using SCP or SFTP, then run the following command:
```
chmod +x ./zOS-Agent-Prereq.sh && ./zOS-Agent-Prereq.sh
```
Enter the path to the meta-main.XXX.zos.pax.Z file that you downloaded when the script prompts.

The script verifies the prerequisites for installing the Instana agent on a z/OS environment. If any prerequisites are missing, the script downloads them.

Set up a security profile (for WebSphere Application Server and WebSphere Liberty monitoring)

To trace IBM WebSphere Application Server and WebSphere Liberty on z/OS, you must set up a security profile by using Resource Access Control Facility (RACF) or other mainframe security products.

You can set up a security profile by using the following approaches:

Create a generic profile

Create a generic profile named BPX.SRV.** in the SURROGAT class with read permission UACC(READ).
Create profiles for specific user IDs

For specific user IDs (for example, USERA), create profile BPX.SRV.USERA with UACC(NONE) and then provide read access to individual surrogate users or groups.

Enable WebSphere Application Server monitorinf

To monitor WebSphere Application Server, you must set up the WebSphere server with following attach flags enabled:

  - Djdk.attach.allowAttachSelf=true
  - Dcom.ibm.tools.attach.enable=yes

Enable JMX for WebSphere Liberty Metrics

To collect the metrics for WebSphere Liberty, you must enable the monitor-1.0 feature. To enable this feature, open the server.xml file, which is located in the <websphere-liberty_install_dir>/usr/servers/<specific_server>/ directory, and then add the following lines to the file:

<featureManager ...>
    ...
    <feature>monitor-1.0</feature>
    ...
</featureManager>

Before you install the agent, make sure that Java Development Kit (JDK) 11 is installed in your system.

Choosing how you want to install the agent

Choose one of the following installation options and click the associated link to continue:

In an online environment (where the host can communicate with the Instana hosted agent repositories), you can install the host agent automatically by using the one-liner installation technique. For more information, see Installing the agent by the automated (one-liner) script.
In an air-gapped environment, you can install and run the agent from a .tar.gz file.

To install and run the agent from the .tar.gz file, download and extract the file, and then start the agent. For more information, see Installing and running the agent from a .tar.gz file.

Installing the agent by the automated (one-liner) script

You can use the agent one-liner script to automatically install the host agent on z/OS-USS layer. To install the host agent by using the one-liner, complete the following steps:

On the home page of the Instana UI, click Deploy Agent.
On the agent deployment catalog page, click the tile z/OS - Automatic Installation (One-liner).
Select the agent packaging mode (Dynamic or Static) that you want to deploy. To understand the difference between static agents and dynamic agents, see Host agent types.
Copy the script to the z/OS-USS layer where you want to install the host agent, and then run the script to install the host agent.

The script is pre-filled with your agent key and host agent endpoint.

If you change any option, then the script parameters are automatically updated.

See the following sample script to install the agent with the Dynamic packaging option:
```
curl -o setup_agent.sh https://setup.instana.io/agent && chmod 700 ./setup_agent.sh && chtag -R -tc 819 ./setup_agent.sh -a <your_agent_key> -d <your_agent_key> -t dynamic -e <host-agent-endpoint>
```
You need root user privileges to install the Instana host agent.

You must first switch to bash shell by using the bash command before running the script.

Now the host agent is installed. View the agent on the infrastructure map by clicking View Deployed Agents.

The public system package repository of Instana is added to your system by creating a repository file. This file adds Instana authenticated repositories to the machine's installation sources.

One-liner script parameters

The one-liner script accepts the following parameters:

Table 1: One-liner script parameters
Parameter	Description
`-a` = (required)	The agent key.
`-d` = (optional)	The Instana download key. If you are using a self-hosted (on-premises) Instana deployment, the key is provided for you by Instana.
`-e` = (required)	The host agent endpoint.
`-m` = (optional)	Sets the Instana agent mode, `apm` (default), `infra`.
`-t` = (optional)	The agent type, `dynamic` (default) or `static`.

Installing and running the agent from a `.tar.gz` file

If you want to install and run the agent from a .tar.gz file, download and extract the file, and then start the agent.

The .tar.gz file is pre-filled with your agent key and host agent endpoint.

Follow the steps to install the agent from a .tar.gz file:

Checking the prerequisites
Downloading the agent
Moving the agent to z/OS- USS layer
Starting the agent

Downloading the agent

Download the agent package from the Instana UI by completing the following steps：

On the home page of the Instana UI, click Deploy Agent.
On the agent deployment catalog page, click the tile Linux - Archive (tar.gz).
In the Packaging section, select the agent type (Dynamic or Static) that you want to deploy. To understand the differences of static agents and dynamic agents, see Host agent types.

For air-gapped environments, use Static packaging.
In the dropdown list, select the platform architecture (operating system - s390x).

The archive file is automatically pre-configured with your Instana account settings so that you only need to extract and start the agent.
Click the download icon to download the host agent package file (tar.gz) from your web browser.

Now the host agent is downloaded.

Moving the agent to z/OS-USS layer

Move the downloaded agent package from your system to the z/OS-USS layer by using SCP or SFTP. See the following example:

scp <path-to-file-instana-agent-linux-s390x.tar.gz> <Username>@<zOS-IP>:/<PATH-TO-COPY-FILE-ON-zOS-USS>

Then, download the setup_agent_airgapped.sh script file from the Instana GitHub repository and move it to z/OS-USS layer.

Starting the agent

After you move the agent package to the z/OS-USS layer, you can start the agent either manually or by using the setup_agent_airgapped.sh script.

Starting the agent by using script

Switch to bash shell and run the setup_agent_airgapped.sh script with a user which has root privilege by using the following commands:

bash

if you moved the script file to the z/OS-USS layer by using SCP or SFTP, then run the following command:

chmod +x ./setup_agent_airgapped.sh && chtag -R -tc 819 ./setup_agent_airgapped.sh && ./setup_agent_airgapped.sh

If you copied and pasted the script file to the z/OS-USS layer instead of using SCP or SFTP, then run the following command:
```
chmod +x ./setup_agent_airgapped.sh && ./setup_agent_airgapped.sh
```

Starting the agent manually

Move the downloaded agent package file to a system-wide accessible location, and run the agent with a user which has root privilege.

Extract the package by running following command:

gzip -d filename.tar.gz && pax -r -f filename.tar

Enable code page setting by running the following command:

chtag -R -tc 819 instana-agent/bin && chtag -R -tc 819 instana-agent/etc

Confirm that the tag is enabled by running the following command:

ls -lTr instana-agent/bin && ls -lTr instana-agent/etc

See the following sample response:

NGHON:/u/nghon/ts/instana-agent #>ls -lTr bin/
total 7360
t ISO8859-1   T=on  -rwxr-xr-x   1 ROOT     OMVS        2659 Mar 18 09:53 stop
t ISO8859-1   T=on  -rwxr-xr-x   1 ROOT     OMVS        2656 Mar 18 09:53 status
t ISO8859-1   T=on  -rwxr-xr-x   1 ROOT     OMVS        2808 Mar 18 09:53 start
t ISO8859-1   T=on  -rw-r--r--   1 ROOT     OMVS       14526 Mar 24 10:56 setenv
t ISO8859-1   T=on  -rwxr-xr-x   1 ROOT     OMVS     2293912 Mar 18 09:53 memory_calculator
t ISO8859-1   T=on  -rwxr-xr-x   1 ROOT     OMVS       16767 Mar 18 09:53 karaf
t ISO8859-1   T=on  -rwxr-xr-x   1 ROOT     OMVS       13806 Mar 18 09:53 inc
t ISO8859-1   T=on  -rwxr-xr-x   1 ROOT     OMVS        4098 Feb 27 04:52 agent-java-check
t ISO8859-1   T=on  -rwxr-xr-x   1 ROOT     OMVS     1357420 Feb 27 04:52 agent-diagnostic.jar
NGHON:/u/nghon/ts/instana-agent #>

Enable automatic character conversion by running the following command:
```
export _BPXK_AUTOCVT=ON
```
You are suggested to set character conversion to ON in your user profile to avoid exporting it each time you log in.
Set the JAVA_HOME environment variable name for Instana agent and set the PATH:

See the following sample commands:
```
export JAVA_HOME=/usr/lpp/zWebSphere/Liberty/V9R0/java/8.0
export PATH=$JAVA_HOME/bin:$PATH
```
You are suggested to set the environment variable either in your user profile or in the instana-agent/bin/setenv file to avoid exporting it each time you log in.
Start the agent by running the following command:
```
INSTANA_AGENT_FOLDER/bin/start
```

To stop the agent, run the following command:

INSTANA_AGENT_FOLDER/bin/stop

To check the status of the agent, run the following command:

INSTANA_AGENT_FOLDER/bin/status

Now the host agent is installed and started. For what you can do after the installation, see What to do next.

Troubleshooting

If you notice errors during the installation or running of the agent, check the log messages and check the troubleshooting tips. For troubleshooting information that is general to all host agents, see Troubleshooting.

Agent startup failure

When you start the Instana agent from cd instana-agent/bin/ && ./start, the agent might fail to start with the following error:

Fails with karaf: JAR files could not be found. Please check tar command if you run into path length limitation. We advise the use of GNU tar for uncompressing the host agent tarball. Could not resolve mvn:org.apache.felix/org.apache.felix.framework/5.6.12

Cause: The agent archive .tar` file is corrupted.

Troubleshoot step: Run the following commands:

ls instana-agent/system/org/apache/felix/org.apache.felix.framework/5.6.12

Expected result: The org.apache.felix.framework-5.6.12.jar file is listed.
Solution: Make sure that the instana-agent-linux-s390x.tar.gz file is extracted on your local environment (MacOS or Windows), and the instana-agent-linux-s390x.tar file is moved to the z/OS-USS layer and extracted by using tar –xvf filename.tar or /pax -r -f filename.tar commands.

Agent unable to attach with WebSphere server

If the Instana agent is unable to attch to the WebSphere server, use the following scripts to debug whether the required flags are enabled to monitor WebSphere Application Server.

WebSphere-zOS-Prereq.sh script

This script verifies the prerequisites for WebSphere monitoring, such as whether the WebSphere attach flag is enabled by using the Instana agent, and responds with all the PIDs that are monitored.

Download the WebSphere-zOS-Prereq.sh script file from the Instana GitHub repository and move it to z/OS-USS layer by using SCP, SFTP, or by copying and pasting.
Switch to bash shell and run the script from USS layer of z/OS with a user which has root privilege by running the following commands:
```
bash
```
- if you moved the script file to the z/OS-USS layer by using SCP or SFTP, then run the following command:
```
chmod +x ./WebSphere-zOS-Prereq.sh && chtag -R -tc 819 ./WebSphere-zOS-Prereq.sh && ./WebSphere-zOS-Prereq.sh
```
- If you copied and pasted the script file to the z/OS-USS layer instead of using SCP or SFTP, then run the following command:
```
chmod +x ./WebSphere-zOS-Prereq.sh && ./WebSphere-zOS-Prereq.sh
```

WebSphere-Pid-Trace-Enable.sh script

This script checks the prerequisites for WebSphere monitoring, such as whether the WebSphere Attach Flag is enabled by using the Instana agent. It prompts you to enter the required PID and verify if the flag is enabled for that PID.

Download the WebSphere-Pid-Trace-Enable.sh script file from the Instana GitHub repository and move it to z/OS-USS layer by using SCP, SFTP, or by copying and pasting.
Switch to bash shell and run the script from USS layer of z/OS with a user which has root privilege by running the following commands:
```
bash
```
- if you moved the script file to the z/OS-USS layer by using SCP or SFTP, then run the following command:
```
chmod +x ./WebSphere-Pid-Trace-Enable.sh && chtag -R -tc 819 ./WebSphere-Pid-Trace-Enable.sh && ./WebSphere-Pid-Trace-Enable.sh
```
- If you copied and pasted the script file to the z/OS-USS layer instead of using SCP or SFTP, then run the following command:
```
chmod +x ./WebSphere-Pid-Trace-Enable.sh && ./WebSphere-Pid-Trace-Enable.sh`
```