Example: sshd

The Secure Shell Daemon application (SSH daemon or sshd) is the daemon program for ssh. This program is an alternative to rlogin and rsh and provides encrypted communications between two untrusted hosts over an insecure network. The sshd is the daemon that listens for connections from clients on port 22.

It is normally started when z/OS® UNIX is initialized. It forks a new process for each incoming connection. The forked processes/connections handle key exchange, encryption, authentication, command execution, and data exchange. These connections show the same jobname and Command/Path and Filter as the SSH daemon does. At sshd startup time its process ID (pid) is written in the /var/run/sshd.pid file.

Any adaptation and configuration changes to the sshd can be done in the sshd configuration file sshd_config. It is located in the /etc/ssh directory.

Keeping the Secure Shell Daemon application (sshd) highly available requires that the sshd will not be detached from its parent process. Additionally, the sshd must be started in a separate shell environment. This shell is needed to establish a unique process which can be monitored. It can be accomplished by starting the sshd with option -D

For shutdown purposes it is required that the process ID file (sshd.pid) is written to your file system. This process ID will be read from that file and used to identify the sshd to terminate.

The ps -ef command supplies further parameters to identify the process referenced as Filter , for example:

ps -ef | grep ssh
 UID	    PID        PPID      C STIME TTY	         TIME CMD
 AUTO1   83886553   67109368  - 14:53:27 ?         0:00 -sh -c /usr/sbin/sshd -D
 AUTO1   83886563   50335037  - 14:53:37 ttyp0000  0:00 grep ssh
 AUTO1   83887096   83886553  - 14:53:28 ?         0:00 /usr/sbin/sshd -D

Process 83886553 represents the address space containing the covering parent shell process for monitoring purposes. Process 83887096 is the sshd itself.

From this output, set the Filter as -c '/usr/sbin/sshd -D' .

Note: Even though the quotation marks are not shown in the output for the command ps -ef, they must not be defined in the Filter field of the USS Control policy.

To check for the required information for the Command/Path issue the ps -e command and look for the process Id of the parent shell:

PID		  TTY TIME		COMMAND 
... 
83886553	?	14:53:27	/bin/sh 
67109368	?	14:34:28	BPXBATCH 
83887096	?	14:53:28	/usr/sbin/sshd 
…

or issue ps -e | grep 83886553 to get the process directly:

83886553	?	14:53:27	/bin/sh

This shows the process ID (PID) for the sshd monitoring process. From this output, set the Command/Path as /bin/sh

Next find out the z/OS user ID that the process is running on by issuing the following z/OS command and locating the user ID in the first column where the process ID (PID) is listed:

D OMVS,PID=83886553

The following output is displayed:

BPXO070I 19.01.02 DISPLAY OMVS 035
 OMVS     000E ACTIVE             OMVS=(PA,F9,11,LO,S0,09)
 USER     JOBNAME  ASID        PID       PPID STATE   START     CT_SECS
 OMVSKERN SSHD     00D0   83886553   67109368 1WI----- 14.53.27      .0
 LATCHWAITPID=         0 CMD=-sh -c /usr/sbin/sshd -D
 THREAD_ID        TCB@     PRI_JOB  USERNAME   ACC_TIME SC  STATE
 21ADF00000000001 008FF1C0 OMVS                    .006 WAT  W

From this output, set the User ID to OMVSKERN

Verify that sshd listens on port 22 as stated above. Use the netstat -a command and evaluate the output by looking for the SSHD jobname:

# netstat -a
 ...
 SSHD    00000049 Listen
 Local Socket:   0.0.0.0..22
 Foreign Socket: 0.0.0.0..0
 …

From this output, set the port to 22.

You can define the UNIX internet daemon (sshd) using the fields of the USS Control policy item for applications (APLs) of type USS in the customization dialogs with, for example, the data in Table 1.

Table 1. Example Customization Dialog Definitions for sshd
	Process	File	Port
Application Name*	SSHD/APL	SSH_FILE/APL	SSH_PORT/APL
User ID	OMVSKERN
Process Command/Path	/bin/sh
Filter	-c '/usr/sbin/sshd -D'
File Name		/var/run/sshd.pid
PORT			22
IP Stack			TCPIP**
* This is the name that was specified for the applications when they were created.** Only if the system is configured for multiple IP stacks.

Define a basic group containing all resources with relationships that indicate that:

The group containing all sshd related resources depends on TCPIP
The file is created by the sshd process and can never be started or created directly by SA z/OS
The sshd process listening on the port can never be started or created directly by SA z/OS.

Figure 1 illustrates the SSHD (modeled as a group) as up and running when the process /bin/sh -c '/usr/sbin/sshd -D' started by user OMVSKERN appears, the file /var/run/sshd.pid exists, and port 22 is in the status 'listen' (sshd listens to this port for incoming login requests). You can only choose a port that is defined in /etc/ssh/sshd_config .

Start definition for SSHD_FILE/APL

None.

Start definition for SSHD_PORT/APL

None.

Start definition for SSHD/APL

CMD: INGUSS JOBNAME=&SUBSJOB &SUBSPATH &SUBSFILTER (&SUBSxxx variables are substituted at runtime)

Stop definitions for SSHD_FILE/APL

These commands remove the file if it has not yet been removed by the sshd process.

Table 2. Pass description for sshd
Number of Pass	Pass Description	Command
1	If not done yet by sshd	INGUSS /bin/rm &SUBSFILE
4	If Pass 1 did not remove the file	INGUSS /bin/rm -f &SUBSFILE

Stop definitions for SSHD/APL

When stopping the SSHD the correct proccess ID from the sshd.pid file must be extracted. The kill command is used to terminate the sshd process itself. Once the sshd command is issued the parent shell used for monitoring purposes terminates and the SSHD/APL will go into a AUTODOWN status.

Use the following commands:

CMD:INGUSS /bin/sh -c '/bin/ps -ef | /bin/grep `/bin/cat /var/run/sshd.pid` | /bin/grep -q sshd && /bin/kill `/bin/cat /var/run/sshd.pid`'

CMD:INGUSS /bin/sh -c '/bin/ps -ef | /bin/grep `/bin/cat /var/run/sshd.pid` | /bin/grep -q sshd && /bin/kill -9 `/bin/cat /var/run/sshd.pid`'

CMD:INGRCLUP &SUBSJOB

These commands and the USS pipe are described here:

Table 3. INGUSS Command and USS Pipe Summary
Command	Task
INGUSS	Is the interface to USS
/bin/sh -c '...'	Provides a fully-equipped shell environment to extract the pid of the sshd
`/bin/cat /var/run/sshd.pid`	Subshell extracts the pid of the sshd: the extract pid is represented by 'extracted_pid' in the left column
/bin/ps -ef	Asks for all USS processes
\| /bin/grep `extracted_pid`	Shows only the sshd process
\| /bin/grep -q sshd	Suppresses output and returns the appropriate return code: return code 0: sshd process exists return code 1: sshd process does NOT exist
&&	Continues only if return code is 0 / if sshd exists
/bin/kill `extracted_pid` or /bin/kill -9 `extracted_pid`	Issues SIGTERM signal against the 'extracted pid' of the sshd

In case, the kill command does not terminate the sshd process use INGRCLUP routine to invoke a z/OS CANCEL command against the address spaces of the sshd and the monitored parent shell.

SA z/OS provides the *USS best practices policy that provides definitions for several automated USS daemons, such as sshd. Common definitions for USS resources can be found in the APL classes starting with C_USS_xxx .

Use also the Unix man pages to get more information about the used USS commands and their parameters.