Recently I was asked to perform some simple WebSphere Application Server (WAS) administrative tasks on the z/OS platform. I have plenty of experience in administering WAS on distributed platforms, but no experience of z/OS. This presented some new challenges to complete normally straight forward tasks such as installing a new EAR which required transferring text/binary files to the host, and checking WAS logs. This blog contains a collection of commands and tips that should provide just enough z/OS orientation to complete such tasks. It assumes that WAS on z/OS has been installed already.
1. Background Reading
Basic z/OS skills from the Information Center: http://publib.boulder.ibm.com/infocenter/zos/basics/index.jsp?topic=/com.ibm.zos.zmidwebs/zmiddle_95.htm
This resource is useful, especially the sections "z/OS concepts" and "Web-based workloads on z/OS".
2. Starting and Stopping WebSphere Components (Dmgr, NodeAgents etc) using Command Line
z/OS provides a UNIX shell (known as Unix System Services (USS)) and utilities to provide an interactive interface to z/OS. You can access this in the normal way via a Telnet client such as Putty, although note the unusual port number. The z/OS administrator will need to provide you with a TSO userid (more on this later).
The commands provided are very similar to other Unix distributions, and you can refer to the command reference here:
The installation paths may differ slightly to WAS on a distributed platform, but you will find the usual scripts such as startManager.sh and startServer.sh:
Of course there are other 'native' z/OS methods to perform the same tasks, but using USS is likely to be most familiar if you don't have z/OS experience.
3. Transferring Files
FTP is the easiest way to upload files (e.g. any configuration or jar files that are not packaged in the EAR). Note that all files must be transferred as binary, including ASCII text files. This is because z/OS uses different character encoding (Extended Binary Coded Decimal Interchange Code (EBCDIC)), so forcing an ASCII transfer will corrupt the files from the point of view of z/OS.
If you are using FileZilla, configure the connection like this:
Also check the permissions once transferred. Depending on the TSO id used, WAS may not be able to read the files. If in doubt use Unix commands like chmod and chgrp to set appropriate permissions on the files and directories.
As a final tip, you can test that any jars uploaded are not corrupted using command jar -tf <jarfile.jar>, this should list the classes contained within the jar.
4. Finding and Viewing WebSphere Log Files
You will not find the WAS logs via the USS interface. To do this you must use a 3270 emulator and start a Time Sharing Option/Extensions (TSO/E) session. This is easier than it sounds!
Again, you will need to use the TSO userid provided by the z/OS Administrator to logon to your TSO session.
To view the log files, use a 3270 emulator such as IBM Personal Communications. The connection link is really easy to configure, just chose the options below and enter the IP address after pressing the Link Parameters button:
Once connected, you will most likely see a see a 'session manager' screen listing the available z/OS systems (also known as MVS by the way). The following screen shows an IBM session manager.
You can start a TSO session with the host by typing the service name and optionally including your userid, e.g. "mvsh0tso <TSO userid>", followed by the Enter key (this is usually the rightmost CTRL or the 'carriage return' key depending on keyboard configuration). If at any time you see a prompt of "***", just press Enter again.
You will be presented with a TSO logon page, where the userid is populated.
Use the tab key to complete the password field, and press the Enter key again. You may (depending on the permissions set up by the z/OS Administrator) see a full list of options for the TSO logon, including the TSO procedure name used (if problems occur with the TSO session, this is a good piece of information that the z/OS Administrator may ask for, to check what parts of TSO you are seeing) and the TSO region size (the “Size” field in the screenshot above). The TSO region size defines the available memory size for your TSO userid. If problems occur, for example when attempting to open very large files, this may need increasing; check with the z/OS Administrator but maxing this out (all 9s) should result in a message telling you what maximum session size you are allowed, which can then be used in a new logon.
Once logged on, most users work with TSO through its menu-driven interface, Interactive System Productivity Facility (ISPF). At the "Ready" prompt, launch this by typing "ispf" followed by Enter (alternatively you may find you are set up to automatically launch ISPF. Once in ISPF you will see a screen similar to the image below (but perhaps with more – or less options)).
To find log files, you must first determine the correct "Job name". A z/OS job is similar to a Unix daemon process. Assuming you already have access to the WAS admin console, you can click on the relevant server in System Administration->Cell->Local Topology to see its short name.
The short name correlates to the z/OS jobs with an additional suffix of:
"N" - z/OS controller
"S" - z/OS servant
"A" - z/OS Adjunct for asynchronous messaging (not started unless MDBs are deployed)
You can refer to the previous links to understand this z/OS terminology. If you want to look at the SystemOut logs for server1, you just need to know that they can be found in the output of the job ending "S", so in my case that was "A700D11S".
Returning then to your TSO logon which we left in an ISPF menu. Note that the one pictured below is not the default, and different installations commonly have differing ideas of the best way to customise the menu.
Type "s" followed by Enter to enter the Spool Display and Search Facility (SDSF) menu. If SDSF is not visible on the initial ISPF menu, ask the z/OS Administrator for the menu path to get there (or simply select each menu option in turn, if there are not 'too many' of them!).
In SDSF you should see a menu something like this. Again, more or less menu items may be displayed, but you will need at least the 'DA' (Display Active) and probably 'I' (for looking at jobs waiting to run), 'O' (for looking at output from jobs that have finished – or crashed!), and 'H' (similar– jobs that have been 'held' from the next process by command).
Next type "da ostc", followed by Enter. This will display the list of currently running jobs, of the particular type known as 'started tasks (ostc is 'Only Started tasks').
Now you will see a list of jobs, which will include jobs correlating to the short name of the servers seen in the WAS admin console. If the list is confusingly long, it can be filtered by typing in the command line “pre <WAS shortname>*” (e.g. pre A700D11*) followed by Enter.
Take care in the 'DA' screen, as this is one of several where entering a single character beside a job will issue a command affecting that whole job (e.g. 'e' restarts the job, 'c' cancels it, 'p' cancels and purges it from the system!). Obviously you 'shouldn't' have the authority to do any of these, but the z/OS Administrator may not be aware of your limited experience and assume you know what not to do.
In my case, the application server logs can be found in the output of Job "A700D11S". Tab to this line, and type "?" in front of the Job, e.g. "A700D11S". Press Enter.
The System.out and STDOUT streams are redirected to SYSPRINT files ("ddname"). The System.err and STDERR streams are redirected to SYSOUT "ddname". Tab to the SYSPRINT or SYSOUT line, and type "s" followed by Enter. This will display the information you would expect to see in the WAS log files.
When using this SDSF to view the log files, the following commands are useful:
pfshow on (shows, and may allow you to customise, the action of function keys that can be used)
m, Ctrl, F8 (Means "max", then down. Takes you to the end of the file)
m, Ctrl, F7 (Means "max", then up. Takes you to the top of the file)
f <string> (Find the string in a forward direction)
f <string> prev (Find the string in a backwards direction)
F5 (repeats a find command)
For common ISPF key settings see:
For an SDSF overview see:
Note that key settings will change in different parts of your ISPF session, to be relevant to the current tasks, but those listed are more often constant (unless customised).
Adam de Leeuw (IBM Innovation Centre)
Alan Ethell (IBM Accelerated Value Program)