Create nmon web pages in an enterprise environment using nmon2rrd

Making you nmon pages more accessible

With nmon, you can capture performance data into a CSV file. This can then be converted into a graphic form using many tools that are available. One such rather good tool is nmon2rrd, which by using rrdtool will convert the time series data into a meaningful graphic HTML page.

David Tansley (david.tansley@btinternet.com), System Administrator, Ace Europe

David TansleyDavid Tansley is a freelance writer. He has 19 years of experience as a UNIX administrator, using AIX the last 9 years. He enjoys jogging and then relaxing by watching Formula 1, but nothing beats riding and touring on his GSA motorbike with his wife.



20 March 2014

Also available in Russian

Introduction

The setup that I share in this article works well for me. Undoubtedly, other system administrators can take a different approach.

Before I get into depth on this article, let me first summarize the tasks involved.

  • Note that you have nmon running and gathering performance data on remote hosts.
  • Install nmon2rrd to a web server.
  • Install prerequisites for rrdtool to a web server.
  • Install rrdtool to a web server.
  • Pull the previous day's nmon data files from remote hosts to a local web server.
  • Convert the nmon files using nmon2rrd and place then in a web directory.
  • Publish the web pages, served through a web menu.

nmon2rrd needs rrdtool, which requires other libraries

To download nmon2rrd, refer to the Resources section. The download link is also the wiki link for nmon2rrd which holds information about the utility. Note that nmon2rrd will not function correctly by itself, and it requires rrdtool as well, which in turn relies on other library dependencies. I have nmon2rrd working both on IBM® AIX® 5.3 and AIX 7.1 version. For AIX 7.1, perzl.org offers all the required binaries. There is also a prebuilt rrdtool RPM by Zoran Majcenic. Refer to the Resources section for more details.

After installing rrdtool along with nmon2rrd, you are all set and ready to convert. The usage from the command line is pretty straight forward and easy to understand.

nmon2rrd -f nmonfile [-d directory] [-x] [-w width] [-h height]  Version:17a
 -f nmonfile  the regular CSV nmon output file
 -d directory  dirname for the output
 -w width  graph width  (default 800, max 1500)
 -h height  graph height (default 300)
 -x  execute the output files
Example:
 nmon2rrd -f m1_030811_1534.nmon -d /webpages/docs/m1/030811 -x

And just to repeat the usage, here is another usage output, using a newly created nmon file.

nmon2rrd_17 –f /opt/nmon/ uk01wrs6010_140224_0900.nmon \
 -d /opt/apache2/htdocs/nmon_graphs/uk01wrs6010_140224_0900 -x

When converting an nmon file, I always use the –x flag, and this means that all images required by the HTML page are generated and placed in the resulting directory.


Capturing nmon data

Within my setup, I capture just short of 24 hours, well 23.75 to be precise. This gives me a good spread of data to convert into a graph. On each host crontab, I have the following entry:

00 09 * * * /usr/bin/nmon -f -t -^ -s 900 -c 95 -m /opt/dump/nmon_data

The above nmon entry equates to:

Capture nmon data every 900 seconds (15 min), for 95 occurrences (snaps).

nmon must put the captured file in /opt/dump/nmon_data. The resulting captured file name will be in the following format.

<hostname>_<yymmdd>_<hhmm>.nmon

That is, for example:

uk01wrs6010_140224_0900.nmon

nmon to capture (-^) Fibre Channel data and (-t) include top processes, and the output to go into a (-f) spreadsheet format.

Be careful about the number of snaps you specify in the capture. In fact, let's discuss that next.


Watch your snap count, it can be too much information

When capturing data, you need to be realistic on how much data you want to display. nmon2rrd can only convert so many snaps, there is absolutely no point in my opinion having more than say 500 to 600 snaps, if viewing the graph on a normal screen. The graphic will certainly not be able to display all of these points in a graph. At worst, your data might get truncated when being converted into HTML using nmon2rrd.


Previous day’s nmon data

We have all the utilities installed, right? Well, not so fast. Remember, our nmon runs for a 24-hour cycle. When we go around all the remote hosts, assuming that the collection starts at 09:00 to collect the nmon files we might want to collect the previous day’s data because the current (today’s) nmon will be running! First, make sure your Network Time Protocol (NTP) service is running and synchronizing with a time server on all hosts and second, have a method for obtaining the previous day’s date in a variable, and this will be used as part of the scp command later. You can get the previous day’s date by using the TZ environment variable , or install GNU date that can do it for sure. Or, install a lightweight date utility such as yest (refer to the Resources section). For example, to produce previous day’s date (assuming that today is Feb 25) in the date format of an nmon file, I can use:

$ yest "%y%m%d"
140224

As can be seen, it matches the previous day's nmon date format of the file:

uk01wrs6010_140224_0900.nmon

To display the date 21 days ago, use: yest -21


Pulling and converting nmon data

To pull the previous day's nmon file from all remote hosts located in /opt/dump/nmon_data to a staging area on your web server, a simple scp command can be run within a loop. I find it best to hold a file containing all the hosts that I will connect with and read through that file. Then, parsing the previous day's date as part of the scp command ensures that I get the previous day’s nmon file. For example, a partial framework follows:

dest_dir=/opt/dump/nmon_data/
incoming=/opt/dump/incoming
yesday=$(yest "%y%m%d")
cat /usr/local/bin/all_nmon_server.txt | while read hosts
do
scp $host:$dest_dir{host}_$yesday*.nmon $incoming
done

In the above code, all_nmon_server.txt is a file that contains a remote host one per line, that is:

host_1
host_2
…

All nmon files are now located in /opt/dump/incoming. So, let's now do some conversion.

For my nmon graphs, I am creating a separate directory for each nmon entry. So, I need to create a directory that represents the remote host's nmon file in the format: <hostname>_yymmdd_<hhmm>, looks familiar right? Easy enough just pull off the host name from the file name? There are many ways to do this, but I'll use:

thisdirec=$(echo $file | awk -F '.' '{print $1}')

Where, the $file variable is the current nmon file name being read through a loop.

That is, uk01wrs6010_140224_0900.nmon becomes uk01wrs6010_140224_0900.

In the following example, my resulting web page and GIF images will be located in: /opt/apache2/htdocs/nmon_graphs/<hostname>_<yymmdd>_<hhmm>

if ! [ -d /opt/apache2/htdocs/nmon_graphs/$thisdirec ]
   then
mkdir -m 754 -p /opt/apache2/htdocs/nmon_graphs/$thisdirec
…

Where, $thisdirec holds the directory name.

After a few days of growth, the directories for a host might be:

$ ls -l |grep uk01wrs6014
drwxrwxr--    2 www      www         2048 Feb 18 09:41 uk01wrs6014_140217_0900
drwxrwxr--    2 www      www         2048 Feb 20 09:18 uk01wrs6014_140219_0900
drwxrwxr--    2 www      www         2048 Feb 21 09:18 uk01wrs6014_140220_0900
drwxrwxr--    2 www      www         2048 Feb 22 09:20 uk01wrs6014_140221_0900

Now, let's convert using nmon2rrd (I have already gone over the usage). If you want to create all the images at the same time with a single click, then append the executable flag '-x' to the command.

nmon2rrd_17 -f $incoming/$file -d /opt/apache2/htdocs/nmon_graphs/$thisdirec –x

The $file, $thisdirec, and $incoming variables have already been noted previously. After running the above all, the GIF and the rrd files (along with the resulting index.html file) will be present in the newly created directory. As noted, you can remove some of these rrd log files, but because they are small in size, I leave them as is. If you get issues with a HTML page not presenting the data correctly, you should review these files.

Now, we have a HTML file called index.html. It is always recommended to rename it to the actual host name it is representing. Also, the HML file will have no link on the page to go back to a menu. We are going to have a menu to view these files, as otherwise, you might have to be there all day trying to pick the right HTML file to view, especially if you have many hosts that have been logged over a long period of time.

Here is an example code snippet to rename the file to a more meaningful file name.

uk01wrs6010_140224_0900.nmon.html

mv /opt/apache2/htdocs/nmon_graphs/$thisdirec/index.html /opt/apache2/htdocs/nmon
_graphs/$thisdirec/$file.html

Next let's put a link in the file, so that the user can quickly go back to the menu. Shortly, I will discuss about the cgi scripts, but for now, assume the following:

Web server host: uk01wrs6010

cgi-bin menu page: nmon_menu.cgi

The following sed command inserts a HTML link in the HTML file that points back to the nmon_menu cgi web page.

sed '1a\
<a href="http://uk01wrs6010/cgi-bin/nmon_menu.cgi/">Back to nmon menu</a>' /opt/a
pache2/htdocs/nmon_graphs/$thisdirec/$file.html >/tmp/holdme
cp -p /tmp/holdme /opt/apache2/htdocs/nmon_graphs/$thisdirec/$file.html

Got the HTML’s, now let’s present them

So, now we are all set to serve these nmon files up when selected by a user within a web environment. The following list provides a summary of the tasks to do next.

  • Create a cgi menu to present the available hosts that have nmon graphs.
  • After a host is selected, present a cgi menu for the available dates for that host containing the nmon graphs.
  • Publish the nmon graph with a HMTL link to go back to the cgi menu.

Select your server

It makes sense to present the hosts within a menu. In the following output, we use the grep command to pattern search for AIX hostnames that form part of the directory name. Note that the nmon graphs are held within the /opt/apache2/htdocs/nmon_graphs directory.

The hostname part is then extracted and duplicate names are removed. Remember that we will have separate directory entries for each nmon data, and therefore, we need to remove the duplicates.

Figure 1. nmon.cgi shows the nmon menu displaying the hosts
figure1

Click to see larger image

Figure 1. nmon.cgi shows the nmon menu displaying the hosts

figure1
echo "Select Server name:<br><SELECT NAME=\"srvname\">"
cd /opt/apache2/htdocs/nmon_graphs
hosts=$(ls | egrep 'hrs|wrs|prs|drs'|awk -F '_' '{print $1}'| uniq)
for loop in $hosts
do
                echo "<OPTION>$loop"
done
echo "</td>"
echo "</table><br>"

After a server is selected, it is parsed ($srvname) as part of the QUERY_STRING to the next cgi-bin script.

Got your server, now select your dates

The user has now a selected a server. Next, the user can select the date of the nmon graph that the user wants to view. Figure 2 shows the dates available when nmon graphs are present for a given host. The next cgi-bin script then gets the server name that was parsed to it from the previous script.

Figure 2. Dates available when nmon graphs are present for a given host
figure2

Click to see larger image

Figure 2. Dates available when nmon graphs are present for a given host

figure2

This is easily obtained using the following code.

server=$(echo $QUERY_STRING | awk -F= '{print $2}')

Next, we use the grep command for all directories containing that server name and pull off the directory name, which as noted previously, would be in the following format.

 uk01wrs6014_140221_0900

Here's the partial code to achieve the above:

<FORM ACTION="/cgi-bin/nmon_display.cgi" METHOD="GET">
<input type="hidden" name="servername" value=$server>
EOF
echo "<BR><BR><BR><BR><BR><BR><BR><BR><BR><TABLE BORDER=0 ALIGN=CENTER width=600>"
echo "<TR align=center><TD>"
echo "Select Run Date Collection for $server (yymmdd_hhmm):<br><SELECT NAME=\"nmon_file\">"
cd /opt/apache2/htdocs/nmon_graphs
 datelist=$(ls -l |grep ^d|grep $server |awk '{print $9}')
for loop in $datelist
do
                echo "<OPTION>$loop"
done
echo "</td>"
echo "</table><br>"

The user has selected an nmon date of the graph the user wants to view. The $servername, $nmon_file variables are now parsed to the next cgi script that displays the actual nmon graph.

Here comes your graph

All we need to do now is present the HTML file. Figure 3 shows the nmon graph with the HTML link. The server name and the nmon file are extracted from the QUERY_STRING variable using the following commands.

servername=$(echo $QUERY_STRING | awk -F '&' '{print $1}' | awk -F'=' '{print $2}')
nmon_file=$(echo $QUERY_STRING | awk -F '&' '{print $2}' | awk -F'=' '{print $2}')
Figure 3. Link inserted (at the top) using the sed command
figure3

Click to see larger image

Figure 3. Link inserted (at the top) using the sed command

figure3

Now, we can serve the page up using the values parsed from the previous script.

echo "<head>"
echo "<META HTTP-EQUIV='Refresh' CONTENT='1; URL=http://uk01wrs6010/nmon_graphs/$
{nmon_file}/${nmon_file}.nmon.html'/>";
echo "</head>"

Refer to Figure 4 for the nmon graph showing the fiber and disk adapter usages.

Figure 4. Fibre and disk adapter
figure4

Click to see larger image

Figure 4. Fibre and disk adapter

figure4

Conclusion

The setup presented here allows for easy access through a web page to view nmon graphs from different hosts. For housekeeping, be sure to regularly run the find command to remove old directories.


Resources


Acknowledgement

Many thanks to Zoran Majcenic for providing the resource files.


Download

DescriptionNameSize
Code samplerrdtool.zip3 MB

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into AIX and Unix on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=AIX and UNIX
ArticleID=966413
ArticleTitle=Create nmon web pages in an enterprise environment using nmon2rrd
publish-date=03202014