nmon2rrd

3 likes Updated 1/15/16, 7:29 AM by naggerTags: None

nmon2rrd - takes nmon files & builds a website of graphs via rrdtool

 

nmon2rrd5g.jpg

The nmon2rrd is a AIX C program that is a sample code give away to explain how you can process the complex nmon output file. You could also compile nmon2rrd for Linux or any UNIX it is a simple filter program.

The nmon file format developed over 10 years and in the early days suffered greatly from Lotus 1-2-3 and Excel input file lilmitations so the file format tried hard to reduce the file size and numbers of lines.

nmon2rrd actually outputs three files for the rrdtool and an index.html file for the website. The rrdtool is used in three different ways:

  • rrd_create file is full of "rrdtool create" commands to create the RRD data base files.
  • rrd_update file puts the reformated nmon data in to the RRD data files with the rrdtool update" commands - this is a large file.
  • rrd_graph file then generates the performance graphs in .gif format ready foe the website using the "rrdtool graph" commands.

 

Assumptions

  • You have rrdtool installed - I get mine and the pre-reqs from http://perzl.org
  • You have a webserver to display the index.html which shows some configuration and the graphs. I use Apache from http://perzl.org
  • If you want to compile the source code or make changes and compile you will need a C complier. I use the normal IBM AIX XLC compiler - pretty sure the GNU C Compiler would be fine too.

 

Syntax to run the nmon2rrd command

# ./nmon2rrd_16

nmon2rrd -f nmonfile [-d directory] [-x] [-w width] [-h height] Version:16a

-f nmonfile the regular CSV nmon output file

-d directory dirname for the output

-w width graph width (default 800, max 1500)

-h height graph height (default 300)

-x execute the output files

Example:

nmon2rrd -f m1_030811_1534.nmon -d /webpages/docs/m1/030811 -x

 

Simple example

You have the nmon file abc.nmon in an otherwise empty directory and you want to have all the output files in the same place

nmon2rrd -f abc.nmon -x

This generates three files which contain the list of commands you then have to run manually to create the RRD data files, update them and then generate the graphs.

rrdtool - <rrd_create

rrdtool - <rrd_update

rrdtool - <rrd_graph

If you want nmon2rrd to also run the commands for you the add the -x option. Note - you MUCH check the log files to make sure there are no errors.

  • If you want to directly put the output files on to a directory or your web server then you can use the -d option.
  • Please check the output of the rrdtool commands to detect any issues in format - the line will include "ERROR" so you can quickly grep to find them.

Here is a list of the last time I ran nmon2rrd as an example on my nice CPU core I get 49 graphs:

cpu01.gif            cpu06.gif            diskbusy.gif         diskxfertotal.gif    jfsfile.rrd          mem10.gif            net.rrd              proc.rrd
cpu01.rrd            cpu06.rrd            diskbusy.rrd         file.gif             jfsinode.gif         mem2.gif             neterror.gif         procrunq.gif
cpu02.gif            cpu07.gif            diskread.gif         file.rrd             jfsinode.rrd         mem3.gif             neterror.rrd         pswitch.gif
cpu02.rrd            cpu07.rrd            diskread.rrd         filerw.gif           lpar.rrd             mem4.gif             netpacket.gif        readwrite.gif
cpu03.gif            cpu08.gif            diskreadtotal.gif    forkexec.gif         lpar1.gif            mem5.gif             netpacket.rrd        rrd_create
cpu03.rrd            cpu08.rrd            diskwrite.gif        ioadapt.gif          lpar2.gif            mem6.gif             nettotal.gif         rrd_graph
cpu04.gif            cpu_all.gif          diskwrite.rrd        ioadapt.rrd          lpar3.gif            mem9.gif             page.rrd             rrd_update
cpu04.rrd            cpu_all.rrd          diskwritetotal.gif   iototal.gif          lpar4.gif            memnew.rrd           page1.gif            rrd_update.log
cpu05.gif            diskbsize.gif        diskxfer.gif         ipc.gif              mem.rrd              memuse.rrd           page2.gif            swapin.gif
cpu05.rrd            diskbsize.rrd        diskxfer.rrd         jfsfile.gif          mem1.gif             net.gif              page3.gif            syscall.gif

Alternatively, you can have the files placed directly on your webservers directories so you don't need to copy them. Here we assume your web-server directory and you may want to have a directory per machine and per date. The nmon file is not altered.

nmon2rrd -f abc.nmon -d /webpages/docs/machineX/ dateY -x

 

Current version 21

Download

  • nmon2rrdv21.tar - 190 KB, C code and binary for AIX 6 TL7 (should work on other later AIX levels)   XYZ
    • Fixed bug on Fibre Channel write data from comment section and "j.gann"
  • nmon2rrdv20.tar- 190 KB, C code and binary for AIX 6 TL7 (should work on other later AIX levels)   XYZ
    • Some fixes
    • Added back a missing MEMUSE graph - sorry about that
    • Added the nmon source for many of the graphs which helps you work out where the stats came from like MEM or MEMNEW or MEMUSE.
    • Addition robustness features in parsing the nmon format.
    • Added two new options so you can make a larger rrdtool database for each stats type.  The idea is you can then add further nmon files to the same database and then graph all of them together - sort of a merge the data at the rrdtool database feature.
      • just run nmon2rrd_20 for the hints. As below:
      •  -n files       make the rrdtool database larger for this number of nmon files (effects the rrd_create file)
      • -g             change the rrd_graph file to make it easier to change "--start 0000001111 --end 0000002222"
      •                 extract the actual start & end seconds: rrdtool first net.rrdrrdtool last net.rrd
      • So run nmon2rrd with -n number option (but no -x), manually create the rrdtool database, and add data manually. Then run nmon2rrd on the later files and only add the data to rrdtool. ON the last file run nmon2rrd with the -g option. Manually add the last data to the rrdtool database. Run the two commands above to get the two times stamps and globally edit them into that rrd_graph script and run it. Sounds harder that it actually is.
    • Added Century to dates, so 14 becomes 2014
    • Made dates on graphs DD/MM/YYYY - more obvious which is the year!
    • If you don't that or you are America add a -b to the command for backward dates YYYY/MM/DD

       
  • nmon2rrdv17a.tar- 163 KB, C code and binary for AIX 6 TL7 (should work on other AIX levels too but not AIX 5 obviously as that is not forward compatible with newer binaries than the OS)
    • Lots of fixes and testing with 60 assorted nmon files.
    • You will need to have rddtool installed.
    • Please compile on AIX 5 or other platform with IBM Compiler or GCC free compiler. Using: cc -O3 -o nmon2rrd nmon2rrd_17.c
    • It is one simple C program and only needs the the C library.
  • Previous version (just in case): nmon2rrdv16a.tar - 140 KB, C code and binary for AIX 6 TL7 (should work on other AIX levels too)

Sample Output in a PDF file

Sample Input nmon files

At the moment nmon2rrd ignores quite a lot of data if it is present in the nmon file. These could be added but someone needs to decide exactly what needs to be graphed.

  • Some data could just be graphed as all the stats are of one type like percentages.
  • Some data is more difficult and we should only graph the data that is really interesting - so I would need specific columns that are desired.
  • Just sating all would meed hundreds more graphs, lots more processing time and little value in many grpahs = a "can't see the wood for the trees" problem

Here is the list - some are not actual generated in new versions of nmon or are only captured if you use extra nmon command line options:

  1. FRCA
  2. DISKRSIZE
  3. DISKWSIZE
  4. UARG
  5. TOP
  6. CPU00
  7. IOADAPT
  8. CPU_EC
  9. CPU_VP
  10. VGBUSY
  11. VGREAD
  12. VGWRITE
  13. VGSIZE
  14. VGXFER6
  15. CPU0
  16. PCPU
  17. SCPU
  18. POOLS
  19. DISKRXFER
  20. DISKRIO
  21. DISKWIO
  22. DISKAVGRIO
  23. DISKAVGWIO
  24. NETSIZE
  25. WLMCPU
  26. WLMMEM
  27. WLMBIO
  28. MEMPAGES4KB
  29. MEMPAGES64KB
  30. MEMPAGES16MB
  31. MEMPAGES16GB
  32. WPARCPU
  33. WPARMEM
  34. RAWLPAR
  35. RAWCPUTOTAL
  36. DISKWAIT
  37. DISKSERV
  38. DISKREADSERV
  39. DISKWRITESERV
  40. DONATE
  41. SEA