Topic
  • 21 replies
  • Latest Post - ‏2015-08-24T00:28:49Z by sun.aus
sun.aus
sun.aus
12 Posts

Pinned topic nmon graph web tools

‏2013-12-09T04:18:13Z |

2014-01-21 New:

Now include CPU/POOLS/MEMORY/PAGING/PROC/AIO/Kernel/Disk/FS/Network/WLM/TOP/Filesystem/SEA

Anyone can download this on, (updated on 2014-Feb-07)

https://drive.google.com/file/d/0B4yhhRexQKzyYndSVlFrYXZRbms/edit?usp=sharing

 

And screenshot file is, or download the attached file,

https://drive.google.com/file/d/0B4yhhRexQKzyV09JVTRLU054bG8/edit?usp=sharing

 

Regards,

--------------------------------------------------------------------------------------------------------------------------------

Hi everyone,

Recently have look nmon2web, nmon2rrd, and so on, they are great tools for store nmon data to rrdtool, but I still think it is not very convenient to make some graph for a period, like one week, one month, etc.

So I wrote several script to transfer nmon data to rrdtool, and then use my own web server which can create nmon graph (including CPU/POOLS/MEMORY/PAGING/PROC/AIO/Kernel/Disk/FS/Network/Filesystem/SEA) dynamically for any period (any means any day/time to any day/time), please see the attached snapshot for webpage and graphs,

I do not know if you are interested on my tools, and I can publish this if anyone think it is useful. Please have a look, and give me some opinion also welcome.

Thanks,

Updated on 2014-02-07T05:15:41Z at 2014-02-07T05:15:41Z by sun.aus
  • sun.aus
    sun.aus
    12 Posts

    Re: nmon graph web tools

    ‏2013-12-09T04:26:41Z  

    Doesnot allow me to upload all graph,

  • fbenfredj
    fbenfredj
    40 Posts

    Re: nmon graph web tools

    ‏2013-12-10T09:34:10Z  

    Hi,

    Beautiful graphes!

    I agree with you that this feature (graphes for a period) doesn't exit in nmon2web.

    But I think it is better to merge your scripts with Bruce's ones in order to have only one product to use.

    Regards

    Faouzi BEN FREDJ

  • fbenfredj
    fbenfredj
    40 Posts

    Re: nmon graph web tools

    ‏2013-12-10T09:39:45Z  

    I also created my own tool and used it for many years. but it is difficult to maintain and to find time to add more feature.

    So, I think, it is better to help Bruce to maintain nmon2web.

     

  • BruceSpencer
    BruceSpencer
    297 Posts

    Re: nmon graph web tools

    ‏2013-12-10T14:47:58Z  
    • fbenfredj
    • ‏2013-12-10T09:39:45Z

    I also created my own tool and used it for many years. but it is difficult to maintain and to find time to add more feature.

    So, I think, it is better to help Bruce to maintain nmon2web.

     

    I've started playing with this in the nmon2web tool.  It takes about 30 seconds to generate all of the charts dynamically on my P5-520 server.  How long does it take on your system? 

     

  • fbenfredj
    fbenfredj
    40 Posts

    Re: nmon graph web tools

    ‏2013-12-10T17:00:47Z  

    I've started playing with this in the nmon2web tool.  It takes about 30 seconds to generate all of the charts dynamically on my P5-520 server.  How long does it take on your system? 

     

    Sorry, Bruce, I did not understand your question. What are you talking about when you say "I've started playing with this in the nmon2web tool"?

  • BruceSpencer
    BruceSpencer
    297 Posts

    Re: nmon graph web tools

    ‏2013-12-10T18:55:01Z  
    • fbenfredj
    • ‏2013-12-10T17:00:47Z

    Sorry, Bruce, I did not understand your question. What are you talking about when you say "I've started playing with this in the nmon2web tool"?

    I meant that I'm planning to add this functionality. to nmon2web.  However, in my initial tests, it takes 30 seconds to dynamically generate the charts on the command line. I'm worried that this is unacceptably long.      

  • sun.aus
    sun.aus
    12 Posts

    Re: nmon graph web tools

    ‏2013-12-10T22:18:48Z  


    I am using the following performance monitoring tools which all running on one AIX, (can run on one LINIX after testing). After I had a look all AIX performance tools, now I choose the followings,

    1. real time nmon: nmon2graphite
       This is a most useful AIX monitoring tools, and it collect data from AIX every 30s, and real time making graphs, and I can not see any tools better than it. It can store very long time nmon data if you have storage, for example, it used 35GB for 35 days nmon data of 75 AIX hosts. This one can also make history graphs for any period.
       More information, please see the author's website.

    2. LPAR2RRD
       This is a great tools as well, and it can collect data from HMC directly every 60s, and so many function inside, and I am using free version.
       More information, please see the author's website.

    3. nmon graph
       This is my one, and I had a look nmon2web (nmon2rrd) very carefully, but nmon2web is little complicated, and I only need maybe one third of function, and most important is I need to make graph for any period.
       My tool is simple, only can make graphs for one host each time. The figure with my own tool,
       size: around 160-200MB per host to store 2 years nmon data,
       time: need around 10-12s to import one day nmon data file to rrdtool,
       graph: need around 6-7s to make over than 30 graphs for one host with any period.

    4. pGraph
       This is a great tool too, and it can make graph from nmon/sar/topas/topas_cec/HMC although the graphs are not great look, so I am using this to store all history data including HMC_Daily, HMC_Weekly, nmon_daily, nmon_weekly, topas_daily, sar, topas_cec.

     

    Updated on 2013-12-11T03:09:40Z at 2013-12-11T03:09:40Z by sun.aus
  • fbenfredj
    fbenfredj
    40 Posts

    Re: nmon graph web tools

    ‏2013-12-11T09:54:52Z  

    Hi sun.aus,

    I am, really, impressed by theses benefits :

       size: around 160-200MB per host to store 2 years nmon data,
       time: need around 10-12s to import one day nmon data file to rrdtool,
       graph: need around 6-7s to make over than 30 graphs for one host with any period.

     

    Is it possible to have the scripts to test them?

     

  • fbenfredj
    fbenfredj
    40 Posts

    Re: nmon graph web tools

    ‏2013-12-11T09:56:36Z  

    I meant that I'm planning to add this functionality. to nmon2web.  However, in my initial tests, it takes 30 seconds to dynamically generate the charts on the command line. I'm worried that this is unacceptably long.      

    Bruce,

    Yes it take me 30s to update RRD files and generate the charts, even for one hour nmon file.

    Really, it is very long.

  • sun.aus
    sun.aus
    12 Posts

    Re: nmon graph web tools

    ‏2013-12-11T11:38:45Z  
    • fbenfredj
    • ‏2013-12-11T09:54:52Z

    Hi sun.aus,

    I am, really, impressed by theses benefits :

       size: around 160-200MB per host to store 2 years nmon data,
       time: need around 10-12s to import one day nmon data file to rrdtool,
       graph: need around 6-7s to make over than 30 graphs for one host with any period.

     

    Is it possible to have the scripts to test them?

     

    Can you leave me your email? and I send you all files

    Updated on 2013-12-11T22:11:50Z at 2013-12-11T22:11:50Z by sun.aus
  • fbenfredj
    fbenfredj
    40 Posts

    Re: nmon graph web tools

    ‏2013-12-11T13:44:27Z  
    • sun.aus
    • ‏2013-12-11T11:38:45Z

    Can you leave me your email? and I send you all files

    I can not access the Google drive URL.

    Here is my email : faouzi.benfredj@laposte.net

     

  • sun.aus
    sun.aus
    12 Posts

    Re: nmon graph web tools

    ‏2013-12-11T22:11:40Z  
    • fbenfredj
    • ‏2013-12-11T13:44:27Z

    I can not access the Google drive URL.

    Here is my email : faouzi.benfredj@laposte.net

     

    Hi fbenfredj,

    send you email with all files.

    google link is working for me last night, now broken, will fix it later.

  • fbenfredj
    fbenfredj
    40 Posts

    Re: nmon graph web tools

    ‏2013-12-12T11:20:32Z  
    • sun.aus
    • ‏2013-12-11T22:11:40Z

    Hi fbenfredj,

    send you email with all files.

    google link is working for me last night, now broken, will fix it later.

    Hi Jing,

    Thanks for sending me your tool files.

    I will test them.

    But, in the README, you say :

    Only the following nmon data will be imported to rrdtool, and the other data will be discarded.
         LPAR
         CPU_ALL
         POOLS
         PROC
         PROCAIO
         MEM
         MEMNEW
         MEMUSE
         PAGE
         FILE
         NET
         FCS includes all fcs read/write and sum of,
         DISK only includes sum of disk read/write, sum of disk io read/write, not include individual disk,
         PAGING only includes sum of PagingSpaceFree,
         JFSFILE
         JFSINODE
         SEA only for VIOS nmon data,
     

     

    What about TOP processes?

    I am very interested in this graph. I use it a lot.

     

  • sun.aus
    sun.aus
    12 Posts

    Re: nmon graph web tools

    ‏2013-12-12T22:11:41Z  
    • fbenfredj
    • ‏2013-12-12T11:20:32Z

    Hi Jing,

    Thanks for sending me your tool files.

    I will test them.

    But, in the README, you say :

    Only the following nmon data will be imported to rrdtool, and the other data will be discarded.
         LPAR
         CPU_ALL
         POOLS
         PROC
         PROCAIO
         MEM
         MEMNEW
         MEMUSE
         PAGE
         FILE
         NET
         FCS includes all fcs read/write and sum of,
         DISK only includes sum of disk read/write, sum of disk io read/write, not include individual disk,
         PAGING only includes sum of PagingSpaceFree,
         JFSFILE
         JFSINODE
         SEA only for VIOS nmon data,
     

     

    What about TOP processes?

    I am very interested in this graph. I use it a lot.

     

    Hi,

    Top data from nmon will be discarded yet, because it is too dynamic, it will be very big for rrdtool. It is discarded just because I do not use it.

    Maybe next version I will add that feature in. But to be honest, top data from nmon is not suitable for rrdtool because it is not consistent.

    Thanks,

    Updated on 2013-12-12T22:12:10Z at 2013-12-12T22:12:10Z by sun.aus
  • fbenfredj
    fbenfredj
    40 Posts

    Re: nmon graph web tools

    ‏2013-12-13T10:26:15Z  
    • sun.aus
    • ‏2013-12-12T22:11:41Z

    Hi,

    Top data from nmon will be discarded yet, because it is too dynamic, it will be very big for rrdtool. It is discarded just because I do not use it.

    Maybe next version I will add that feature in. But to be honest, top data from nmon is not suitable for rrdtool because it is not consistent.

    Thanks,

    Hi Jing,

    In the nmon_news.cfg, I put  DATE_CMD=/usr/bin/date and I had these errors :

    /usr/bin/date: illegal option -- d
    Usage: date [-u] [+Field Descriptors]
    /usr/bin/date: illegal option -- d
    Usage: date [-u] [+Field Descriptors]
    /usr/bin/date: illegal option -- d
    Usage: date [-u] [+Field Descriptors]
    /usr/bin/date: illegal option -- d
    Usage: date [-u] [+Field Descriptors]
    /usr/bin/date: illegal option -- d
    Usage: date [-u] [+Field Descriptors]
    /usr/bin/date: illegal option -- d
     

    How to correct rrddata?

    Thanks

    Faouzi

  • sun.aus
    sun.aus
    12 Posts

    Re: nmon graph web tools

    ‏2013-12-13T11:16:08Z  
    • fbenfredj
    • ‏2013-12-13T10:26:15Z

    Hi Jing,

    In the nmon_news.cfg, I put  DATE_CMD=/usr/bin/date and I had these errors :

    /usr/bin/date: illegal option -- d
    Usage: date [-u] [+Field Descriptors]
    /usr/bin/date: illegal option -- d
    Usage: date [-u] [+Field Descriptors]
    /usr/bin/date: illegal option -- d
    Usage: date [-u] [+Field Descriptors]
    /usr/bin/date: illegal option -- d
    Usage: date [-u] [+Field Descriptors]
    /usr/bin/date: illegal option -- d
    Usage: date [-u] [+Field Descriptors]
    /usr/bin/date: illegal option -- d
     

    How to correct rrddata?

    Thanks

    Faouzi

    Hi,

    has to use date command from coreutils rpm, normally is /opt/freeware/bin/date, or the following command working,

    date -d 'yesterday'

    you can download coreutils from

    http://www.perzl.org/aix/index.php?n=Main.Coreutils

    Thanks,

    Regards,

    Updated on 2013-12-13T11:17:15Z at 2013-12-13T11:17:15Z by sun.aus
  • fbenfredj
    fbenfredj
    40 Posts

    Re: nmon graph web tools

    ‏2013-12-13T15:19:40Z  
    • sun.aus
    • ‏2013-12-13T11:16:08Z

    Hi,

    has to use date command from coreutils rpm, normally is /opt/freeware/bin/date, or the following command working,

    date -d 'yesterday'

    you can download coreutils from

    http://www.perzl.org/aix/index.php?n=Main.Coreutils

    Thanks,

    Regards,

    the AIX date command does not have '-d' option.

    So I used /opt/freeware/bin/date.

    No I have this error :

    The requested URL /nmon_news/tmp/<?php+echo+session_id()?> was not found on this server

    What rpm/version of PHP should I install?

    Regards

     

  • sun.aus
    sun.aus
    12 Posts

    Re: nmon graph web tools

    ‏2013-12-14T05:17:16Z  
    • fbenfredj
    • ‏2013-12-13T15:19:40Z

    the AIX date command does not have '-d' option.

    So I used /opt/freeware/bin/date.

    No I have this error :

    The requested URL /nmon_news/tmp/<?php+echo+session_id()?> was not found on this server

    What rpm/version of PHP should I install?

    Regards

     

    Hi,

    need install PHP for httpd, any version is ok.

    Only use PHP session ID function to identify multiple user using at the same time.

    Regards,

  • sun.aus
    sun.aus
    12 Posts

    Re: nmon graph web tools

    ‏2014-01-09T04:20:32Z  
    • fbenfredj
    • ‏2013-12-12T11:20:32Z

    Hi Jing,

    Thanks for sending me your tool files.

    I will test them.

    But, in the README, you say :

    Only the following nmon data will be imported to rrdtool, and the other data will be discarded.
         LPAR
         CPU_ALL
         POOLS
         PROC
         PROCAIO
         MEM
         MEMNEW
         MEMUSE
         PAGE
         FILE
         NET
         FCS includes all fcs read/write and sum of,
         DISK only includes sum of disk read/write, sum of disk io read/write, not include individual disk,
         PAGING only includes sum of PagingSpaceFree,
         JFSFILE
         JFSINODE
         SEA only for VIOS nmon data,
     

     

    What about TOP processes?

    I am very interested in this graph. I use it a lot.

     

    Hi,

    TOP graph is available now, please download from link on the top.

    Regards,

  • sun.aus
    sun.aus
    12 Posts

    Re: nmon graph web tools

    ‏2014-01-21T03:02:05Z  

    2014-01-21 New:

    Now include CPU/POOLS/MEMORY/PAGING/PROC/AIO/Kernel/Disk/FS/Network/WLM/TOP/Filesystem/SEA

    Anyone can download this on, (updated on 2014-Feb-07)

    https://drive.google.com/file/d/0B4yhhRexQKzyYndSVlFrYXZRbms/edit?usp=sharing

     

    And screenshot file is, or download the attached file,

    https://drive.google.com/file/d/0B4yhhRexQKzyV09JVTRLU054bG8/edit?usp=sharing

     

    Regards,

    Attachments

    Updated on 2014-02-07T05:15:56Z at 2014-02-07T05:15:56Z by sun.aus
  • sun.aus
    sun.aus
    12 Posts

    Re: nmon graph web tools

    ‏2015-08-24T00:28:49Z  

    Recently Update:

    1. Rewrote those ksh script to perl, and does not rely on ksh and date command any more.

    2. Now can import Linux nmon data to rrdtool, and also can make graph from Linux nmon rrd data, but linux nmon data only includes, (    CPU_ALL,       PROC,       MEM,       NET,   JFSFILE,  TOP,    DISK only includes sum of disk read/write, not include individual disk,)
     

    Testing was done on AIX with in-built AIX perl (perl.rte 5.10.1.250) and the following rpms,

    rrdtool-1.4.7-3
    httpd-2.2.24-2
    httpd-devel-2.2.24-2
    httpd-manual-2.2.24-2
    php-5.2.17-2
    php-common-5.2.17-2

     

    But need the following perl moudles, RRDp.pm and Table.pm which can be copied directly without installation.

     

    Will give the download link shortly.

     

    Thanks,

    Updated on 2015-08-24T00:32:20Z at 2015-08-24T00:32:20Z by sun.aus