Topic
  • 17 replies
  • Latest Post - ‏2014-01-09T17:32:10Z by fbenfredj
BruceSpencer
BruceSpencer
297 Posts

Pinned topic New Version of nmon2web

‏2013-12-22T14:54:05Z |

The December 2013 release of nmon2web is available at

http://ca.astound.net/~baspence/AIXtip/nmon2web.htm

 

The new version adds the following functionality

  • Create graphs for any period **
  • "nmon2web consolidator" - show "what if" performance for any group of partitions
  • Added dedicated LPARs to server performance (previously just the shared pool)
  • setup.ksh script - automate the installation/upgrade

Both new and existing users should use the new setup.ksh script.  Existing nmon2web installations need additional directories and permissions for the new functionality.

** Note:  "creating graphs for any period" is slow (20+seconds). The problem is the number of charts that have to be created.

 

 

  • fbenfredj
    fbenfredj
    40 Posts

    Re: New Version of nmon2web

    ‏2014-01-02T11:22:37Z  

    Hi Bruce,

    During upgrade from v2.0.6 to v2.1.0 I got theses 2 errors :

    error 1:

    Can't open upgrade.ksh: No such file or directory, <> line 3518.

    Caused by theses lignes

    perl -p -i -e "s?BBBBBB?$DOCUMENTROOT?" nmon2web.pl nmon2web*.cgi upgrade.ksh
    perl -p -i -e "s?CCCCCC?$WEB_SUBDIR?" nmon2web.pl nmon2web*.cgi upgrade.ksh

    error 2:

    cp: aixtipshead.gif: No such file or directory

    The gif file does not exist in the new version.

     

    I think that theses errors have no impact on the upgrade process (upgrade.ksh is not used).

     

    But I have other problems that impact production :

    error 3 :

    chown: index.html: Not owner

    chown: readme.html: Not owner

    chown: nmon2web.cgi: Not owner

    chown: nmon2web_consolidator.cgi: Not owner
    chown: printenv.cgi: Not owner

     

    Error 4 :

    When I want to have trend charts :

    You don't have permission to access /cgi-bin/nmon2web.cgi on this server.

    In fact, in the index.html you put theses lines :

    <form name="trend" action="/cgi-bin/nmon2web.cgi" target="_blank">
    <form name="aggregate" action="/cgi-bin/nmon2web.cgi" target="_blank">
    <form name="consolidate" action="/cgi-bin/nmon2web_consolidator.cgi">

     

    but in my http.conf I had this line :

    ScriptAlias /nmon2web-cgi/ "/usr2/produits/apache2222/cgi-bin/nmon2web/"

    So I have to change the ScriptAlias to/cgi-bin/

    Error 5 :

    In the index.html there is a problem with the choice of the "day" in "Daily Host Performance Chart" :

    it is marked "Trend" instead of the list of day numbers.

     

    Regards

    Faouzi

  • fbenfredj
    fbenfredj
    40 Posts

    Re: New Version of nmon2web

    ‏2014-01-02T17:36:16Z  

    Bruce,

     

    I have another problem : all the day lists in all the form are empty or has only the value "1".

    So I can not select the day of the chart date or startdate or enddate.

     

     

  • BruceSpencer
    BruceSpencer
    297 Posts

    Re: New Version of nmon2web

    ‏2014-01-02T18:08:10Z  
    • fbenfredj
    • ‏2014-01-02T11:22:37Z

    Hi Bruce,

    During upgrade from v2.0.6 to v2.1.0 I got theses 2 errors :

    error 1:

    Can't open upgrade.ksh: No such file or directory, <> line 3518.

    Caused by theses lignes

    perl -p -i -e "s?BBBBBB?$DOCUMENTROOT?" nmon2web.pl nmon2web*.cgi upgrade.ksh
    perl -p -i -e "s?CCCCCC?$WEB_SUBDIR?" nmon2web.pl nmon2web*.cgi upgrade.ksh

    error 2:

    cp: aixtipshead.gif: No such file or directory

    The gif file does not exist in the new version.

     

    I think that theses errors have no impact on the upgrade process (upgrade.ksh is not used).

     

    But I have other problems that impact production :

    error 3 :

    chown: index.html: Not owner

    chown: readme.html: Not owner

    chown: nmon2web.cgi: Not owner

    chown: nmon2web_consolidator.cgi: Not owner
    chown: printenv.cgi: Not owner

     

    Error 4 :

    When I want to have trend charts :

    You don't have permission to access /cgi-bin/nmon2web.cgi on this server.

    In fact, in the index.html you put theses lines :

    <form name="trend" action="/cgi-bin/nmon2web.cgi" target="_blank">
    <form name="aggregate" action="/cgi-bin/nmon2web.cgi" target="_blank">
    <form name="consolidate" action="/cgi-bin/nmon2web_consolidator.cgi">

     

    but in my http.conf I had this line :

    ScriptAlias /nmon2web-cgi/ "/usr2/produits/apache2222/cgi-bin/nmon2web/"

    So I have to change the ScriptAlias to/cgi-bin/

    Error 5 :

    In the index.html there is a problem with the choice of the "day" in "Daily Host Performance Chart" :

    it is marked "Trend" instead of the list of day numbers.

     

    Regards

    Faouzi

    I see it. I'll work on it later today. 

     

     

     

  • BruceSpencer
    BruceSpencer
    297 Posts

    Re: New Version of nmon2web

    ‏2014-01-03T01:37:55Z  
    • fbenfredj
    • ‏2014-01-02T17:36:16Z

    Bruce,

     

    I have another problem : all the day lists in all the form are empty or has only the value "1".

    So I can not select the day of the chart date or startdate or enddate.

     

     

    I've fixed the errors in nmon2web_20140102.tar.gz  and uploaded it to my web server. 

    Here's what went wrong:

    Error1:   The upgrade.ksh file was used in testing, and shouldn't have been included in setup.ksh.  

    Error2:   The gif file was left out of the tar file by accident. 

    Error3:   Can't assign ownership of a file, unless you're root.  So I changed "chown" to "chgrp".  The purpose is to allow the web process rwx.

    Error4:   This is a customization issue.  In the index.html file, change the three occurrences of "cgi-bin" to "nmon2web-cgi".  Also change the CGI-BIN variable in setup.ksh.. 

    Error5:   This is a "January" bug in the index.html file.  I set the Start_month = end_month -1.  Works great in December, but not January. 

     

  • fbenfredj
    fbenfredj
    40 Posts

    Re: New Version of nmon2web

    ‏2014-01-03T09:13:39Z  

    I've fixed the errors in nmon2web_20140102.tar.gz  and uploaded it to my web server. 

    Here's what went wrong:

    Error1:   The upgrade.ksh file was used in testing, and shouldn't have been included in setup.ksh.  

    Error2:   The gif file was left out of the tar file by accident. 

    Error3:   Can't assign ownership of a file, unless you're root.  So I changed "chown" to "chgrp".  The purpose is to allow the web process rwx.

    Error4:   This is a customization issue.  In the index.html file, change the three occurrences of "cgi-bin" to "nmon2web-cgi".  Also change the CGI-BIN variable in setup.ksh.. 

    Error5:   This is a "January" bug in the index.html file.  I set the Start_month = end_month -1.  Works great in December, but not January. 

     

    Hi Bruce,

    I can not see the new version in your web server http://ca.astound.net/~baspence/AIXtip/nmon2web.htm

     

  • BruceSpencer
    BruceSpencer
    297 Posts

    Re: New Version of nmon2web

    ‏2014-01-03T15:50:39Z  
    • fbenfredj
    • ‏2014-01-03T09:13:39Z

    Hi Bruce,

    I can not see the new version in your web server http://ca.astound.net/~baspence/AIXtip/nmon2web.htm

     

    Try it now. 

  • fbenfredj
    fbenfredj
    40 Posts

    Re: New Version of nmon2web

    ‏2014-01-03T16:25:28Z  

    Try it now. 

    Now is OK.

    Thanks!

    In the setup.ksh, you missed to change one "chown" command : chown  $WEBGROUPID *.cgi

     

  • fbenfredj
    fbenfredj
    40 Posts

    Re: New Version of nmon2web

    ‏2014-01-03T17:11:25Z  
    • fbenfredj
    • ‏2014-01-03T16:25:28Z

    Now is OK.

    Thanks!

    In the setup.ksh, you missed to change one "chown" command : chown  $WEBGROUPID *.cgi

     

    I have a problem with the charts for any period. It can not find *.gif

     

     

  • BruceSpencer
    BruceSpencer
    297 Posts

    Re: New Version of nmon2web

    ‏2014-01-04T14:24:04Z  
    • fbenfredj
    • ‏2014-01-03T17:11:25Z

    I have a problem with the charts for any period. It can not find *.gif

     

     

    This could be caused by a number of things.  Here's the checklist of things to check.

    1.  Does the web server error_log file show anything after attempting to run the trend charts?

    The trend charts reside in the following directory

    $DOCUMENTROOT/$SUBDIRECTORY/$SerialNumber/[Shared|Dedicated]/$hostname/trend

    2.  Does the trend directory have rwx permission for all?

    3.  Does the trend directory contain gif files (after requesting charts?).

    If yes, check the <img> tags for the gif files in index.html.  It needs to point to the web full path

    <img src=/$SUBDIRECTORY/$SerialNumber/[Shared|Dedicated]/$hostname/trend/file.gif>

    <img src=/nmon2web/021066C4P/Shared/web1/trend/cpu_abs.gif alt=cpu_abs.gif>

    4. If the trend directory does not contain gif files, check the permissions on the rrd_graph_cgi. (Should be readable by the web server process)

    5. Can you run the rrd_graph_cgi file from the command line? 

    rrdtool - < rrd_graph_cgi

    Did it produce the gif files?  Were there any error messages?

     

    Let me know what you find.  Also send the index* and rrd* files from the trend directory.

     

  • fbenfredj
    fbenfredj
    40 Posts

    Re: New Version of nmon2web

    ‏2014-01-06T17:35:28Z  

    This could be caused by a number of things.  Here's the checklist of things to check.

    1.  Does the web server error_log file show anything after attempting to run the trend charts?

    The trend charts reside in the following directory

    $DOCUMENTROOT/$SUBDIRECTORY/$SerialNumber/[Shared|Dedicated]/$hostname/trend

    2.  Does the trend directory have rwx permission for all?

    3.  Does the trend directory contain gif files (after requesting charts?).

    If yes, check the <img> tags for the gif files in index.html.  It needs to point to the web full path

    <img src=/$SUBDIRECTORY/$SerialNumber/[Shared|Dedicated]/$hostname/trend/file.gif>

    <img src=/nmon2web/021066C4P/Shared/web1/trend/cpu_abs.gif alt=cpu_abs.gif>

    4. If the trend directory does not contain gif files, check the permissions on the rrd_graph_cgi. (Should be readable by the web server process)

    5. Can you run the rrd_graph_cgi file from the command line? 

    rrdtool - < rrd_graph_cgi

    Did it produce the gif files?  Were there any error messages?

     

    Let me know what you find.  Also send the index* and rrd* files from the trend directory.

     

    1. No error in web server error.log
    2. yes, the trend directory have rwx permission for all
    3. if I use "Specify (Month:Day:Year:Hour)"  there are no gif files in the trend directory after requesting charts. But, if I use for example, Last 7 Days, gif files exist.
    4. rrd_graph_cgi is owned by web server user.
    5. in the case I choose "Specify (Month:Day:Year:Hour)" : I have this error ERROR: start time: unparsable trailing text: '...07:00'

    You will find attached index* and rrd* files from the trend directory

    Attachments

  • BruceSpencer
    BruceSpencer
    297 Posts

    Re: New Version of nmon2web

    ‏2014-01-07T05:33:56Z  
    • fbenfredj
    • ‏2014-01-06T17:35:28Z
    1. No error in web server error.log
    2. yes, the trend directory have rwx permission for all
    3. if I use "Specify (Month:Day:Year:Hour)"  there are no gif files in the trend directory after requesting charts. But, if I use for example, Last 7 Days, gif files exist.
    4. rrd_graph_cgi is owned by web server user.
    5. in the case I choose "Specify (Month:Day:Year:Hour)" : I have this error ERROR: start time: unparsable trailing text: '...07:00'

    You will find attached index* and rrd* files from the trend directory

    I am unable to reproduce the problem in item #5.  I suspect this is an rrdtool error associated with the "--start" parameter in the rrd_graph_cgi file. 

    To verify, can we rerun #5, and send the resulting rrd_graph_cgi file?    (actually the first line in the file is all I need) I'm looking for the --start  and --end parameters.  Here's an example.

    graph cpu_all.gif -r -l 0 -u 100 -v "Percent Stacked" --start "05/21/2013 00:00" --end  "05/24/2013 07:00" ....

    This command ran successfully on my server (rrdtool v 1.4.7)

     

    PS If my theory is correct, you can reproduce the error manually by running this this command

    rrdtool - < rrd_graph_cgi

     

     

     

  • fbenfredj
    fbenfredj
    40 Posts

    Re: New Version of nmon2web

    ‏2014-01-07T11:43:13Z  

    I am unable to reproduce the problem in item #5.  I suspect this is an rrdtool error associated with the "--start" parameter in the rrd_graph_cgi file. 

    To verify, can we rerun #5, and send the resulting rrd_graph_cgi file?    (actually the first line in the file is all I need) I'm looking for the --start  and --end parameters.  Here's an example.

    graph cpu_all.gif -r -l 0 -u 100 -v "Percent Stacked" --start "05/21/2013 00:00" --end  "05/24/2013 07:00" ....

    This command ran successfully on my server (rrdtool v 1.4.7)

     

    PS If my theory is correct, you can reproduce the error manually by running this this command

    rrdtool - < rrd_graph_cgi

     

     

     

     
    I found the exact of the problem!

    The problem is caused by this line in the setup.ksh : WEB_SUBDIR="/"

    WEB_SUBDIR should empty when URL is http://abc.com/index.html and not "/".

    In setup.ksh you put theses lines :

    # URL subdirectory for nmon2web index.html
    # If browser URL is http://abc.com/index.html
    #       then WEB_SUBDIR="/"
    # If browser URL is http://abc.com/nmon2web/index.html
    #       then WEB_SUBDIR="/nmon2web"
    # WEB_SUBDIR="/nmon2web"

     

    They should be changed as in nmon2web.pl :

    # WEB_SUBDIR is the URL subdirectory for the nmon2web files.
    # URL:  http://abc.com/$WEB_SUBDIR/index.html
    # $WEB_SUBDIR='';  # no subdirectory.  http://abc.com/index.html
    # $WEB_SUBDIR='/nmon2web';  # http://abc.com/nmon2web/index.html

     

     

     

  • fbenfredj
    fbenfredj
    40 Posts

    Re: New Version of nmon2web

    ‏2014-01-07T16:45:16Z  
    • fbenfredj
    • ‏2014-01-07T11:43:13Z
     
    I found the exact of the problem!

    The problem is caused by this line in the setup.ksh : WEB_SUBDIR="/"

    WEB_SUBDIR should empty when URL is http://abc.com/index.html and not "/".

    In setup.ksh you put theses lines :

    # URL subdirectory for nmon2web index.html
    # If browser URL is http://abc.com/index.html
    #       then WEB_SUBDIR="/"
    # If browser URL is http://abc.com/nmon2web/index.html
    #       then WEB_SUBDIR="/nmon2web"
    # WEB_SUBDIR="/nmon2web"

     

    They should be changed as in nmon2web.pl :

    # WEB_SUBDIR is the URL subdirectory for the nmon2web files.
    # URL:  http://abc.com/$WEB_SUBDIR/index.html
    # $WEB_SUBDIR='';  # no subdirectory.  http://abc.com/index.html
    # $WEB_SUBDIR='/nmon2web';  # http://abc.com/nmon2web/index.html

     

     

     

    But I have always the ERROR: start time: unparsable trailing text: '...00:00', When I use "Specify (Month:Day:Year:Hour)".

    There is an example of rrdtool graph command :

    graph cpu_all.gif -r -l 0 -u 100 -v "Percent Stacked" --start "12/20/2013 00:00" --end  "01/07/2014 16:00" --width 800 --height 300 --title "Over all CPU Utilization val-unx-web002 ((12/20/2013 00:00 - 01/07/2014 16:00))" DEF:User=../rrddata/cpu_all.rrd:UserPct:AVERAGE AREA:User#0000FF:"User%" DEF:Sys=../rrddata/cpu_all.rrd:SysPct:AVERAGE STACK:Sys#00FF00:"Sys%" DEF:Wait=../rrddata/cpu_all.rrd:WaitPct:AVERAGE STACK:Wait#FF0000:"Wait%"
     


     

  • BruceSpencer
    BruceSpencer
    297 Posts

    Re: New Version of nmon2web

    ‏2014-01-07T19:31:22Z  
    • fbenfredj
    • ‏2014-01-07T16:45:16Z

    But I have always the ERROR: start time: unparsable trailing text: '...00:00', When I use "Specify (Month:Day:Year:Hour)".

    There is an example of rrdtool graph command :

    graph cpu_all.gif -r -l 0 -u 100 -v "Percent Stacked" --start "12/20/2013 00:00" --end  "01/07/2014 16:00" --width 800 --height 300 --title "Over all CPU Utilization val-unx-web002 ((12/20/2013 00:00 - 01/07/2014 16:00))" DEF:User=../rrddata/cpu_all.rrd:UserPct:AVERAGE AREA:User#0000FF:"User%" DEF:Sys=../rrddata/cpu_all.rrd:SysPct:AVERAGE STACK:Sys#00FF00:"Sys%" DEF:Wait=../rrddata/cpu_all.rrd:WaitPct:AVERAGE STACK:Wait#FF0000:"Wait%"
     


     

    So if I understand correctly,  the charts are now being created when you specify the dates, but the web server reports the "unparsable" error.   The cgi command (graph cpu_all.gif ...) syntax is correct, but we're still getting the error.

    If so, I'll need your help to debug.  I can't reproduce the error.  It's working on my server.  

    The error message is related to the rrdtool.  It doesn't like the start time we provided.   

    Can you try different start times to see it it helps. Copy the first line out of the rrd_graph_cgi file, and change the times.  For example

    --start "12/20/2013 00:01"

    --start "12/20/2013 midnight"

    --start "12/2/2013 00:01 AM"

    or remove the quotes

    --start 12/20/2013 00:00

    As I mentioned, you can copy the first line in the rrd_graph_cgi to another file, and test the changes by running the rrdtool manually  (rrdtool - <  test_file)

     

    Here's a list of the some of the time options for rrdtool

    http://oss.oetiker.ch/rrdtool/doc/rrdfetch.en.html

     

    Let me know how it goes.

     

    PS Were you using the old setup.ksh script?  The current script removes the double slashes from the fullpath name. See the 3rd line in the "Main" section of the script.

    # Filesystem full path to web server directory
    WEB_FULLPATH="$DOCUMENTROOT/${WEB_SUBDIR#\/}" # remove the initial slash (/)
     

     

     

  • fbenfredj
    fbenfredj
    40 Posts

    Re: New Version of nmon2web

    ‏2014-01-08T17:54:43Z  

    So if I understand correctly,  the charts are now being created when you specify the dates, but the web server reports the "unparsable" error.   The cgi command (graph cpu_all.gif ...) syntax is correct, but we're still getting the error.

    If so, I'll need your help to debug.  I can't reproduce the error.  It's working on my server.  

    The error message is related to the rrdtool.  It doesn't like the start time we provided.   

    Can you try different start times to see it it helps. Copy the first line out of the rrd_graph_cgi file, and change the times.  For example

    --start "12/20/2013 00:01"

    --start "12/20/2013 midnight"

    --start "12/2/2013 00:01 AM"

    or remove the quotes

    --start 12/20/2013 00:00

    As I mentioned, you can copy the first line in the rrd_graph_cgi to another file, and test the changes by running the rrdtool manually  (rrdtool - <  test_file)

     

    Here's a list of the some of the time options for rrdtool

    http://oss.oetiker.ch/rrdtool/doc/rrdfetch.en.html

     

    Let me know how it goes.

     

    PS Were you using the old setup.ksh script?  The current script removes the double slashes from the fullpath name. See the 3rd line in the "Main" section of the script.

    # Filesystem full path to web server directory
    WEB_FULLPATH="$DOCUMENTROOT/${WEB_SUBDIR#\/}" # remove the initial slash (/)
     

     

     

    No, the charts are not created when I specify "start date" and "end date" with "time-of-day" ==> because rrdtool report "unparsable" error.

    And theses cases the charts are created and correctly viewed :

    Today
    Last 2 Days
    Last 3 Days
    Last 7 Days
    Month
    Quarter
    6 Months
    Year

     

    I tried different starttimes with time-of-day and no one function. rrdtool always report "unparsable" error.

    I think that I have to update rrdtool RPM. I use the 1.2 version.

    NMON2WEB is installed on the same lpar as "LPAR2RRD" which is delivered with rrdtool 1.2.

     

  • fbenfredj
    fbenfredj
    40 Posts

    Re: New Version of nmon2web

    ‏2014-01-08T18:09:59Z  
    • fbenfredj
    • ‏2014-01-08T17:54:43Z

    No, the charts are not created when I specify "start date" and "end date" with "time-of-day" ==> because rrdtool report "unparsable" error.

    And theses cases the charts are created and correctly viewed :

    Today
    Last 2 Days
    Last 3 Days
    Last 7 Days
    Month
    Quarter
    6 Months
    Year

     

    I tried different starttimes with time-of-day and no one function. rrdtool always report "unparsable" error.

    I think that I have to update rrdtool RPM. I use the 1.2 version.

    NMON2WEB is installed on the same lpar as "LPAR2RRD" which is delivered with rrdtool 1.2.

     

    To answer your question :

    PS Were you using the old setup.ksh script?  The current script removes the double slashes from the fullpath name. See the 3rd line in the "Main" section of the script.

    # Filesystem full path to web server directory
    WEB_FULLPATH="$DOCUMENTROOT/${WEB_SUBDIR#\/}" # remove the initial slash (/)

    No, I were using new setup.ksh (version 2.1.1).

    But the problem is with the creation of index.html in trend directory. The index.html is created by nmon2web.pl by copyning daily index.html and then changing it with this command :

    
    $html[$i] =~ s/img src=/img src=$targetdir\//; 
    
    with 
    $targetdir=join('/',$WEB_SUBDIR,$SN,$TYPE,$HOSTNAME,'trend'); 
    ==> "img src=//..."
    

    The WEB_SUBDIR in nmon2web.pl is initialized with setup.ksh script

    
    perl -p -i -e "s?CCCCCC?$WEB_SUBDIR?" nmon2web.pl nmon2web*.cgi
    

     

  • fbenfredj
    fbenfredj
    40 Posts

    Re: New Version of nmon2web

    ‏2014-01-09T17:32:10Z  
    • fbenfredj
    • ‏2014-01-08T18:09:59Z

    To answer your question :

    PS Were you using the old setup.ksh script?  The current script removes the double slashes from the fullpath name. See the 3rd line in the "Main" section of the script.

    # Filesystem full path to web server directory
    WEB_FULLPATH="$DOCUMENTROOT/${WEB_SUBDIR#\/}" # remove the initial slash (/)

    No, I were using new setup.ksh (version 2.1.1).

    But the problem is with the creation of index.html in trend directory. The index.html is created by nmon2web.pl by copyning daily index.html and then changing it with this command :

    <pre dir="ltr"> $html[$i] =~ s/img src=/img src=$targetdir\//; with $targetdir=join('/',$WEB_SUBDIR,$SN,$TYPE,$HOSTNAME,'trend'); ==> "img src=//..." </pre>

    The WEB_SUBDIR in nmon2web.pl is initialized with setup.ksh script

    <pre dir="ltr"> perl -p -i -e "s?CCCCCC?$WEB_SUBDIR?" nmon2web.pl nmon2web*.cgi </pre>

     

    Hi Bruce,

    I found a solution to my problem of AT-style date-time : we have to use the perl command "str2time" which change AT-style date-time to EPOCH time.

    So I changed nmon2web.cgi script by adding theses commands :

    • at the start of the script :
    
    use Date::Parse;
    
    • In the host_trend function, after these lines :
    
    if ( $form_data{'endhour'} > 23 ) {
             $end .= " 23:59";
     } else {
             $end .= " " . $form_data{'endhour'} .":00";
     }
    
    

    I added these 2 lines :

    
           $start=str2time ($start);
            $end=str2time ($end);
    

    NOW, I HAVE MY CHARTS FOR ALL OPTIONS OF "Host Performance Trend"!!!! :-))

     

    Is it possible to have a new version of NMON2WEB that use str2time function?

    Thanks