Topic
23 replies Latest Post - ‏2013-01-02T12:46:31Z by nagger
SystemAdmin
SystemAdmin
2404 Posts
ACCEPTED ANSWER

Pinned topic nmon2rrd vworking binary with AIX 7.1 and topas_nmon

‏2012-10-02T12:43:17Z |
Attached binary should work with topas_nmon and aix 7.1.
Will attach source to next thread

Vesa
Updated on 2013-01-02T12:46:31Z at 2013-01-02T12:46:31Z by nagger
  • SystemAdmin
    SystemAdmin
    2404 Posts
    ACCEPTED ANSWER

    Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

    ‏2012-10-02T12:44:54Z  in response to SystemAdmin
    And here is the ugly source code.

    Sorry, I'm not a C coder - just desperate :P

    Vesa

    Attachments

    • SystemAdmin
      SystemAdmin
      2404 Posts
      ACCEPTED ANSWER

      Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

      ‏2012-12-31T10:16:39Z  in response to SystemAdmin
      Oops,
      Here is the C source of the 12p version of the nmon2rrd, which I have forgotten to attach. Sorry for the delay :}

      Happy New Year,
      Vesa

      Attachments

  • armink
    armink
    27 Posts
    ACCEPTED ANSWER

    Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

    ‏2012-10-19T13:58:11Z  in response to SystemAdmin
    Many thanks for making things work with AIX 7.1!

    Can you describe the differences between the earlier versions (e.g. in AIX 6.1 and 5.3)?
    Your binary works fine with AIX 7.1 but some graphs (pools, entitlement, memory, system parameters) are missing with nmon in AIX 6.1 and 5.3.
    • SystemAdmin
      SystemAdmin
      2404 Posts
      ACCEPTED ANSWER

      Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

      ‏2012-10-24T10:51:07Z  in response to armink
      Sure,
      Nmon2rrd requires nmon data to be specific and previously it detected it by using nmon version information (12h, 12a,...). So, using topas_nmon this no longer works, since version
      is always "topas_nmon". To overcome this, I added detection of the AIX version. For example, if aix version starts with 7 then use setup x for data, if aix version is 6.1.6.15 then use another
      set.

      Since I do not have all possible topas_nmon and Aix version combinations, I have not added them all and obviously it does not work with all AIX versions. There are a lot of holes
      concerning AIX 6.1 versions. I try to guess them and add them when I have more time to do it.

      If your AIX is not supporting topas_nmon, use old nagger nmon and nmon2rrd. I try to avoid using nmon (12) version to record data, when there is topas_nmon available.

      In order to add support for different data sets, I have to peek in to data what is causing the graphs to become empty. Usually its that there is a new column in the
      data. But in 7.1 there has been other issues as well, like unnecessary whitespaces in the data causing C to fail when its expecting a number instead of a character.

      Armink, could you specify AIX 6.1 version which do not work with this one and I can try to make an another compilation for you?
      • armink
        armink
        27 Posts
        ACCEPTED ANSWER

        Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

        ‏2012-10-31T17:54:59Z  in response to SystemAdmin
        I have tested things with AIX 5.3 TL12, AIX 6.1 TL6 and TL7... all with "native" topas_nmon.
        If you don't run this version I can supply you with the nmon data.
        Please tell me what data (e.g. what rows) you need and I'll post it here.

        Thanks in advance!
        Armin
        • SystemAdmin
          SystemAdmin
          2404 Posts
          ACCEPTED ANSWER

          Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

          ‏2012-11-05T10:43:59Z  in response to armink
          Send me first the 4th line of the nmon report of the both machines. That
          can be found from data as well.

          For example:
          4. AIX: 6.1.6.15
          Vesa
          • armink
            armink
            27 Posts
            ACCEPTED ANSWER

            Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

            ‏2012-11-06T12:50:50Z  in response to SystemAdmin
            What forth line do you mean? In the nmon dump file or in the resulting Excel sheet or anywhere else?
            In the nmon dump it's the 8th line and is like "AAA,AIX," and then
            "5.3.12.5" 5.3 TL12 SP5
            "6.1.6.15" 6.1 TL6 SP5
            "6.1.7.2" 6.1 TL07 SP2
            "6.1.7.3" 6.1 TL07 SP3
            "6.1.7.15" 6.1 TL07 SP4
            "6.1.7.16" 6.1 TL07 SP5

            AAA,version is always TOPAS-NMON
            • SystemAdmin
              SystemAdmin
              2404 Posts
              ACCEPTED ANSWER

              Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

              ‏2012-11-06T12:56:02Z  in response to armink
              Yes!
              That is exactly what I was looking for...

              Now i can proceed. BTW, which SP of 6.1 TL7 is equivalent for 6.1 TL6 SP5???
              • armink
                armink
                27 Posts
                ACCEPTED ANSWER

                Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

                ‏2012-11-06T13:50:15Z  in response to SystemAdmin
                6.1 TL6 SP5 is 6100-06-05-1115
                6.1 TL7 SP0 is 6100-07-00-1140
                6.1 TL7 SP1 is 6100-07-01-1141

                so for me it looks like there is no equivalent.
                • SystemAdmin
                  SystemAdmin
                  2404 Posts
                  ACCEPTED ANSWER

                  Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

                  ‏2012-11-06T21:32:42Z  in response to armink
                  Yep, they seem to be about 25 weeks newer - which makes this a bit easier I hope.
            • SystemAdmin
              SystemAdmin
              2404 Posts
              ACCEPTED ANSWER

              Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

              ‏2012-11-06T22:09:07Z  in response to armink
              Attached binary worked with my 6.1.7.2 and 6.1.7.16 when i tested it. Hope this helps?
              (Luckily there were installed while I was abscent for a month :D )

              There is another change as well, binary supports now up to 1500 snapshots instead of 1024.
              So you can gather daily data with exatch 60 second interval which brings 1440 samples
              on a normal day and 1500 when daylight saving times occur.

              Armink, could you send 5.3.12.5 data for inspection. You can alter sensitive data
              manually if you like. 10 snaphots is enough. Or just paste it here?
              • SystemAdmin
                SystemAdmin
                2404 Posts
                ACCEPTED ANSWER

                Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

                ‏2012-11-06T23:40:49Z  in response to SystemAdmin
                Armin, how does the 5.3.12.5 first line of PROC look like in the data?

                PROC,Processes lpar,Runnable,Swap-in,pswitch,syscall,read,write,fork,exec,sem,msg,asleep_bufio,asleep_rawio,asleep_diocio

                or

                PROC,Processes lpar,Runnable,Swap-in,pswitch,syscall,read,write,fork,exec,sem,msg
                • SystemAdmin
                  SystemAdmin
                  2404 Posts
                  ACCEPTED ANSWER

                  Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

                  ‏2012-11-06T23:50:28Z  in response to SystemAdmin
                  And Armin, 5.3.12.5 MEMUSE, is it
                  MEMUSE,Memory Use lparx,%numperm,%minperm,%maxperm,minfree,maxfree,%numclient,%maxclient

                  or

                  MEMUSE,Memory Use lpary,%numperm,%minperm,%maxperm,minfree,maxfree,%numclient,%maxclient, lruable pages

                  PS. Here you can see a little mistake in the data by the coders,
                  the whitespace before lruable pages, which is in the actual data too

                  MEMUSE,T1023,18.3,3.0,90.0,960,1088,18.3,90.0, 4052736.0

                  PS2. I have a new version waiting for your responses, Armin ;-)
                  • armink
                    armink
                    27 Posts
                    ACCEPTED ANSWER

                    Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

                    ‏2012-11-08T15:17:08Z  in response to SystemAdmin
                    In AIX 5.3 TL12 SP5 (topas_nmon) it's
                    PROC,Processes lparx,Runnable,Swap-in,pswitch,syscall,read,write,fork,exec,sem,msg,asleep_bufio,asleep_rawio,asleep_diocio
                    and
                    MEMUSE,Memory Use lparx,%numperm,%minperm,%maxperm,minfree,maxfree,%numclient,%maxclient

                    Since the value "lruable pages" is missing, there is no white space problem either.

                    I'll check the new version and post my feedback.

                    Thanks a lot for your assistance!
                    • armink
                      armink
                      27 Posts
                      ACCEPTED ANSWER

                      Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

                      ‏2012-11-08T17:52:47Z  in response to armink
                      There seems to be a difference in the nmon output (more columns!) if nmon runs with AME enabled.
                      This is probably the reason why we don't get certain graphs.

                      Here is an example for AIX 7.1 (AME on 6.1 is the same):
                      MEM,Memory lparx,Real Free %,Virtual free %,Real free(MB),Virtual free(MB),Real total(MB),Virtual total(MB),Size of the Compressed pool(MB), Size of true memory(MB), Expanded memory size(MB), Size of the Uncompressed pool(MB)
                      MEMNEW,Memory New lparx,Process%,FScache%,System%,Free%,Pinned%,User%,Compressed Pool%
                      MEMUSE,Memory Use lparx,%numperm,%minperm,%maxperm,minfree,maxfree,%numclient,%maxclient, lruable pages
                      PAGE,Paging lparx,faults,pgin,pgout,pgsin,pgsout,reclaims,scans,cycles,Compressed pool pgins, Compressed pool pgouts
                      PROC,Processes lparx,Runnable,Swap-in,pswitch,syscall,read,write,fork,exec,sem,msg,asleep_bufio,asleep_rawio,asleep_diocio

                      Notice that there are many leading whitespaces... this sucks :-)

                      There are still (other) problems with certain TLs of 5.3 and 6.1 TL6 on lpars without AME. I'll post more if I find a pattern.

                      Regards,
                      Armin
                      • SystemAdmin
                        SystemAdmin
                        2404 Posts
                        ACCEPTED ANSWER

                        Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

                        ‏2012-11-09T07:42:58Z  in response to armink
                        This AME will cause lots of head scratching :P

                        To MEM, MEMNEW and PAGE AME has an influence. We need a extra detection for AME to the nmon2rrd...
                        MEMUSE and PROC are as usual.

                        I would be kind of easy thing to do, if AME had been mentioned in the AAA fields.
                        I checked our P7 aix 7.1 data where AME is off and there nothing that refers to it.
                        • SystemAdmin
                          SystemAdmin
                          2404 Posts
                          ACCEPTED ANSWER

                          Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

                          ‏2012-11-09T13:47:34Z  in response to SystemAdmin
                          AME with MEM and MEMNEW resolved, PAGE left to the next week to be coded.
                          Have a nice weekend...
                          • SystemAdmin
                            SystemAdmin
                            2404 Posts
                            ACCEPTED ANSWER

                            Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

                            ‏2012-11-12T07:20:13Z  in response to SystemAdmin
                            Good morning
                            Here is attached latest AME aware 12p version of the nmon2rrd. Now Paging and Memory graphs shouldn't be
                            empty when AME is on.

                            Armin could you test it?

                            I will test it on these AIX versions where AME is off:
                            5.3.0.56
                            5.3.11.3
                            5.3.8.3
                            6.1.4.7
                            6.1.6.15
                            6.1.6.3
                            6.1.7.16
                            6.1.7.2

                            And AME on:
                            7.1.1.16
                            • SystemAdmin
                              SystemAdmin
                              2404 Posts
                              ACCEPTED ANSWER

                              Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

                              ‏2012-11-15T21:29:54Z  in response to SystemAdmin
                              Now tested and works like charm...

                              Armin, when you are back, please test it and report if its any better now?

                              R Vesa
                    • SystemAdmin
                      SystemAdmin
                      2404 Posts
                      ACCEPTED ANSWER

                      Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

                      ‏2012-11-09T07:05:14Z  in response to armink
                      OK, I will add detection for the 5.3.12.5.

                      Extra spaces in the data have been resolved by translating the spaces to zero's
                      ...0,1,2, 123 --> ...0,1,2,0123
                    • SystemAdmin
                      SystemAdmin
                      2404 Posts
                      ACCEPTED ANSWER

                      Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

                      ‏2012-11-09T07:55:18Z  in response to armink
                      One additional question regarding 5.3.12.5, is there "Folded" column in the LPAR field?

                      LPAR,Logical Partition lparx,PhysicalCPU,virtualCPUs,logicalCPUs,poolCPUs,entitled,weight,PoolIdle,usedAllCPU%,usedPoolCPU%,Sh
                      aredCPU,Capped,EC_User%,EC_Sys%,EC_Wait%,EC_Idle%,VP_User%,VP_Sys%,VP_Wait%,VP_Idle%,Folded,Pool_id
                    • SystemAdmin
                      SystemAdmin
                      2404 Posts
                      ACCEPTED ANSWER

                      Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

                      ‏2012-11-09T08:41:09Z  in response to armink
                      Attached version 12o where is 5.3.12.5 detection and it expects that there is the Folded field in the LPAR data.
                      Please, test it Armin.
                      • nagger
                        nagger
                        1594 Posts
                        ACCEPTED ANSWER

                        Re: nmon2rrd vworking binary with AIX 7.1 and topas_nmon

                        ‏2013-01-02T12:46:31Z  in response to SystemAdmin
                        Hi,
                        I have just released nmon2rrd version 16a and you can find it at
                        https://www.ibm.com/developerworks/wikis/display/WikiPtype/nmon
                        • This version 16 has drastic changes which extracts column header names from the nmon file which makes it less sensitive to new columns and removed loads of code working around the variations in file output over the years.
                        • It has also been beautified with the AIX cb -s command to fix indentation mistakes
                        • Change comments (peoples names and company names) removed including mine - this is a sample code
                        • This needs to be done to remove ownership issues in the future and gets messy.

                        I really appreciate the fixes supplied and have tried to include them but this version needs a good regression test.
                        • Note the MEM section columns now match the nmon output file. Does this cause a problem with older .rrd files that you have been adding to for a longer period?
                        • please let me know either way - as we could add a work around.

                        Can anyone testing the new version and finding a failure please report the issue and the problem file so I can reproduce the issue? but NOT reported here 22 replies down the list is hard to find - Can we have a new Forum Entry would be good with say "nmon2rrd v16 problem" in the title.

                        Thanks Nigel Griffiths