IC5Notice: We have upgraded developerWorks Community to the latest version of IBM Connections. For more information, read our upgrade FAQ.
Topic
  • 5 replies
  • Latest Post - ‏2013-03-18T13:25:41Z by SystemAdmin
SystemAdmin
SystemAdmin
2402 Posts

Pinned topic explanation needed, changes in nmon data from power5/aix6 to power7/aix7

‏2013-03-06T19:42:52Z |
Hi
On older aix versions nmon showed that entitlement usage and process top10 stats were in line.
If entitlement showed usage of 4 cores at certain time, process stats showed the same amount
of usage.

Now on power7, process top10 shows much less usage compared to the entitlement. Difference is noticeable!

I would like to hear opinions where this might come from?

  • Might this be caused by overhead due bad system/lpar design - like too low processor entitlements compared to
the average cpu load? (near,far,distant...whatever...thing)?
Updated on 2013-03-18T13:25:41Z at 2013-03-18T13:25:41Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    2402 Posts

    Re: explanation needed, changes in nmon data from power5/aix6 to power7/aix7

    ‏2013-03-13T12:28:54Z  
    tprof -skex output

    Process PID TID Total Kernel User Shared Other
    ======= === === ===== ====== ==== ====== =====
    Total 100.00 76.44 7.73 15.83 0.00

    It really doesn't look good on this 8 VP machine with SMT4.

    Suggestions? Anyone?
  • Steve_ATS
    Steve_ATS
    40 Posts

    Re: explanation needed, changes in nmon data from power5/aix6 to power7/aix7

    ‏2013-03-14T18:47:15Z  
    tprof -skex output

    Process PID TID Total Kernel User Shared Other
    ======= === === ===== ====== ==== ====== =====
    Total 100.00 76.44 7.73 15.83 0.00

    It really doesn't look good on this 8 VP machine with SMT4.

    Suggestions? Anyone?
    You'll need to compare to topas. I'd guess if nmon reports a higher percentage then it is normalizing utilization to a core (ie, physc * %user) whereas topas calculates relative to physc alone.

    You may find a full answer searching the forum, I'm not sure this answers your question:
    https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14733422&#14733422

    Not sure what your tprof question is. You should be using -R with that and those are percentages relative to consumed PURR (physc) and kernel time always includes ceded idle time, which is different from CPU tools.
  • SystemAdmin
    SystemAdmin
    2402 Posts

    Re: explanation needed, changes in nmon data from power5/aix6 to power7/aix7

    ‏2013-03-15T09:30:44Z  
    • Steve_ATS
    • ‏2013-03-14T18:47:15Z
    You'll need to compare to topas. I'd guess if nmon reports a higher percentage then it is normalizing utilization to a core (ie, physc * %user) whereas topas calculates relative to physc alone.

    You may find a full answer searching the forum, I'm not sure this answers your question:
    https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14733422&#14733422

    Not sure what your tprof question is. You should be using -R with that and those are percentages relative to consumed PURR (physc) and kernel time always includes ceded idle time, which is different from CPU tools.
    http://www.ibm.com/developerworks/forums/thread.jspa?messageID=14812102

    Is it so, that now on SMT4 top processes are relative to logical cores. It is then guite natural to being lower that Physc. On power5 it logical and physical were practically the same thing, and showed values which aligned?
  • Steve_ATS
    Steve_ATS
    40 Posts

    Re: explanation needed, changes in nmon data from power5/aix6 to power7/aix7

    ‏2013-03-15T19:24:11Z  
    http://www.ibm.com/developerworks/forums/thread.jspa?messageID=14812102

    Is it so, that now on SMT4 top processes are relative to logical cores. It is then guite natural to being lower that Physc. On power5 it logical and physical were practically the same thing, and showed values which aligned?
    That link is the best description, I could not find it searching myself.

    Tracking only makes sense for single-threaded processes that can effectively only execute on one SMT thread and thus map to a single lcpu. Multi-threaded processes in POWER5/6 may have been more biased to execute on SMT pairs mapping to a core, but there's really no guarantee of that at any level.

    At some level, aggregated PURR consumption for a process could be collected and correlated to process accounting mechanisms, or provide an alternative accounting mechanism. The former is fairly hard, the latter would probably not be adopted because it would go against decades of unix history.

    POWer5/POWER6 over-reported utilization and fewer SMT threads, higher VP thresholds make it look more "correct" IMO - Can't find the original whitepaper, but contents reposted here:
    https://www.ibm.com/developerworks/mydeveloperworks/wikis/home/wiki/Not%20AIX/page/Understanding%20Processor%20Utilization%20on%20Power%20Systems%20-%20AIX?lang=en
  • SystemAdmin
    SystemAdmin
    2402 Posts

    Re: explanation needed, changes in nmon data from power5/aix6 to power7/aix7

    ‏2013-03-18T13:25:41Z  
    • Steve_ATS
    • ‏2013-03-15T19:24:11Z
    That link is the best description, I could not find it searching myself.

    Tracking only makes sense for single-threaded processes that can effectively only execute on one SMT thread and thus map to a single lcpu. Multi-threaded processes in POWER5/6 may have been more biased to execute on SMT pairs mapping to a core, but there's really no guarantee of that at any level.

    At some level, aggregated PURR consumption for a process could be collected and correlated to process accounting mechanisms, or provide an alternative accounting mechanism. The former is fairly hard, the latter would probably not be adopted because it would go against decades of unix history.

    POWer5/POWER6 over-reported utilization and fewer SMT threads, higher VP thresholds make it look more "correct" IMO - Can't find the original whitepaper, but contents reposted here:
    https://www.ibm.com/developerworks/mydeveloperworks/wikis/home/wiki/Not%20AIX/page/Understanding%20Processor%20Utilization%20on%20Power%20Systems%20-%20AIX?lang=en
    Thanks Steve, its pretty clear now!