IC5Notice: We have upgraded developerWorks Community to the latest version of IBM Connections. For more information, read our upgrade FAQ.
Topic
  • 10 replies
  • Latest Post - ‏2013-07-25T01:39:05Z by nasica88
nasica88
nasica88
24 Posts

Pinned topic nmon in a shared processor partition

‏2013-07-15T11:08:43Z |

nmon displays the CPU usage of a shared processor partition with an entitlement of 3.00 and 10 virtual CPUs, as using 0.65 core of physical core, when there is one process running consuming 100% of a single core.   SMT was turned off completely.

Is this normal ? 

Updated on 2013-07-15T11:11:03Z at 2013-07-15T11:11:03Z by nasica88
  • willschm
    willschm
    45 Posts

    Re: nmon in a shared processor partition

    ‏2013-07-15T22:14:31Z  

    Are you able to use 'top' or 'ps'  (or anything really), to see what it believes is running and consuming that cpu?

  • nasica88
    nasica88
    24 Posts

    Re: nmon in a shared processor partition

    ‏2013-07-15T22:58:12Z  
    • willschm
    • ‏2013-07-15T22:14:31Z

    Are you able to use 'top' or 'ps'  (or anything really), to see what it believes is running and consuming that cpu?

    Sure. 

    By the way, I'd be obliged if somebody can show me how to correct that incorrect information of "64 CPUs" at the head of sar display.

     

    Attachments

  • Bill_Buros
    Bill_Buros
    151 Posts

    Re: nmon in a shared processor partition

    ‏2013-07-16T13:06:53Z  
    • nasica88
    • ‏2013-07-15T22:58:12Z

    Sure. 

    By the way, I'd be obliged if somebody can show me how to correct that incorrect information of "64 CPUs" at the head of sar display.

     

    wrt "64 CPU"..    how many cores/CPUs are active in that LPAR?     cat /proc/cpuinfo...    or look in /proc/ppc64/lparcfg ..

    I tried sar on a similar RHEL 6.4 LPAR, and that count showed the correct number of virtual CPUs running.

    When you run sar -P ALL, does the bubble program stick to one CPU?

  • Bill_Buros
    Bill_Buros
    151 Posts

    Re: nmon in a shared processor partition

    ‏2013-07-16T13:34:14Z  
    • nasica88
    • ‏2013-07-15T22:58:12Z

    Sure. 

    By the way, I'd be obliged if somebody can show me how to correct that incorrect information of "64 CPUs" at the head of sar display.

     

    by the way, looking at your earlier vm3_prof.png file, the max virtual processors is set to 16, and with SMT=4, that might account for sar picking up 64 CPU.

  • willschm
    willschm
    45 Posts

    Re: nmon in a shared processor partition

    ‏2013-07-16T20:02:18Z  

    >  Is this normal ? 

    I believe so.   The process is using ~ 100% of the core (as seen by the OS).    

    You may notice in the ps output that you occasionally get 103% or 108% .. thats due to the lpar being uncapped and occasionally utilizing extra processing power above what it is strictly assigned.    

  • nagger
    nagger
    2 Posts

    Re: nmon in a shared processor partition

    ‏2013-07-22T14:58:48Z  

    I duplicated your environment as far as I can see. POWER7, SLES11 SP2, SMT=off. E=3 VP=5 (not 10) with nmon 14g from the IBM repository.  Then ran yes >/dev/null and get this (oh nuts the Forum can't take a picture file upload) Here is a text version - use the Formatted Style. Nope: that formats it all wrong 3 reties later - Courier and normal Style seems to not screw up.

     

    ┌nmon─14g──────[H for help]───Hostname=purple8──────Refresh= 2secs ───15:48.47───
    │ CPU Utilisation ───────────────────────────────────────────────────────────────
    │---------------------------+- Phys. CPU used:   1.00 ------------------------+ 
    │CPU  User%  Sys% Wait% Idle|0          |25         |50          |75       100| 
    │  1 100.0   0.0   0.0   0.0|UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU> 
    │  2   0.0   0.0   0.0 100.0| >                                               | 
    │  3   0.0   0.0   0.0 100.0| >                                               | 
    │  4   0.0   0.0   0.0 100.0| >                                               | 
    │  5   0.0   0.0   0.0 100.0| >                                               |  
    │---------------------------+-------------------------------------------------+  
    │Avg  20.0   0.0   0.0  80.0|UUUUUUUUUU >                                     |  
    │---------------------------+-------------------------------------------------+  
    │─────────────────────────────────────────────────────────────────────────────────

     

    Which looks OK, so yours is a bit of a mystery.  I noticed that with SMT=4 that is ping ponged from 0.75 to 0.125 each time and slowing down nmon screen updates to once every 4 secinds (I hit + once). leveled that out to 1.0 all the time.

    I hope this helps, cheers mr_nmon = Nigel Griffiths

    Updated on 2013-07-22T15:04:18Z at 2013-07-22T15:04:18Z by nagger
  • nasica88
    nasica88
    24 Posts

    Re: nmon in a shared processor partition

    ‏2013-07-22T22:41:37Z  
    • nagger
    • ‏2013-07-22T14:58:48Z

    I duplicated your environment as far as I can see. POWER7, SLES11 SP2, SMT=off. E=3 VP=5 (not 10) with nmon 14g from the IBM repository.  Then ran yes >/dev/null and get this (oh nuts the Forum can't take a picture file upload) Here is a text version - use the Formatted Style. Nope: that formats it all wrong 3 reties later - Courier and normal Style seems to not screw up.

     

    ┌nmon─14g──────[H for help]───Hostname=purple8──────Refresh= 2secs ───15:48.47───
    │ CPU Utilisation ───────────────────────────────────────────────────────────────
    │---------------------------+- Phys. CPU used:   1.00 ------------------------+ 
    │CPU  User%  Sys% Wait% Idle|0          |25         |50          |75       100| 
    │  1 100.0   0.0   0.0   0.0|UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU> 
    │  2   0.0   0.0   0.0 100.0| >                                               | 
    │  3   0.0   0.0   0.0 100.0| >                                               | 
    │  4   0.0   0.0   0.0 100.0| >                                               | 
    │  5   0.0   0.0   0.0 100.0| >                                               |  
    │---------------------------+-------------------------------------------------+  
    │Avg  20.0   0.0   0.0  80.0|UUUUUUUUUU >                                     |  
    │---------------------------+-------------------------------------------------+  
    │─────────────────────────────────────────────────────────────────────────────────

     

    Which looks OK, so yours is a bit of a mystery.  I noticed that with SMT=4 that is ping ponged from 0.75 to 0.125 each time and slowing down nmon screen updates to once every 4 secinds (I hit + once). leveled that out to 1.0 all the time.

    I hope this helps, cheers mr_nmon = Nigel Griffiths

    If SMT=4 was on, it would be correct that "Phy CPU used" should be 0.65.  Maybe this is a bug related to SMT stats in RHEL 6.4 ?

  • nagger
    nagger
    2 Posts

    Re: nmon in a shared processor partition

    ‏2013-07-23T09:13:35Z  
    • nasica88
    • ‏2013-07-22T22:41:37Z

    If SMT=4 was on, it would be correct that "Phy CPU used" should be 0.65.  Maybe this is a bug related to SMT stats in RHEL 6.4 ?

    Hi,

    I retried the test scenario on SLES 11.2 with SMT=4 - it still works fine but still shows the fluctuation until you monitor slightly longer term.

    I don't have RHEL6.4 online at the moment to test it.

    Can I suggest you try the simple test of running "yes" then nmon hit C and + ?   To see what you get.

    Then (if still wrong) collect some nmon data for me too look at. I may need you to follow up with collecting the /proc files that nmon uses for the raw data to determine if it is a problem there.

    thanks Nigel

  • Bill_Buros
    Bill_Buros
    151 Posts

    Re: nmon in a shared processor partition

    ‏2013-07-23T13:37:41Z  
    • nagger
    • ‏2013-07-22T14:58:48Z

    I duplicated your environment as far as I can see. POWER7, SLES11 SP2, SMT=off. E=3 VP=5 (not 10) with nmon 14g from the IBM repository.  Then ran yes >/dev/null and get this (oh nuts the Forum can't take a picture file upload) Here is a text version - use the Formatted Style. Nope: that formats it all wrong 3 reties later - Courier and normal Style seems to not screw up.

     

    ┌nmon─14g──────[H for help]───Hostname=purple8──────Refresh= 2secs ───15:48.47───
    │ CPU Utilisation ───────────────────────────────────────────────────────────────
    │---------------------------+- Phys. CPU used:   1.00 ------------------------+ 
    │CPU  User%  Sys% Wait% Idle|0          |25         |50          |75       100| 
    │  1 100.0   0.0   0.0   0.0|UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU> 
    │  2   0.0   0.0   0.0 100.0| >                                               | 
    │  3   0.0   0.0   0.0 100.0| >                                               | 
    │  4   0.0   0.0   0.0 100.0| >                                               | 
    │  5   0.0   0.0   0.0 100.0| >                                               |  
    │---------------------------+-------------------------------------------------+  
    │Avg  20.0   0.0   0.0  80.0|UUUUUUUUUU >                                     |  
    │---------------------------+-------------------------------------------------+  
    │─────────────────────────────────────────────────────────────────────────────────

     

    Which looks OK, so yours is a bit of a mystery.  I noticed that with SMT=4 that is ping ponged from 0.75 to 0.125 each time and slowing down nmon screen updates to once every 4 secinds (I hit + once). leveled that out to 1.0 all the time.

    I hope this helps, cheers mr_nmon = Nigel Griffiths

    Nigel, for what it's worth, you can "attach" a jpeg/png image to each reply here.     It'll show up in the attachment list for the reply..      I haven't figured out how to self-reference that image in the reply..  but in practice, it's not really needed.

  • nasica88
    nasica88
    24 Posts

    Re: nmon in a shared processor partition

    ‏2013-07-25T01:39:05Z  
    • nagger
    • ‏2013-07-23T09:13:35Z

    Hi,

    I retried the test scenario on SLES 11.2 with SMT=4 - it still works fine but still shows the fluctuation until you monitor slightly longer term.

    I don't have RHEL6.4 online at the moment to test it.

    Can I suggest you try the simple test of running "yes" then nmon hit C and + ?   To see what you get.

    Then (if still wrong) collect some nmon data for me too look at. I may need you to follow up with collecting the /proc files that nmon uses for the raw data to determine if it is a problem there.

    thanks Nigel

    Sorry for my late reply.

    'yes' does not use up one full core, I observe.  I attach everything that you asked, including /proc/<pid_of_yes>.

     

    [root@VM3 ~]# ps -ef | grep yes
    root      4204  4170 25 10:21 pts/0    00:04:08 yes
    root      4370  4188  0 10:37 pts/1    00:00:00 grep yes
    [root@VM3 ~]# tar -cvf /tmp/4204.tar 4204
    ...
    4204/schedstat
    4204/cpuset
    4204/cgroup
    4204/oom_score
    4204/oom_adj
    4204/oom_score_adj
    4204/loginuid
    4204/sessionid
    4204/coredump_filter
    4204/io
     

    Attachments