Topic
  • 5 replies
  • Latest Post - ‏2010-05-27T19:58:32Z by nfont
chwilk
chwilk
5 Posts

Pinned topic Avoiding SMT issues with IRQ balance

‏2010-05-21T16:25:33Z |
I'll occasionally get errors in /var/log/messages indicating that irqbalance has tried to peg an interrupt to offline CPUs in SMT=1 or SMT=2 modes.

According to irqbalance documentation, I should set the IRQBALANCE_BANNED_CPUS bitmask to avoid this problem.

For a four-socket, eight core P7 system, my mask would look like 32 hexadecimal "e" characters, which on RHEL 5 can be set in /etc/sysconfig/irqbalance

I just wanted to run this by the forum to see if my logic made sense, and whether anybody else has run into this, tried this fix, or has contrary experience.
Updated on 2010-05-27T19:58:32Z at 2010-05-27T19:58:32Z by nfont
  • SystemAdmin
    SystemAdmin
    706 Posts

    Re: Avoiding SMT issues with IRQ balance

    ‏2010-05-25T20:26:54Z  
    Checking. Just to be sure, is the system completely in SMT=1 or SMT=2 mode? in other words, ppc64_cpu reports the correct mode for the whole system?

    
    # ppc64_cpu --smt smt=2
    


    Second, is this causing any unexpected behavior?
  • SystemAdmin
    SystemAdmin
    706 Posts

    Re: Avoiding SMT issues with IRQ balance

    ‏2010-05-26T13:36:47Z  
    Also, just to confirm.

    You should have irqbalance-0.55-15.el5 on these systems?
  • chwilk
    chwilk
    5 Posts

    Re: Avoiding SMT issues with IRQ balance

    ‏2010-05-26T16:08:07Z  
    Also, just to confirm.

    You should have irqbalance-0.55-15.el5 on these systems?
    Yes, that's the version we've got. I think the errors occur when an smt=4 system goes to smt=2 or smt=1.
  • chwilk
    chwilk
    5 Posts

    Re: Avoiding SMT issues with IRQ balance

    ‏2010-05-26T19:27:16Z  
    It would be a useful feature for ppc64_cpu to log changes in smt state (and others) to syslog when successfully completed.
  • nfont
    nfont
    1 Post

    Re: Avoiding SMT issues with IRQ balance

    ‏2010-05-27T19:58:32Z  
    • chwilk
    • ‏2010-05-26T19:27:16Z
    It would be a useful feature for ppc64_cpu to log changes in smt state (and others) to syslog when successfully completed.
    This is pretty easy to do, and I have no problem adding this to ppc64_cpu. I do think that the logging my be better off coming from somewhere else. There are other tools that can change the smt state of the system in addition to a root users echo'ing 1's and 0's into sysfs hotplug cpus.

    I think something that would catch all changes to smt state may be a better solution, but am not sure how easy that would be.