Topic
5 replies Latest Post - ‏2010-05-27T19:58:32Z by nfont
chwilk
chwilk
4 Posts
ACCEPTED ANSWER

Pinned topic Avoiding SMT issues with IRQ balance

‏2010-05-21T16:25:33Z |
I'll occasionally get errors in /var/log/messages indicating that irqbalance has tried to peg an interrupt to offline CPUs in SMT=1 or SMT=2 modes.

According to irqbalance documentation, I should set the IRQBALANCE_BANNED_CPUS bitmask to avoid this problem.

For a four-socket, eight core P7 system, my mask would look like 32 hexadecimal "e" characters, which on RHEL 5 can be set in /etc/sysconfig/irqbalance

I just wanted to run this by the forum to see if my logic made sense, and whether anybody else has run into this, tried this fix, or has contrary experience.
Updated on 2010-05-27T19:58:32Z at 2010-05-27T19:58:32Z by nfont
  • SystemAdmin
    SystemAdmin
    706 Posts
    ACCEPTED ANSWER

    Re: Avoiding SMT issues with IRQ balance

    ‏2010-05-25T20:26:54Z  in response to chwilk
    Checking. Just to be sure, is the system completely in SMT=1 or SMT=2 mode? in other words, ppc64_cpu reports the correct mode for the whole system?

    
    # ppc64_cpu --smt smt=2
    


    Second, is this causing any unexpected behavior?
  • SystemAdmin
    SystemAdmin
    706 Posts
    ACCEPTED ANSWER

    Re: Avoiding SMT issues with IRQ balance

    ‏2010-05-26T13:36:47Z  in response to chwilk
    Also, just to confirm.

    You should have irqbalance-0.55-15.el5 on these systems?
    • chwilk
      chwilk
      4 Posts
      ACCEPTED ANSWER

      Re: Avoiding SMT issues with IRQ balance

      ‏2010-05-26T16:08:07Z  in response to SystemAdmin
      Yes, that's the version we've got. I think the errors occur when an smt=4 system goes to smt=2 or smt=1.
  • chwilk
    chwilk
    4 Posts
    ACCEPTED ANSWER

    Re: Avoiding SMT issues with IRQ balance

    ‏2010-05-26T19:27:16Z  in response to chwilk
    It would be a useful feature for ppc64_cpu to log changes in smt state (and others) to syslog when successfully completed.
    • nfont
      nfont
      1 Post
      ACCEPTED ANSWER

      Re: Avoiding SMT issues with IRQ balance

      ‏2010-05-27T19:58:32Z  in response to chwilk
      This is pretty easy to do, and I have no problem adding this to ppc64_cpu. I do think that the logging my be better off coming from somewhere else. There are other tools that can change the smt state of the system in addition to a root users echo'ing 1's and 0's into sysfs hotplug cpus.

      I think something that would catch all changes to smt state may be a better solution, but am not sure how easy that would be.