• 2 replies
  • Latest Post - ‏2012-08-27T14:10:15Z by osc
17 Posts

Pinned topic May GPFS make IPoIB unavailable?

‏2012-08-24T13:16:46Z |
We are using GPFS on WIndows 2008 and Linux. Parts of the windows nodes would encounter IPoIB unavailable problem every a few days. When problem occurs, ping 90% nodes get no response, thus GPFS becomes unmounted. However, if we run mmshutdown, the IB would become available, ping all nodes will then get response. Would GPFS influence IPoIB on Windows? Thanks.
Updated on 2012-08-27T14:10:15Z at 2012-08-27T14:10:15Z by osc
  • FelipeKnop
    35 Posts

    Re: May GPFS make IPoIB unavailable?

    It's unlikely that GPFS is causing the problem with IPoIB, since GPFS is merely a "user" of IPoIB. Perhaps the only impact GPFS may have is provoking (possibly heavy) load on the network.

    I've been advised that one should look at the Windows event logs to see if there are any messages from the IB stack that may give clues. Also, it's worth checking the state of the node, such as memory consumption etc, while IPoIB is having issues.

  • osc
    11 Posts

    Re: May GPFS make IPoIB unavailable?

    Can you supply some more details about your setup? Are you using just the TCP layer or are you using IB Verbs? Versions of OFED, or whatever drivers for IB you are using? Have you looked at the servers when the problem occurs and tried to see what outstanding requests are on them?

    I have a vague recollection about a problemlike this with TCPoIB and Linux/Windows crossmounts due to problems with Windows driver implementations that caused IB traffic to become a DDOS on the Winodws nodes, but that was several years ago. I'll have to check my notes.