We are using GPFS on WIndows 2008 and Linux. Parts of the windows nodes would encounter IPoIB unavailable problem every a few days. When problem occurs, ping 90% nodes get no response, thus GPFS becomes unmounted. However, if we run mmshutdown, the IB would become available, ping all nodes will then get response. Would GPFS influence IPoIB on Windows? Thanks.
FelipeKnop 120000G51H35 Posts
Re: May GPFS make IPoIB unavailable?2012-08-24T17:57:48ZThis is the accepted answer. This is the accepted answer.It's unlikely that GPFS is causing the problem with IPoIB, since GPFS is merely a "user" of IPoIB. Perhaps the only impact GPFS may have is provoking (possibly heavy) load on the network.
I've been advised that one should look at the Windows event logs to see if there are any messages from the IB stack that may give clues. Also, it's worth checking the state of the node, such as memory consumption etc, while IPoIB is having issues.
osc 100000U5GE11 Posts
Re: May GPFS make IPoIB unavailable?2012-08-27T14:10:15ZThis is the accepted answer. This is the accepted answer.Can you supply some more details about your setup? Are you using just the TCP layer or are you using IB Verbs? Versions of OFED, or whatever drivers for IB you are using? Have you looked at the servers when the problem occurs and tried to see what outstanding requests are on them?
I have a vague recollection about a problemlike this with TCPoIB and Linux/Windows crossmounts due to problems with Windows driver implementations that caused IB traffic to become a DDOS on the Winodws nodes, but that was several years ago. I'll have to check my notes.