Topic
3 replies Latest Post - ‏2009-05-30T18:10:39Z by walterchen_austria
walterchen_austria
walterchen_austria
10 Posts
ACCEPTED ANSWER

Pinned topic infiniband GX-Adapter on p520 with rhel5U2 "HCA version not 0x1000003"

‏2009-01-30T10:27:49Z |
Hello!

Does anybody have experience with infiniband on rhel5 and ofed 1.4 on power? We want to migrate some p520 from aix and tested infiniband, the driver ehca is used (or the IB-GX-adapter)

But the speed is not correct, we have a 4x SDR
root@c36l tmp# ibstatus
Infiniband device 'ehca0' port 1 status:
default gid: fe80:0000:0000:0000:0002:5500:0238:2f00
base lid: 0x76
sm lid: 0x2
state: 4: ACTIVE
phys state: 5: LinkUp
rate: 30 Gb/sec (12X)

Infiniband device 'ehca0' port 2 status:
default gid: 0000:0000:0000:0000:0002:5500:0238:2f40
base lid: 0x0
sm lid: 0x0
state: 1: DOWN
phys state: 5: LinkUp
rate: 30 Gb/sec (12X)

root@c36l tmp#
.)ANd the main Problem is we can´t activate IPoIB.
root@c36l tmp# ifconfig ib0 up
SIOCSIFFLAGS: Invalid argument

.) we can´t start the fabric manager
root@c36l tmp# opensm

OpenSM 3.2.5_20081207
Reading Cached Option File: /etc/opensm/opensm.conf
Command Line Arguments:
Log File: /var/log/opensm.log

OpenSM 3.2.5_20081207

Entering DISCOVERING state

Using default GUID 0x2550002382f00

Error from osm_opensm_bind (0x2A)
Perhaps another instance of OpenSM is already running
Exiting SM
########################
In a pdf from IBM 48D109AEd01.pdf we found following notice:

"If HCA hardware version is not 0x1000003", contact IBM Support" but we don´t have a software support
and we have hardware version 0x1000002
root@c36l tmp# ibv_devinfo
hca_id: ehca0
node_guid: 0002:5500:0238:2f00
sys_image_guid: 0000:0000:0000:0000
vendor_id: 0x5076
vendor_part_id: 0
hw_ver: 0x1000002
phys_port_cnt: 2
port: 1
state: PORT_ACTIVE (4)
max_mtu: 2048 (4)
active_mtu: 2048 (4)
sm_lid: 2
port_lid: 118
port_lmc: 0x00

port: 2
state: PORT_DOWN (1)
max_mtu: 2048 (4)
active_mtu: 2048 (4)
sm_lid: 0
port_lid: 0
port_lmc: 0x00

###############

Can anybody help us?
root@c36l tmp# ibstat
CA 'ehca0'
CA type:
Number of ports: 2
Firmware version:
Hardware version:
Node GUID: 0x0002550002382f00
System image GUID: 0x0000000000000000
Port 1:
State: Active
Physical state: LinkUp
Rate: 30
Base lid: 118
LMC: 0
SM lid: 2
Capability mask: 0x00000068
Port GUID: 0x0002550002382f00
Port 2:
State: Down
Physical state: LinkUp
Rate: 30
Base lid: 0
LMC: 0
SM lid: 0
Capability mask: 0x00000068
Port GUID: 0x0002550002382f40
root@c36l tmp# /etc/init.d/openibd status
root@c36l tmp# /etc/init.d/openibd status

HCA driver loaded

Configured IPoIB devices:
ib0 ib1

Currently active IPoIB devices:
The following OFED modules are loaded:

rdma_ucm
qlgc_vnic
ib_sdp
rdma_cm
ib_addr
ib_ipoib
ib_ehca
ib_ipath
mlx4_core
mlx4_ib
ib_mthca
ib_uverbs
ib_umad
ib_sa
ib_cm
ib_mad
ib_core
iw_cxgb3

root@c36l tmp#
Updated on 2009-05-30T18:10:39Z at 2009-05-30T18:10:39Z by walterchen_austria
  • SystemAdmin
    SystemAdmin
    706 Posts
    ACCEPTED ANSWER

    Re: infiniband GX-Adapter on p520 with rhel5U2 "HCA version not 0x1000003"

    ‏2009-02-27T19:09:37Z  in response to walterchen_austria
    We are having a similar problem on a P550. Currently working with the LOP team to resolve. Have you had any luck. What fabric are you connected to (Voltaire, Cisco, QLOGIC)?

    You can reach me directly at bill.morton@e-isys.com
    • walterchen_austria
      walterchen_austria
      10 Posts
      ACCEPTED ANSWER

      Re: infiniband GX-Adapter on p520 with rhel5U2 "HCA version not 0x1000003"

      ‏2009-05-30T18:09:47Z  in response to SystemAdmin
      sorry for my late answer
      we have to change the gx-infiniband adapters (using Topspin/Cisco-fabric)

      this will become expensive :-(

      regards
      walter
  • walterchen_austria
    walterchen_austria
    10 Posts
    ACCEPTED ANSWER

    Re: infiniband GX-Adapter on p520 with rhel5U2 "HCA version not 0x1000003"

    ‏2009-05-30T18:10:39Z  in response to walterchen_austria
    i will close this thread now