We have 2 P6 servers running AIX 6.1 TL6. We have an Internet connection to a remote datacenter about 10 miles away with a 20MB connection, (30Mb at main site) and only get about 5Mb throughput where the Windows hosts can sustain 17-19Mb connection when using Iperf or FTP as a test.
We're trying to use GLVM to replicate filesystems which sends large packets much like FTP. On the LAN this worked fine, but over the WAN we're only getting 2-5Mb throughput. We've tried FTP and Iperf and get similare results. Using Iperf as a test throughput is slow only when AIX is the sender, if AIX is in server mode thoughput is fine from a Windows host. Ping times between the hosts/sites are about 1ms.
When I look at the IPTraces in Wireshark I'll see multiple data packets then an ACK, then it will stumble and retransmit and receive multiple ACK's. Since the main server is uplinked at 10G I tried tweaking a few parms such as large_send and chksum_offload but this did not help. I suspect it has something to do with the RTT round trip time although it's my understanding TCP/IP is suppose to self adjust during a connection. TCP Window size remains at 65536 so it's not filling the buffer. Thoughput to either server over LAN connection is fine.
Any ideas what to change? I'd attribute it to the Internet or Time Warner except the Windows hosts essentially get max circuit speed.
Pinned topic Poor network throughput over WAN between 2 AIX servers compared to Windows?
Answered question This question has been answered.
Unanswered question This question has not been answered yet.
Updated on 2012-04-23T03:09:13Z at 2012-04-23T03:09:13Z by Roden
Roden 270003SR3R1 Post
Re: Poor network throughput over WAN between 2 AIX servers compared to Windows?2012-04-23T03:09:13ZThis is the accepted answer. This is the accepted answer.Some thoughts:
If you use iperf set the window size.
Check the MTU size (netstat -r or TCP handshake from iptrace).
Check the no attributes isno and rfc1323 are enabled (set to 1).
Check the network interface settings with ifconfig -a (tcp_sendspace and rfc1323).
Check the actual socket transmit settings with netstat -ano.
Check if there are known issues, aka open a PMR.
It might be that AIX is transmitting too much too fast over the WAN, and when you get multiple retransmits, enable sack with no to reduce the number of retransmits. To reduce the initial window size, after retransmits, you can set tcp_init_window to 1 and enable rfc2414. And you can experiment with setting tcp_maxburst (specifies the number of back-to-back packets that TCP can send before pausing to allow those packets to be forwarded to their destination).
There are some new no attributes with AIX 7.1 relating to WAN and congestion window management, such as hstcp.