Separating the network streams
The purpose of these test runs was to show the impact of separating the network streams between the clients and the WebSphere® Application Server from the streams between the WebSphere Application Server and the database.
One setup used one shared OSA network card for the connections from the clients and the database to the WebSphere Application Server, the other one used two cards, one for each connection, for details see Network setup.

Observations
As seen in Figure 1, with two network cards, the transaction rate increases by 27% and the CPU load also increases. The slight decrease in ITR shows that the costs are increasing slightly.
Conclusion
Using a separate network card for the traffic to the client and to the database results in a very good ETR improvement of 27%. This shows that sharing one network card for all traffic is a bottleneck. All further measurements we performed were done with separate network cards for the client and the database traffic. The increase in CPU cost (decreasing ITR) is so low it might be explained as just noise. Interestingly, both parameters are changing in the same direction, which is consistent because an increase in instructions per transaction is always correlated with an increase in CPU load.
Network considerations
Figure 2 shows the network throughput and packet rate summed up for send and receive on all interfaces of the WebSphere Application Server for the scenarios with the database on z/OS® where all traffic goes over one OSA card and where the traffic to the database goes over separate connections, either an OSA card or HiperSockets.

For the scenario with the database on z/OS, the throughput increases up to 64% with the separate database connections and HiperSockets.