Our MQ on z/OS performance guy had problems where a couple of weeks ago the MQ throughput over TCP/IP dropped from 100 MB/Second down to about 20 MB/Second. No one else complained, so it clearly must be an MQ problem (of course). Nothing else had changed - honest guv!
After lots of pulling of hair they found if they plugged the optical fibre so it bypassed a switch, the performance was good again. They did something to the switch and every thing was good again... till last mid week, when the performance dropped again. The weekend before they had done some housekeeping and unplugged and replugged the optical fibre.
They unplugged the optical fibre and used a microscope to look at the end. There was a small speck of dust, so they cleaned it, replace the fibre and it worked, giving good throughtput.
We think the speck of dust degraded the throughput, but not badly enough to produce an alert.
So the lesson learned, is unplugging and replugging can change the throughput!