Tracking Technical Trends - WebSphere MQ (Part II)
MarkWomack 270000PC6X Visits (9140)
It's been over five months since I last analyzed trends for the PMR issues that we see most at Level 2. My last blog entry seemed to have some positive effect on alerting people on what to avoid. Now, I'll take a look at some interesting new problems IBM MQ has helped resolve as we close in on the end of Summer and look forward to what the 4th quarter will bring.
For the z/OS platform, the System SSL features ship with the Operating System (synonymous with the GSKit shipping with the distributed platforms). Made available in the Spring, the fixes for SSL APAR OA47183 corrected an issue where some unnecessary SMF records were being created. As coincidence would have it, while the code improvements implemented by its PTFs (UA76977, UA76978, UA76979, UA76980) operated seamlessly with customers that used RACF as their ESM (External Security Manager), those that happened to use ACF2 began to experience SSL server subtask abnormal terminations which would write messages to the CHIN joblog similar to:
CSQX153E CSQXSERV SSL server subtask ended abnormally, TCB=tcb_address reas
This could be followed by various CEE and z/OS UNIX error messages (CEE3796I and BPXP018I). For IBM MQ this disables the use of SSL as all such channels can fail to start.
When I took a look at some of these, I found that not only could IBM MQ be affected, but so could some other WebSphere products. Because System SSL was now issuing calls for "RACROUTE REQUEST=AUTH, STATUS=ACCESS" these were being flagged only by ACF2 (as a matter of its security policy). The policy's requirement was that a task be APF authorized. This caused a percolation of this error to result in an LE abend.
ACF2 realized the flaw and so provides what CA calls a SAFDEF (which basically defines the SAF environment, telling CA-ACF2 what to do when it has to process a SAF call). For IBM MQ that SAFDEF should look something like:
INSERT SAFDEF.mqapf ID(mqapf) PROGRAM(CSQXSERV)
I recommend, after the SAFDEF is incorporated, that the MQ subsystems be recycled. As I mentioned, this can affect other products that ACF2 provides security function for. The SAFDEF for those other products (such as WebSphere Application Server - WAS) will differ from the one for IBM MQ, so users should contact ⇒CA Technologies for ACF2 in order to confirm what the SAFDEF should specify.
On the distributed side of MQ, users of FTE (on AIX boxes) that are migrating should be aware of the strict System Requirements documented at "System Requirements for WebSphere MQ V7.5" in the Compiler section. As the product is migrated or used on AIX platforms, to ensure it functions as expected, the level of XL C/C++ can be no more recent than AIX 12.1 (and its future fixpacks). Should a higher level be installed, it would need to be downgraded, otherwise commands such as fteListAgents, and even the ability to start the agent, may be lost. (Note that downgrading the version of XL C++ runtimes should never go past the point of the version that was shipped with the AIX operating system itself. This is because running applications that have been compiled/linked with newer versions of the runtime would then not be supported to run with the older versions of the XL C++ Runtime; so this is a serious planning consideration to take into account). Users can see how this could impact applications that have already been compiled at higher levels.
While it is always recommended that MQ users employ the highest levels of secure transport (avoiding any known vulnerabilities such as POODLE), there are many users that still have a business requirement to use the older protocols because of their specific business arrangement. Specifically, in the z/OS environment, APARs, like PI28800, have been designed to disable the use of outdated protocols in favor of higher levels of TLS and/or FIPs support. That said, in cases where the required remedial action is to rely on older protocols, with such APARs in place, users will need to follow the specific steps outlined in order to fall back to the older security protocols, and, as a best practice, should also (a) issue REFRESH SECURITY TYPE(SSL), and restart the listener on the QMGR that will be receiving requests for a secure connection. This will allow the protocol change to apply to any new channel connections that are established. If this is not done, the user may see messages such as CSQX635E (indicating that the specified CipherSpec is invalid); this is a good indication that the protocol change has not been fully put into effect.
I'll continue to analyze trends over the next several months and update the blog in early 2016 with those issues which could affect a significant part of the MQ world. Stay tuned.