What happened in Vegas
IBM WebSphere MQ security implementations have not always received the attention or priority that they deserve. As I mentioned in an earlier developerWorks column, many WebSphere MQ installations have no security, or worse, have it implemented with holes and only think that they are secure. When I wrote that article last January, WebSphere MQ was not a very high profile product outside of its community of administrators and developers. To some extent, you could say we all enjoyed security by obscurity. All that changed on August 3, 2007 in Las Vegas.
Last July, the WebSphere MQ list server buzzed with news of an upcoming session at Def Con 15 detailing security exploits against WebSphere MQ. The session, presented by Martyn Ruks of MWR Infosecurity, was entitled "MQ Jumping" and the abstract included the following description:
This presentation will uncover the truth behind WebSphere MQ security as it is deployed in the real world and will look at how the software can be abused by an attacker resulting in remote code execution. The talk will focus on methods for analysing the security controls that can be used to protect an installation of MQ and the limitations of each of them. Following on from this section of the talk a number of methods will be presented for compromising both the message data and the Operating System through the MQ service. This will culminate in a demonstration of some of the attacks presented in the talk, followed by a discussion about the methods that exist for protecting an installation and ensuring that security breaches do not occur.
In case you are not familiar with it, Def Con bills itself as "the largest underground hacking convention in the world." The presentation abstract alone was intriguing. The notion of presenting it to this particular audience was downright scary. The next day I booked a flight to Vegas and a room at the convention hotel.
The audience and the presentation both lived up to their billing. The main-tent room had begun to fill for the MQ Jumping session even before the previous speaker had wrapped up his talk. By the time Mr. Ruks took the podium, the room was filled to capacity and attendees lined the aisles and stood crowded in the back. Imagine every hardcore hacker from every movie or television show you have ever seen: they were all in the room.
The presentation started out very much like the one I give at IMPACT and at Transaction and Messaging conferences. He spoke about the default channels and how to exploit them, about the command server, the trigger monitor, and initiation queues. But then he entered into uncharted territory.
In his research, Mr. Ruks had discovered a method by which he could cause a client channel to start with administrative authority, even though it was protected by a security exit and a hardcoded MCAUSER. To his credit, he disclosed the vulnerability but not the exact details of how to exercise it (although the thought occurred to me that I might be one of only a few people in attendance for whom this lack of instruction would present an obstacle).
Although you might feel uneasy about the idea of a security exposure being presented at a "hackers's convention" (and you should), it might reassure you to know that the problem had been reported to IBM prior to the conference. Another security research team, working independently at National Australia Bank, discovered the same vulnerability Mr. Ruks spoke about and, in fact, is credited by IBM with first discovery. Both teams had reported their findings to IBM well in advance of Def Con and a patch was released by IBM on August 6.
What does all this mean? Was the Def Con session about breaking into WebSphere MQ or securing it? Ultimately, the answer is that public disclosure results in better security. It keeps vendors focused on continuous improvement and it puts software users on notice that securing only the obvious vulnerabilities is not good enough. Feedback from independent security consultants and customers results in a more secure and robust product -- but only if you enable it. As Mr. Ruks and I both point out in our presentations, a large number of companies visited have not enabled WebSphere MQ security -- which is the problem I am hoping to address here.
The remainder of this article explains how to update your installation with the WebSphere MQ channel patch mentioned above, plus additional stepsyou can take to keep your queues secure. You will also find more information about WebSphere MQ security in Resources.
Update your channel code
The first thing you will want to do is upgrade to WebSphere MQ V220.127.116.11 or apply the channel security patch. If you are unable to upgrade or apply the patch, a workaround is to set an invalid SSLCIPH value on unused channels. This will cause any connection attempts to fail before the vulnerable code can be exercised. There isn't any way to eliminate the vulnerability on channels that are legitimately used, but for these channels, enabling SSL and filtering down to very specific SSLPEER values at least limits the potential population of attackers.
After you have patched or upgraded, what other steps can you take to keep the queue manager secure? If there are users or applications connecting in bindings mode (using shared memory), the operating system will authenticate them using an ID and a password. If the server is reasonably secure, so are local logins. The trick is in how you authenticate remotely connected users, applications connecting in client mode, and connections from other queue managers.
Remote connections to a queue manager typically occur over a channel. When no user ID is presented to the channel, authorization checks are made against the user ID associated with the process running the channel -- and that ID is always privileged. An MCA channel (one between two queue managers) can be configured to authorize based on the ID contained within the message header, but without authentication, the ID in the message header cannot be trusted.
A similar situation exists with the MQI (client) channels. The WebSphere MQ client code will pass the logged-in user ID that calls it, but that is only a default. The programming interface allows for the ID presented in a client connection request to be set from within the application. This overrides the ID obtained from the system. Obviously, these IDs cannot be trusted either unless you can arrange to authenticate them.
You might be thinking at this point that enabling SSL would provide the authentication required. Well, it does and it doesn't. When using a client to connect over an SSL channel, the SSL protocol starts and ends with the channel agent. The authentication provided by SSL does not extend down to the API calls. Once the connection is established, everything outside the SSL exchange (the connection request, the API calls) works exactly as it does without SSL. So even if the queue manager is configured to accept only SSL authenticated connections from John Doe, the MQ API still allows John to assert a different identity in his connection request. If John asserts the identity of Jane Doe, the queue manager accepts it without question, even over an SSL-enabled channel. If John chooses to assert the mqm ID, he can gain administrative privileges.
In summary, authentication must be performed locally on the receiving queue manager. If SSL channels are the basis for authentication, an exit must be used to tie the SSL certificate identity to an MCAUSER so that access policies can be applied to API calls. An exit is also required for password-based authentication. Alternatively, WebSphere MQ Extended Security Edition can be used to sign individual messages and enforce access policies end to end.
But I need a fix NOW!
At many of the security assessments I have performed, I have had to report to the company that they were allowing anonymous administrative access to their messaging system through unsecured channels. Naturally, they were anxious to plug the holes, but deploying exits across the entire messaging network can be a huge undertaking -- and exits are only the beginning. In shops where access has never been secured, it is common to find many applications and users connecting with administrative privileges. Locking down the channels will break the access of all these users. It can be a painstaking process to break the access of each application, figure out on a queue-by-queue basis what privileges are really needed, and then fix them. For these reasons, I am often asked whether there is any option that effectively locks down administrative access but imposes minimal impact to applications or users, does not require exits or SSL, and which can be deployed as quickly as possible.
Happily, the answer is "yes!" You can hardcode the MCAUSER in the channel definitions with a user ID that has access to all of the WebSphere MQ resources except the administrative ones. But be aware that there is a trade off. With the default settings, you were able to authorize different users with different permissions, and the users would get authorization errors if they tried to get into each others' queues. But with no authentication, any user could simply pretend to be an administrator. When you hardcode an MCAUSER, you can lock down administrative access, but you lose the ability to distinguish one user from another. There are no more authorization errors. I would argue that having one ID that you can both trust and enforce is MUCH better than having a lot of IDs that you can neither trust nor enforce. Until you are ready to implement a real authentication scheme, this is a good compromise.
Before you start locking down channels, it will be necessary to provision two dedicated user IDs and groups that are accessible to each queue manager. For the purpose of this article. I am going to assume a UNIX environment where the user ID and group name can be identical. These should be standard application service accounts, and by that I mean non-login accounts that are not shared by any other applications or users. Each group should have one and only one ID in it. Each ID should be a member of one and only one group. One of the ID and group pairs is for MCA channels and the other is for MQI channels so let's call them mqmmca:mqmmca and mqmmqi:mqmmqi.
The channels that connect queue managers to one another are known as MCA channels because they use a message channel agent. An inbound MCA channel can be a receiver, requestor, or cluster receiver. If you have looked closely at the system administration manual, you may have noticed that tools like runmqsc and WMQ Explorer can use one queue manager as a proxy to administer another queue manager using MCA channels. This means that, in the default configuration, any queue manager can be administered by any adjacent queue manager.
Clearly, if you wish to restrict administrative access, you need to consider MCA channels. Here is a script of setmqaut commands that block administrative access over MCA channels. These would be applied to all non-default channels on the queue manager. (Your default channels should have been set to MCAUSER('nobody') a long time ago, but that's a topic for another article.)
# Allow MCAUSER to connect. Needs +set and +setall per IBM docs. setmqaut -m QMGR -g mqmmca -t qmgr -all +connect +inq +set +setall # Grant MCAUSER default policy of "allow all" to all queues. Channels # put messages so no need for get, browse, etc. Also needs +setall. setmqaut -m QMGR -g mqmmca -n '**' -t queue -all +put +setall # Now deny access to administrative queues setmqaut -m QMGR -g mqmmca -n SYSTEM.ADMIN.COMMAND.QUEUE -t queue -all +none setmqaut -m QMGR -g mqmmca -n SYSTEM.DEFAULT.INITIATION.QUEUE -t queue -all +none # Restrict access to any additional initiation queues as well.
This, of course, is the bare minimum and restricts only administrative access. Adjacent queue managers can still put messages to SYSTEM.* queues and any application queues, but this is probably good enough for a network of trusted queue managers.
Once the connections from other queue managers are locked down to your satisfaction, you need to consider the MQI channels. These are represented by SVRCONN definitions. Hopefully, your SYSTEM.DEF.SVRCONN and SYSTEM.AUTO.SVRCONN already have MCAUSER('nobody') set and are not actively used. If this is not the case, these channels will need to receive the updates described here. Whatever you do, don't leave these channels without an MCAUSER set.
We should also pause a moment and consider SYSTEM.ADMIN.SVRCONN. If the WebSphere MQ administration team uses a desktop client to access the queue managers, the configurations proposed here will break that access. But in the absence of authentication, leaving the channel unsecured opens up administrative privileges to anyone. There are two options here: either implement authentication in a limited fashion just for the administration team, or require administrators to log onto the server to make configuration changes. To simplify this discussion, we will assume the latter case. After all, you would not be considering this option if you were prepared to move forward with SSL and/or exits in the first place.
After the changes are applied, SYSTEM.ADMIN.SVRCONN will still be available and desktop tools, such as WebSphere MQ Explorer will still work, up to a point. Users will be able to browse, get or put messages, inquire on any object, and even set objects (enabling triggering, for example), but they will not be able to create or delete objects or control channels.
# Allow MCAUSER to connect to QMgr setmqaut -m QMGR -g mqmmqi -t qmgr -all +connect +allmqi +dsp # Allow inquire/display of non-queue objects # Some of these object types are new for v6 setmqaut -m QMGR -g mqmmqi -n '**' -t namelist -all +dsp +inq setmqaut -m QMGR -g mqmmqi -n '**' -t process -all +dsp +inq setmqaut -m QMGR -g mqmmqi -n '**' -t authinfo -all +dsp +inq setmqaut -m QMGR -g mqmmqi -n '**' -t channel -all +dsp setmqaut -m QMGR -g mqmmqi -n '**' -t service -all +dsp setmqaut -m QMGR -g mqmmqi -n '**' -t listener -all +dsp setmqaut -m QMGR -g mqmmqi -n '**' -t clntconn -all +dsp # Default allow-all to all queues setmqaut -m QMGR -g mqmmqi -n '**' -t queue -all +allmqi +dsp # This gets us back to almost full admin but positions us to set # additional restrictions. # Restrict access to SYSTEM.** queues. Browsing and display of these # queues is reasonable but not PUT, SET, ALTUSR, etc. setmqaut -m QMGR -g mqmmqi -n 'SYSTEM.**' -t queue -all +inq +browse +dsp # Allow limited access to command queue. setmqaut -m QMGR -g mqmmqi -n 'SYSTEM.ADMIN.COMMAND.QUEUE' -t queue -all +inq +put +dsp # Allow access to SYSTEM.MQEXPLORER.REPLY.QUEUE if it exists. setmqaut -m QMGR -g mqmmqi -n 'SYSTEM.MQEXPLORER.REPLY.QUEUE' -t queue -all +inq +dsp +put # Allow access to SYSTEM.DEFAULT.MODEL.QUEUE. setmqaut -m QMGR -g mqmmqi -n 'SYSTEM.DEFAULT.MODEL.QUEUE' -t queue -all +allmqi +dsp # Restrict access to DLQ. This assumes SYSTEM DLQ is used. If not, # you might want to script this to inquire on the QMgr DLQ property # and set that. setmqaut -m QMGR -g mqmmqi -n 'SYSTEM.DEAD.LETTER.QUEUE' -t queue -all +put +inq +browse +dsp
Some things to be aware of:
The setmqaut commands shown above will enable users to do almost anything that they could before, except for administrative commands and update-type commands on SYSTEM.* queues. Be aware, though, that only users in the mqm group are allowed to inquire on SYSTEM.AUTH.DATA.QUEUE. This is not a problem for most applications, but it causes saveqmgr (the MS03 SupportPac) to fail when run remotely.
Concerning the script commands, wildcarded parameters should always be quoted. If not, the shell will attempt to expand them using file names in the current directory. If no filenames match, the command is passed in and works as intended. If only one file name matches, the file name is substituted for the queue name, and the setmqaut command is executed but fails silently. If two or more file names match, the setmqaut command will fail outright. This can be a devil to debug because the authorization script may work one day and fail the next, and nothing in the script itself has changed.
In each of the setmqaut commands listed above, the permissions always start with -all because successive commands are cumulative. Running setmqaut once with +dsp and once with +inq results in an effective permission of +dsp +inq. Starting with -all insures that the permissions listed are the only ones in force after the command line is executed. The only other way to insure that unwanted permissions do not creep in is to dump them and compare them to your script. The -all saves that extra step.
Commitment to security
You have seen how to lock down administrative access when strong authentication methods such as SSL or channel security exits are not immediately available. The tradeoff in locking down administrative access in this way is that channels authorize using a single user ID and group for all messages. This is intended for limited use as a workaround only until meaningful authentication can be implemented. Anyone implementing these configurations is strongly advised to do so as the first phase of a longer term plan that ultimately implements a strong authentication mechanism for remote connections.
- Computerworld, WebSphere MQ security in the news
- WebSphere MQ Library
- WebSphere MQ Extended Security Edition
- PartnerWorld, Some IBM Business Partners offer WebSphere MQ security exits
- Understand the XML Data Model
Get products and technologies
- WebSphere MQ Channel security patch
- IBM Support Flashes site, New patches and fixes
- SupportPac IC72: WebSphere BI Brokers - Sample Control Center Security Exits
- SupportPac MS0R: Security Exit
- SupportPac MS81: WebSphere MQ Internet Pass-Thru
- BlockIP2 Security exit with source code
- IY86343: ConnectionName passed to exit or CONNAME displayed in runmqsc is 0.0.0.0