In each column, Mission: Messaging discusses topics designed to encourage you to re-examine your thinking about IBM® WebSphere® MQ, its role in your environment, and why you should pay attention to it on a regular basis.
If you are in retail, you probably already know that PCI DSS is the abbreviation for Payment Card Industry Data Security Standard. If you are not in retail, chances are good that you will know those six letters well before too long, if you donât know them already.
PCI DSS is a security standard for systems that process any kind of payment card transactions. Although PCI DSS applies specifically to the payment card industry, it is certain to have much broader impact. In the absence of a formal standard, the measure of what is considered due diligence or reasonable care often boils down to whatever the prevailing practices are. But what if the prevailing practices are wholly inadequate, as is the case with the vast majority of WebSphere MQ installations? (See why) In this situation, is it reasonable for security architects to design inadequate controls because everyone else is doing it? Or should they turn to an objective standard such as PCI DSS for guidance, even though the industry they are in might not governed by that standard? If I am your shareholder, I am hoping you picked the second option.
According to the PCI Security Standards Council Web site, "the PCI DSS is a multifaceted security standard that includes requirements for security management, policies, procedures, network architecture, software design and other critical protective measures." The standard is based on six principles, each with one or more requirements. Each requirement has a number of sub-requirements. An outline of the requirements and sub-requirements is available in the Security Assessment Procedures document.
I have received a number of e-mails asking how to "make WebSphere MQ compliant with PCI DSS." The answer, of course, is that only a few of the twelve requirements of PCI DSS are technology-focused, so it is impossible to take any product out of context and "make it PCI DSS compliant." However, PCI DSS compliance will definitely influence the configuration of the messaging network, and so in that respect itâs worth exploring what that might look like. In this article, I will discuss some of the PCI DSS requirements to see how they might be applied in a WebSphere MQ context. This area is still emerging and there is not yet a consensus on how these standards might apply to WebSphere MQ -- even within the payment card industry -- but I hope to spark a dialog on the subject. Due to space limitations I will concentrate on only those requirements that are most relevant to WebSphere MQ, starting with Requirement 2.
Requirement 2: Do not use vendor-supplied defaults for system passwords and other security parameters
Things start to get really interesting right out of the gate from a WebSphere MQ standpoint, because the default configuration of WebSphere MQ allows anonymous administrative access. In this case, âother security parametersâ can apply to the channelâs MCAUSER, SSLCIPH, SSLPEER, and SCYEXIT attributes. If any inbound channel (Receiver, Requestor, Cluster Receiver and Server connection channels are inbound) does not have MCAUSER set, then it allows administrative access. If the channel is not also authenticated using SSLPEER or an exit, then the administrative access is anonymous. The most basic security task on any queue manager is to set these values to restrict administrative access. If this is not done, no other security configuration matters. So, if the any channel definition is set to the default settings, all bets are off. The exception is when using a supplemental security package like WebSphere MQ Extended Security Edition, but such measures are usually applied in addition to channel security rather than as a replacement.
Many of the sub-requirements in this section can be applied in a WebSphere MQ context as well. Section 2.2 says to "develop configuration standards for all system components. Assure that these standards address all known security vulnerabilities..." Many of my clients have a wide variety of configurations on their queue managers because they lack a baseline configuration specification. The configurations tend to drift around depending on who built the queue manager and what they thought was a good practice at the time. Because there was no formal baseline, there is rarely any attempt go back and update older builds to the newer configurations, so the gaps widen over time. Iâm glad to see this as a security requirement, but itâs worth doing anyway just to reduce human error and make the network interoperable and consistent.
Section 2.3 calls for non-console administrative access to be encrypted. In a WebSphere MQ context, this would apply to desktop tools, such as WebSphere MQ Explorer or SupportPac M071. If central browser-based tools are in use, this would call for SSL to be used first between the browser and the central server, and then again for the connection from the server to the queue manager. In my prior articles, I have said that an encrypted credential exchange followed by an unencrypted channel session was the bare minimum requirement to secure administrative traffic, and I still stand by this. However, as a growing number of shops apply the PCI DSS standard, the notion of what is considered âdue careâ will undoubtedly change over time to unequivocally require encryption for all remote administrative access. Considering how few WebSphere MQ shops have effectively locked down administrative access, one would hope an objective standard of due care such as PCI DSS would be used to determine reasonableness rather than prevailing practices.
Requirement 3: Protect stored cardholder data
If "cardholder data" doesnât apply to you, feel free to substitute "regulated data," "sensitive data," or "personally identifiable information" as applicable. Section 3.2 says "do not store sensitive authentication data subsequent to authorization (even if encrypted)." This has been a point of concern for many of my clients because they do not have direct control of the ephemeral data used by WebSphere MQ to process messages. We know that the data will most likely be written to the log file if the message is persistent. This means the data sits in the log files until they are overwritten (in the case of circular logs) or are deleted (in the case of linear logs). Many shops actually archive their linear log files and so the data might be retained for years. Some shops use disk mirroring to copy logs and queues for disaster recovery, leaving a viable copy of the data sitting in a data center that is probably not monitored closely.
This requirement raises more questions than I have answers at this point in time. Users have only recently begun to apply these standards in a WebSphere MQ context, and best practices are still emerging. Even so, I have had the opportunity to discuss a few different approaches with clients and the focus was to minimize the chance for messages to be flushed to disk. Here are some of the strategies we looked at:
Use non-persistent messages for data that the standard requires to not be persisted. This eliminates writes to the log files. I have long been a proponent of using non-persistent messages, even for update transactions. This is possible if the state is maintained in the applications and not in the message, and if the applications have the ability to resync with each other. But that is a topic for another article.
Reduce the MAXDEPTH and MAXMSGL on the queue and tune it so that the entire queue can be held in memory. This does not provide a guarantee, but you could make a good case for it being "reasonable care."
Do not hold the transport to a higher standard than the system of record. Okay, I admit this is a bit of a stretch, but more than one shop has decided that if the system of record stores sensitive data, then the transport does not need to adhere to a higher standard. As the administrator of the messaging network, I would make sure my network was as secure as the system of record before taking this stand. If the system of record has access constraints by row and column and my queue manager lets anyone on the intranet to browse the queues, then maybe I do need to hold the transport to a higher standard.
Sections 3.5 and 3.6 call for a number of controls on encryption keys. In WebSphere MQ terms, these are the user certificates and signer certificates stored in the keyrings for each queue manager. In the WebSphere MQ world, where very few shops use SSL, these controls may seem onerous. But as far as the PCI Security Council is concerned, they are the bare minimum required. Section 3.6.4 calls for key replacement yearly, or more often, as deemed necessary. Section 3.6.5 requires a method to securely destroy old keys. Together, these requirements suggest that disciplined key management is a requirement for WebSphere MQ to be considered compliant with the PCI Data Security Standard.
I know of many shops where the administrators and developers are part of the same team and separation of duties is reserved for shops with the luxury of resources. I can imagine their reaction to Section 3.6.6, which calls for separation of duties within the key management process itself such that "it requires two or three people, each knowing only their part of the key, to reconstruct the whole key." For the moment Iâll be happy if the WebSphere MQ community develops a core competency in key management with any number of participants. But if you are in a PCI-regulated shop, you might need to step up on this one. In some cases, this requirement can be partially addressed in hardware such as a DataPower appliance or a crypto-card.
This principle does not address the WebSphere MQ configurations directly so much as the human processes that manage those configurations. This is a very important section because it recognizes that security is not a target configuration but rather a discipline that is practiced regularly and evolves over time. Whenever I pitch a security engagement, I include a skills transfer component in the proposal, and some clients reject it because of this. If they insist they can buy security as a package, install it, and then walk away, I usually refuse to take the engagement because it is very likely to fail.
Requirement 6: Develop and maintain secure systems and applications
There are a number of sub-requirements in this section that directly apply to WebSphere MQ. The very first one, in fact, calls for installation of vendor security patches within 30 days of their release. The last security-specific patch for WebSphere MQ that I recall was Fix Pack 188.8.131.52, which closed a channel vulnerability. If the WebSphere MQ community were to follow the lead of the payment card industry, we would all be at v184.108.40.206 or higher by now. (Fix Pack 220.127.116.11 is current as of this writing.)
Section 6.2 calls for a process to identify newly discovered security vulnerabilities. In other words, someone in the organization should be subscribing to newsletters or RSS feeds in which WebSphere MQ vulnerabilities are announced. IBM announces such things in the My Support news feeds.
Section 6.4 addresses change control. From a WebSphere MQ perspective, changes are typically lumped in with the application releases. Although the applications are usually in compliance with this part of the specification (document the impact of the change, obtain management approvals, document the test results, and provide a backout plan), users frequently do not hold the WebSphere MQ changes to the same standard. For example, the usual method for updating WebSphere MQ configurations is to update the object definition script and rerun it. Very few shops actually keep generations of the object definition scripts, let alone store them in a configuration management system, as might be suggested by this particular PCI DSS requirement.
The requirements addressing this principle are all about authorization. But you cannot have meaningful authorization without also having meaningful authentication. There needs to be some reasonable assurance that you are who you say you are before determining what resources you can access. If the default settings for WebSphere MQ are not updated after installation, the authentication of remote users is only a good faith effort and can easily be bypassed. But it is possible to authenticate remote users at the link level using the native facilities of the product. With this configuration, an ID is assigned to the channel at connection time and lasts for the duration of the session. When the session is from a single user or application, this might be sufficient. But what if the connection is from another queue manager? Or the entire cluster? Is one identity sufficient to authorize all messages from a remote queue manager or the entire cluster? If not, then link-level authentication might not be sufficient. Per-message authentication is possible using WebSphere MQ Extended Security Edition, which cryptographically signs each message that is produced. When the identity is bound to the message, much finer grained access control is possible. This is a case where what we might consider âPCI compliant authenticationâ depends on the network topology and the degree to which applications share queue managers.
Requirement 7: Restrict access to cardholder data by business need-to-know
WebSphere MQâs link-level authentication extends to the API to enable authorization per-queue. This functionality can be extended per-message using WebSphere MQ Extended Security Edition. The key to compliance with this requirement will be to integrate the access controls across the application, the network, and WebSphere MQ.
For example, when using a gateway or hub in combination with channel security, the hub is authorized to all legitimate destinations in the network. Therefore, any message passing through the hub is also authorized to these destinations -- even though it might not have originated from an authorized application or user. Similarly, applications are typically authorized broadly and are expected to mediate the access of individual users when interacting with WebSphere MQ. When access controls are delegated out to the gateway or the application in this fashion, it is necessary to strongly authenticate the connection to be sure it originates with the trusted component (for example, using SSL) at the very least. In some cases, performing authentication and access control on each message might be a better solution to meet this requirement.
Requirement 9: Restrict physical access to cardholder data
One of the most common reasons given for not using SSL on WebSphere MQ channels is that the servers are in a locked data center on the âtrustedâ network. Ignoring for the moment that most security incidents are perpetrated internally, it is interesting to note that the PCI DSS standard takes a broader view of what âphysical accessâ includes. Section 9.1.2 calls for restricting physical access to network jacks. In this case, physical access does not mean the ability to put your hands on a disk drive full of data and walk away with it. Transferring the data to your own disk drive and walking away with that would qualify as physical access, as defined here. This is an important distinction given the number of shops where the queue managers are all accessible from any point in the internal network and thus from a number of relatively unprotected network access points.
Requirement 10: Track and monitor all access to network resources and cardholder data
The first sub-requirement in this section calls for establishment of âa process for linking all access to system components (especially access done with administrative privileges such as root) to each individual user.â Many shops put administrators in the mqm group and have them perform day-to-day tasks under their own user ID. I donât like this practice because it leads to system components running under personal user IDs. There is also a subtle problem that WebSphere MQ will authorize any resource to the primary group of the user who created it. So, if the administratorâs primary ID is staff and he or she creates a new queue manager, then by default it is authorized to the staff group with full control. This might sound like an obscure problem, but I have seen it many times.
What we can do to achieve compliance here is to implement keystroke logging on the mqm account and let administrators assume the mqm account using su, sudo, runas, or the platform equivalent. If a central administration tool is used, it would typically control user access and store the required logs.
Additional requirements in this section call for monitoring significant security events. For WebSphere MQ, this could mean parsing the error log file and/or enabling and monitoring queue manager events.
Requirement 11: Regularly test security systems and processes
This requirement is often unmet in a WebSphere MQ shop. WebSphere MQ is famous for the wonderful quality that once you set it up it runs forever without additional intervention. Unfortunately, a static system gradually becomes more vulnerable as new exploits are discovered. This requirement calls for testing security controls annually and for running vulnerability scans quarterly. Penetration tests are to be run annually or after any significant change, such as the introduction of a new server, upgrading an operating system upgrade, or adding a sub-net. How far is the WebSphere MQ community from meeting this requirement? Currently, there are no recognized standard baselines to scan against, no comprehensive tool kit to perform the scans, and not a lot of demand for these things. Perhaps PCI DSS will help to change that situation.
Requirement 12: Maintain a policy that addresses information security
Most shops have an enterprise-level information security plan by now, so you might think that this requirement has been met. However, one of the sub-requirements is to implement an incident response plan and I do not believe I have ever seen a formal description of what constitutes a WebSphere MQ security incident, much less procedures for how to handle it. In fact, Iâve seen the opposite. One user recently had a rogue application attach to an external service and reuse the local response queue. The existing application and the rogue application started consuming each otherâs response messages. Even though neither application recognized response messages destined for the other, no alarm was raised, and both applications simply discarded the unrecognized messages. In another case, a queue manager was continuously emitting authorization messages because an application was retired and its service account was removed. Automation using the old channel was generating access events using the non-existent account. Rather than shut down the automation properly and delete the channel, the events were disabled.
If you wanted to address the requirement for an incident response plan in a WebSphere MQ context, you would need formal documents describing how to address such things as poison messages, authorization events, denial of service attacks, and so forth. Many shops already have one or more of documents of this type, but often not in a security context. In the case of the client described above, the poison message plan merely called for the application to survive the poison message, but it did not require that the message be saved for forensic analysis or to raise an alarm.
Obviously I havenât covered all the things you might want to look at to address compliance with PCI DSS. If you are subject to that standard and want a deeper dive, a services engagement is probably in order. But for everyone else, I hope this got you thinking about what "reasonable care" or "due diligence" means in a WebSphere MQ context. Most of the mitigations discussed here are not widely practiced in the WebSphere MQ community today. However, that doesnât make them any less reasonable.
I truly believe that in ten years this will no longer be the case. WebSphere MQ administrators will routinely enable SSL, utilize per-message authentication, strongly authenticate users, change the default settings, restrict administrative rights, and perform penetration tests. We will operate as if we are in a hostile environment just as Web administrators do today. But I do wonder how we get there from where we are now. Will it take a series of middleware breaches to get the user community at large to routinely secure their messaging network? Or will we just do it because it is the reasonable thing to do?
Mission: Messaging: End-to-end encryption with WebSphere MQ Advanced Message Security
Payment Card Industry Security Standards Council
PCI DSS specification
and supporting documents
PCI DSS Security Assessment Procedures
Extended Security Edition product information
Center: WebSphere MQ Security V6.0
WebSphere MQ security heats up
Author's Web page: T-Rob.net
Get products and technologies
IBMers' Blog on Messaging
The Vienna WebSphere MQ List server
developerWorks WebSphere MQ forum
T.Rob Wyatt is a Senior Managing Consultant with IBM Software Services for WebSphere who assists customers with the administration, architecture, and security of WebSphere MQ. Recently he has focused on WebSphere MQ security, publishing in the IBM WebSphere Developer Technical Journal and presenting at the IMPACT and European Transaction and Messaging conferences. T.Rob also hosts The Deep Queue, a monthly WebSphere MQ security podcast.