Integrating secure ATM banking systems using WebSphere Message Broker


A bank ATM debit transaction is any transaction that requires the customer to enter a Personal Identification Number (PIN). For security reasons, the ATM servers usually require the PIN code to be encrypted by a two-level encryption process before being entered into any card or transaction authentication process. The encryption process is usually done by a secure hardware system.

IBM has a high-security, high-throughput cryptographic subsystem that facilitates any encryption required by the ATM system: the 4765 Cryptographic Security Module validated to FIPS 140-2, Overall Level 4 (highest level of security). Although integrating the 4765 Cryptographic Security Module is a simple process, you would sometimes have to integrate with a third -party encryption system that would need more programming effort. Nevertheless, IBM® WebSphere® Message Broker (hereafter called Message Broker) has the capabilities to facilitate it. In this article, an IST/Switch is used as the ATM system, and a third -party Host Security Module (HSM) appliance is used as the encryption appliance.

The HSM generally provides a variety of functions to implement key management, PIN management and verification, and Message Authentication Code (MAC) processing. In the underlying application, Message Broker calls two HSM commands to encrypt the PIN and generate a 16-digit PIN block. Then it uses this PIN block as input to any debit process that runs on the ATM system. Figure 1 shows the integration architecture for the secure ATM system:

Figure 1. Integration architecture for secure ATM system
integration architecture for secure ATM system
integration architecture for secure ATM system

Section 1, "Integrating the HSM" shows you how to integrate Message Broker with the HSM Appliance, and describes issues and recommendations when dealing with the HSM. Section 2, "Integrating the IST/Switch" describes the integration of the ATM system (the IST/Switch), and includes troubleshooting information.

1. Integrating the HSM

The HSM appliance used in this article is the Thales HSM 8000. Generally, the HSM acts as a peripheral to the host computer and performs cryptographic processing in a physically secure environment on behalf of the Host. The processing is performed by the HSM in response to commands that it receives via a TCP/IP connection. In this scenario, the HSM appliance is used for PIN encryption.

This section shows you how to communicate with the appliance to run commands from within Message Broker. You will need to run two commands to get an encrypted PIN block of 16 hexadecimal digits using a 4-digit clear PIN code in two steps. Figure 2 shows the Message Broker subflow that calls a single HSM command:

Figure 2. Message Broker subflow for HSM command execution
Message Broker subflow for HSM command execution
Message Broker subflow for HSM command execution

The first compute node in the flow prepares the HSM command request message and formats it in a bit stream format of type BLOB to be outputted to the TCPIP Client Output to HSM node. Listing 1 below shows the Embedded SQL (ESQL) statements to formulate that request message:

Listing 1. Request code for first HSM command
CREATE LASTCHILD OF OutputLocalEnvironment DOMAIN('MRM'); 
SET OutputLocalEnvironment.MRM.MsgLength = X'0017'; --23 chars Length
SET OutputLocalEnvironment.MRM.HSMMessageHeader = SUBSTRING(rEnv.MsgUId FROM 1 FOR 4);
SET OutputLocalEnvironment.MRM.HSMCommandCode = 'BA';
SET OutputLocalEnvironment.MRM.HSMInputPIN = rEnv.CardPIN;

-- <<<<<<<< Take the 12 digits before rightmost digit (the check digit) >>>>>>>>>>>
SET OutputLocalEnvironment.MRM.HSMAccountNum = 
    SUBSTRING(rEnv.CardNum From(Len - 12) FOR 12); 

SET BlobBitstream = ASBITSTREAM(OutputLocalEnvironment.MRM 
    OPTIONS options 
    TYPE 'HSMMessage_Request1'
    FORMAT 'Text1');

As shown above, the command uses the code BA, which corresponds to the Encrypt a Clear PIN command. The first parameter after the command code is the clear text PIN, left-justified and padded with hexadecimal F to the encrypted PIN length. The second parameter is the 12 rightmost digits of the account number or card number, excluding the check digit (the rightmost digit).

The last statement in Listing 1 generates the request message in its final format to be sent to the HSM. It includes the bitstream of the command message. This bit stream generation keeps the length (first message element) in its binary hexadecimal format, while the other message elements are in the ASCII code of the characters (CCSID 1208).

For this scenario, you must create a data model in a message set that in turn will handle the data types, data lengths, and padding of the message elements. The MRM parser is used to generate a fixed-length message in a physical text format.

The length of the response message to this command will vary depending on the success or failure status, and therefore Message Broker reads it in two steps. In the first step, it reads only the first four characters that contain the hexadecimal length of the message. Then the Get Length compute node sets this length in the appropriate environment variable to dynamically define the length of the record read by the second TCP/IP Client Input node ReceiveFromHSM2. For details on how to program the variable-length receiving TCP/IP nodes, and for information on dealing with TCP/IP connections and ISO 8583, see the developerWorks article Integrating with TCP/IP using WebSphere Message Broker.

Finally, the record is read and parsed using a PARSE function according to the data model in the message set, as shown in Listing 2:

Listing 2: Parsing HSM response
CREATE LASTCHILD OF Environment.Variables.CIB.HSMResponse DOMAIN('MRM')
    TYPE Environment.Variables.CIB.HSMResponseMsgName
    FORMAT 'Text1');

The result of this command is a five-character encrypted PIN using the input four digits, and you use this encrypted PIN as the input to the second command. You run the second command the same way as the first command, with a command code of JG. It returns an encrypted PIN block of 16 hexadecimal characters to be used in the request message of the IST/Switch.

Issues and recommendations when dealing with HSM

Here are some issues that a development team will face when integrating the HSM:

  • Many parameters are defined when the HSM is installed, so you should retrieve them from the HSM owner for use during development. For example, Encrypted PIN length and Message header length are two key parameters in almost all commands, and the HSM owner defines them at HSM installation time.
  • Formats for some of the parts of the request and response messages are not explicitly mentioned in the documents, and it would take a long time or even trial-and-error to be detected.
  • For example, one of the key message parts is the message length and format. The information on this message element was not clear in any of the supporting sources of information. The result of the investigations revealed that, in addition to this element's existence, the element length is two bytes in a binary hexadecimal format, and those two bytes are sent via TCP/IP in their hexadecimal binary format, not in ASCII-coded characters like the other parameters.

2. Integrating the IST/Switch

IST/Switch from FIS Corporation is a system that supports most of the transactions required to implement an Electronic Funds Transfer / Point of Sale (EFT/POS) network. For more information on IST/Switch, see the IST/Switch product page on the FIS Corp. web site. IST/Switch communicates with the host using the ISO 8583 standard over TCP/IP. For more information on ISO 8583 and on using Message Broker for such TCP/IP communication, see the developerWorks article Integrating with TCP/IP using WebSphere Message Broker.

The ISO 8583 standard defines several message classes that determine the type of transaction being requested. For this article, one of the financial message class was used -- the message with Message Type Identifier (MTI) value of 0200. This message request is used to request a financial authorization for a transaction. A 0210 message is the MTI of the response message to the 0200 message. Figure 3 below shows the Message Broker flow developed for the entire financial transaction authorization process. It calls the HSM subflow twice to run the two encryption commands, then uses this result to prepare the IST/switch ISO 8583 request message:

Figure 3. Financial authorization flow
Financial authorization flow
Financial authorization flow
Listing 3. ESQL code to create IST/Switch ISO 8583 request message
CREATE LASTCHILD OF OutputLocalEnvironment DOMAIN('MRM'); 
--<<<<<Generating the binary bitmap using the element number list. >>>>>>>
SET Items.a[] = LIST{3,4, 7, 11, 12, 13, 18, 22, 35, 37, 41, 43, 49, 52}; 
DECLARE MessageLength CHAR '168';
DECLARE BitmapBit BIT CAST ('00000000' AS BIT); 
CREATE LASTCHILD OF OutputLocalEnvironment.MRM NAME 'PrimaryBitmap';
DECLARE rTargetMrm REFERENCE TO OutputLocalEnvironment.MRM.PrimaryBitmap;
CALL CreateBitMaps(rItems , rTargetMrm, BitmapBit);
SET OutputLocalEnvironment.MRM.MsgLen = MessageLength;
SET OutputLocalEnvironment.MRM.MTI = MTI;
SET OutputLocalEnvironment.MRM.BinaryBitmap = BitmapChar; --Set the right value.

SET OutputLocalEnvironment.MRM.ProcessingCode = '303000';
SET OutputLocalEnvironment.MRM.AmountTransaction = '000000000000';
SET OutputLocalEnvironment.MRM.TransmissionDateAndTime = 
SET OutputLocalEnvironment.MRM.SystemsTraceAuditNumber = 
    SUBSTRING(rEnvesbXML.MsgUId From(Len - 5) FOR 6);
SET OutputLocalEnvironment.MRM.TimeLocalTransaction = 
SET OutputLocalEnvironment.MRM.DateLocalTransaction = 
SET OutputLocalEnvironment.MRM.MerchantType = '6011'; 
SET OutputLocalEnvironment.MRM.PointOfServiceEntryMode = '901';
SET Len = LENGTH(rEnvesbXML.CardNum);
SET OutputLocalEnvironment.MRM.Track2Data.Length = Len; 
SET OutputLocalEnvironment.MRM.Track2Data.Track2Data = rEnvesbXML.CardNum; 
SET OutputLocalEnvironment.MRM.RetrievalReferenceNumber = 
    SUBSTRING(text FROM 2) || OutputLocalEnvironment.MRM.SystemsTraceAuditNumber;
SET OutputLocalEnvironment.MRM.CardAcceptorTerminalIdentification = '00000000';
SET OutputLocalEnvironment.MRM.CardAcceptorNameLocation = 
SET OutputLocalEnvironment.MRM.CurrencyCodeTransaction = '818'; 
SET OutputLocalEnvironment.MRM.PersonalIdentificationNumberData = 
SET BlobBitstream = ASBITSTREAM(OutputLocalEnvironment.MRM OPTIONS options 
    SET 'ISO_ATM_HSM_MsgSt' 
    TYPE 'ISO8583Message_64_Write' 
    FORMAT 'Text1');
SET BlobBitstream = SUBSTRING(BlobBitstream FROM 65);

Dealing with IST/Switch

As described above for the HSM, retrieve the definition of the ISO 8583 custom header from the IST/Switch owner. For the underlying case, it is four bytes that contain the entire message length in characters.

The key success parameter of the ISO 8583 communication is the correctness of the bitmap values so that they reflect the enabled message elements. This is the condition for correct parsing at the receiving side. For this purpose, a spreadsheet tool has been developed that automatically transforms field selections into a hexadecimal bitmap of 16 characters for the sending side. For the other direction, the receiving side, it transforms the bitmap hexadecimal values to the corresponding enabled elements of the 128 elements (in case of primary and secondary bitmaps) for use in reviewing the parsed message at development time. This technique reduced the probability of sending or parsing incorrect bitmap values during the investigation phase.

For the production phase, it is important to develop ESQL code to automate the generation of both the bitmaps and the ISO request in addition to parsing the received message. In the underlying case, this code uses both the ISO element indices to be used in the request and their total length as inputs, and generates both the corresponding binary bitmap and the entire request in its right format. Listing 3 above shows the code part pertaining to the request message. The function CreateBitMaps() creates both the 64-integer-element bitmap tree and the binary 64-bit bitmap element using indices of the ISO field elements to be used in the request.

An IBM reusable asset containing a message set of a single bitmap (64 message elements) has been used as the core part in this work. This message set is used in parsing the received 64-element ISO 8583 messages. It is enough for any application where the ATM system responses will not exceed the primary bitmap to the secondary one. Otherwise, you have to use an extended copy of the message set that contains two bitmaps and the 128-element message. For details of the 128 elements of the ISO 8583 message, see ISO 8583 data elements.

To send the ISO 8583 messages, you have to add another message to the message set. It should contain the 64-integer-element primary and secondary bitmap elements at the beginning, so that its bitstream part can be easily removed before sending, as they are used only as the number of occurrence references to the message elements at creation time (the last statement in Listing 3 above removes this part). Following those two elements is the entire actual message that will be sent to the IST/Switch, as listed below and shown in Figure 4:

  • The four-character header that contains the entire message length in characters -- MsgLen
  • The MTI (0200 for this case)
  • The binary bitmap that reflects the existence of the different message elements
  • The ISO message elements
Figure 4. ISO 8583 message model for writing ISO request
 ISO 8583 message model for writing ISO request
ISO 8583 message model for writing ISO request

Using a double-bitmap (128 message fields) ISO 8583 message type to parse a single-bitmap response message does not work properly. The reason is that the MRM parser cannot calculate the number of occurrences for the elements related to the secondary bitmap, as it depends on the bitmap relevant bit value, whereas the bitmap itself (the secondary) has a zero number of occurrences (disabled). The solution to this problem is to use a 64-fields message type to parse the single-bitmap received messages and a 128-field message to parse the double-bitmap messages (see the dynamic message type selection in Listing 4 below).

In Listing 3 above, the function stretchBitMap checks every bit in the binary bitmap received from the IST/Switch and fills the value of the corresponding integer element of the message bitmap data complex type, where each integer element of that local bitmap object corresponds to a bit of the binary bitmap. This reflection of the binary bitmap in integer elements enables you to use them as references to the number of occurrences of the message elements.

Listing 4: IST/Switch ISO 8583 response parsing code
CREATE LASTCHILD OF OutputLocalEnvironment DOMAIN ('MRM');
DECLARE rTargetMrm REFERENCE TO OutputLocalEnvironment.MRM;
SET BlobBitstream = stretchBitMap(rTargetMrm, BitmapBit, 'PrimaryBitmapS');
DECLARE MsgType CHAR 'ISO8583Message_64'; --The 64 element Msg.
IF OutputLocalEnvironment.MRM.Bit1 = 1 THEN    -- If secondary bitmap is enabled
    SET MsgType = 'ISO8583Message_128';    -- The 128 element Msg.
    SET readLen =16; 
    SET tmpMap = CAST(CAST(SUBSTRING(InMessageBlob FROM readIndx FOR readLen) 
        AS CHAR CCSID 1208) AS BLOB);
    SET readIndx = readIndx + readLen;
    SET BitmapBit = CAST(tmpMap AS BIT);
    SET OutputLocalEnvironment.MRM = NULL;
    CREATE LASTCHILD OF OutputLocalEnvironment DOMAIN('MRM');
    DECLARE rTargetMrm REFERENCE TO OutputLocalEnvironment.MRM;
    SET BlobBitstream = BlobBitstream || stretchBitMap(rTargetMrm, BitmapBit, 
SET BlobBitstream = BlobBitstream || SUBSTRING(InMessageBlob FROM readIndx);
    SET 'ISO_ATM_HSM_MsgSt'  
    TYPE MsgType
    FORMAT 'Text1');


This article described banking ATM debit card security systems from the business integration perspective, and showed you how to use Message Broker's powerful features to create secure ATM systems by integrating the HSM and the IST/Switch systems. It described the important Message Broker message flows of the integration code as well as the important parts of the associated ESQL code. The article also touched on best practices based on field experience in order to avoid potential issues and facilitate potential investigations for business integration developers, technical leads, architects, and business analysts involved in integrating similar ATM security systems.

Downloadable resources

Related topics


Sign in or register to add and subscribe to comments.

ArticleTitle=Integrating secure ATM banking systems using WebSphere Message Broker