International standards are used to make industries more efficient and effective by providing specifications for products and services to ensure that they are safe, reliable, and of good quality. ISO 20022 is the platform proposed by the International Organization for Standardization (ISO) to develop all financial messages. However, one challenge to the proposal of using ISO 20022 standards for inbound and outbound messages is when existing data models or messaging formats already in place do not easily conform to the standards and structures imposed by the ISO messages.
This article provides an overview of ISO 20022 message formats and demonstrates how they can be implemented in solutions using data mapping techniques and tooling. The information in this article is applicable to business analysts, solution architects, and subject matter experts working in the payment processing industry.
ISO 20022 overview
ISO 20022 can be considered to be a universal financial industry message framework and is the platform proposed by ISO for developing all financial messages. ISO 20022 is a multipart standard, developed and maintained by the ISO Technical Committee responsible for standardization in the field of banking, securities, and other financial services (ISO/TC 68). ISO 20022 does not describe the messages themselves; it describes a common platform for the development of standardized messages using:
- A modelling methodology (based on UML) to capture the business transactions and associated message flows across financial business areas in a syntax-independent way.
- A central dictionary of business terms commonly used in communications between financial institutions.
- A set of XML design rules to convert the messages described in UML into XML schemas. It should be noted that XML is not a prescribed message syntax, but it is often preferred and is the syntax used in this article.
ISO 20022 realizes end-to-end processing across domains and geographies that currently use vastly different standards and information formats. Financial institutions exchange large amounts of information with other financial institutions and with their customers. Such exchanges only work if the sender and receiver of a message have a common understanding of how to interpret this information. The ability to map different messaging standards is an important aspect of interoperability across the industry. It allows for the seamless execution of a business process by various counterparts with different levels of automation and time-to-market requirements or capacity.
The model described above enables the financial services industry to hold the basic foundations of their business in a centralized data dictionary and derive the financial messaging that they use. The modelling methodology decouples the business rules from the format of the physical message being exchanged. The models evolve to meet changing business needs while the message formats evolve to take advantage of the latest innovations in technology. This enables the industry to leverage current technologies and to adapt quickly to future change using the flexibility that is inherent in the ISO 20022 standard.
As a standard, ISO 20022 supports convergence and co-existence. The use of a single standard is desirable in the long term, but in the interim a number of standards continue to co-exist. One of the big catalysts for the adoption of ISO 20022 in the payments arena is the Single Euro Payments Area (SEPA), which is currently replacing domestic retail credit transfers and direct debits with standardised European payments that use ISO 20022 messages. From February 1, 2014, the European Commission has mandated that SEPA replace all local payment systems. The consequence of this is that after that date the overall majority of regular payments and collections in the euro zone must be processed as SEPA payments. For banks, it is mandatory to use ISO 20022 in interbank SEPA communication. (It should be noted that ISO 20022 is not obligatory for other parties that want to use the bank's services.) A key driver for this is that many current formats are unable to support some of the requirements for SEPA such as the provision of a mandatory IBAN value.
There are a number of appealing advantages in adopting ISO 20022 for SEPA:
- It supports improved straight through processing (STP), which means the automatic processing of an order without human intervention.
- Because the entire chain in a transaction uses a uniform format for payments, processing time is shorter and the likelihood of errors is lower.
- ISO 20022 is also a particularly transparent format and is human-readable. Tracing and solving errors or inconsistencies is simpler.
- Almost all large banks accept the standard. Therefore, the number of payment formats can be reduced considerably, which will reduce the maintenance costs of these formats.
- It improves the quality of data exchanged between financial institutions by introducing structure and consistency to the data dictionary.
The SEPA data formats are applicable for two SEPA payment types: SEPA Credit Transfer (SCT) and SEPA Direct Debit (SDD). The standards for these schemes facilitate payment initiation, processing, and other functions such as reconciliation or refunds in line with the Rulebooks created by the European Payments Council.
Implementation of the SEPA payment schemes uses a number of ISO Message types, which are described more fully in the following sections.
ISO payment messages
In the ISO context, the standard describes the agreement on what information is expressed, while the syntax is the format or the language used to express that information. A message definition provides a clear classification of the information and data formats (field lengths, codes, character sets) that can be exchanged between parties and can be looked at logically as a description of what data is exchanged in the message, its structure, and what it means. These logical definitions can be mapped to the business definitions defined in ISO 20022. Although ISO does not dictate the syntax of the messages, XML is the most widely used syntax for message specifications, and XML message schemas are derived from the ISO UML message models.
ISO 20022 messages are available for the complete end-to-end payments chain: customer-to-bank (payment), bank-to-bank (payment clearing and settlement), and reporting (cash management). These financial message definitions are divided into business areas (these are well-recognised functional domains in the industry) and are identified by business area codes (4 characters). These codes are:
- acmt – Account Management
- admi – Administration
- caaa – Acceptor to Acquirer Card Transactions
- camt – Cash Management
- catm – Terminal Management
- pacs – Payments Clearing & Settlement
- pain – Payments Initiation
- reda – Reference Data
- seev – Securities Events
- semt – Securities Management
- sese – Securities Settlement
- setr – Securities Trade
- trea – Treasury
- tsmt – Trade Services Management
- tsin – Trade Services Initiation
The message flows that represent a particular business transaction comprise messages from the business areas defined above. For example, a customer-initiated direct debit initiation message (pain.008.001.03) will result in a customer payment status report message (pain.002.001.04). Full details of the message flows for all business areas can be found in the ISO Payments Messages (see Resources) section of the ISO 20022 website.
The following section describes the scenario and use of ISO 20022 Customer Direct Debit Initiation messages.
Sample direct debit messages
A direct debit or direct withdrawal is a financial transaction in which one person (creditor) withdraws funds from another person's (debtor) bank account. Formally, the person who directly draws the funds (the payee) instructs his or her bank to collect (that is, debit) an amount directly from another's (the payer's) bank account designated by the payer and pay those funds into a bank account designated by the payee. Direct debits result in cash transfers between debtors and creditors through infrastructures or correspondent banks. They can be exchanged as single instructions, but are traditionally grouped following some common characteristics and typically exchanged in a batch mode for efficiency.
Using ISO standards, a Payment Initiation (pain.008.001.03) message is sent by an initiating party to the forwarding agent or creditor agent. It is used to request a single collection or bulk collections of funds from the various debtor's accounts for a creditor.
Figure 1 (see larger image) shows a sequence diagram provided by ISO to describe the positive scenario in initiating a direct debit. In the scenario:
- The CustomerDirectDebitInitiation message is sent by the initiating party (Creditor) to the forwarding agent or creditor agent. It is used to request single or bulk collections of cash from one or various debtor accounts to a creditor.
- The FItoFICustomerDirectDebit message is sent by a financial institution to another financial institution, directly or through a clearing system. It is used to clear direct debit instructions initiated by non-financial institution customers.
- The positive CustomerPaymentStatusReport message (single or grouped) and FIToFIPaymentStatusReport message (single or grouped) is sent by the receiver of an instruction to inform the receiver that the instruction has been received and can be processed.
Figure 1. Sample direct debit message
Listing 1 shows a customer direct debit initiation message sent by the Smith insurance company to its account servicer, AAAAUS29. AAAAUS29 offers a special service to Smith, under agreement VERPA-1. The direct debit initiation message contains a single collection instruction. This instruction is for 985 GBP to be collected from account 789101, owned by debtor Lee and serviced by agent CCCCUS27. This is a one-off direct debit, which has notified Lee on 8 June 2013, with reference SMITH2435/2012. The payment is for a car insurance premium. The requested collection date is 13 July 2012, and charges related to the handling of the instruction will be shared between Smith insurance company and the debtors.
Listing 1. Customer direct debit initiation message
<?xml version="1.0" encoding="UTF-8"?> <?valid true?> <?description Sample Message?> <Document xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:iso:std:iso:20022:tech:xsd:pain.008.001.03"> <CstmrDrctDbtInitn> <GrpHdr> <MsgId>CAVAY1234</MsgId> <CreDtTm>2013-06-02T14:25:00</CreDtTm> <NbOfTxs>1</NbOfTxs> <CtrlSum>985</CtrlSum> <InitgPty> <Nm>Smith</Nm> <PstlAdr> <StrtNm>Virginia Lane</StrtNm> <BldgNb>3</BldgNb> <PstCd>W12 8QT</PstCd> <TwnNm>London</TwnNm> <Ctry>GB</Ctry> </PstlAdr> <CtctDtls> <Nm>J. Thompson</Nm> <EmailAdr>Thompson@Smith.com</EmailAdr> </CtctDtls> </InitgPty> </GrpHdr> <PmtInf> <PmtInfId>CAVAY/88683</PmtInfId> <PmtMtd>DD</PmtMtd> <BtchBookg>false</BtchBookg> <ReqdColltnDt>2013-07-13</ReqdColltnDt> <Cdtr> <Nm>Smith</Nm> <PstlAdr> <StrtNm>Virginia Lane</StrtNm> <BldgNb>3</BldgNb> <PstCd>W12 8QT</PstCd> <TwnNm>London</TwnNm> <Ctry>GB</Ctry> </PstlAdr> </Cdtr> <CdtrAcct> <Id> <Othr> <Id>789123</Id> </Othr> </Id> </CdtrAcct> <CdtrAgt> <FinInstnId> <BICFI>AAAAUS29</BICFI> </FinInstnId> </CdtrAgt> <DrctDbtTxInf> <PmtId> <EndToEndId>AY090327/456</EndToEndId> </PmtId> <PmtTpInf> <InstrPrty>NORM</InstrPrty> <SvcLvl> <Prtry>VERPA-1</Prtry> </SvcLvl> <SeqTp>OOFF</SeqTp> </PmtTpInf> <InstdAmt Ccy="GBP">985</InstdAmt> <ChrgBr>SHAR</ChrgBr> <DrctDbtTx> <PreNtfctnId>Smith2435/2013</PreNtfctnId> <PreNtfctnDt>2013-06-08</PreNtfctnDt> </DrctDbtTx> <DbtrAgt> <FinInstnId> <BICFI>CCCCUS27</BICFI> </FinInstnId> </DbtrAgt> <Dbtr> <Nm>Lee</Nm> <PstlAdr> <StrtNm>Cross Road</StrtNm> <BldgNb>45</BldgNb> <PstCd>E56 7HY</PstCd> <TwnNm>London</TwnNm> <Ctry>GB</Ctry> </PstlAdr> </Dbtr> <DbtrAcct> <Id> <Othr> <Id>789101</Id> </Othr> </Id> </DbtrAcct> <RmtInf> <Ustrd>CAR INSURANCE PREMIUM</Ustrd> </RmtInf> </DrctDbtTxInf> </PmtInf> </CstmrDrctDbtInitn> </Document>
In response, a positive customer payment status report message (pain.002.001.04) is sent to acknowledge that the message passed validation and was accepted based on the customer profile.
Listing 2. Customer payment status report message
<?xml version="1.0" encoding="UTF-8"?> <?valid true?> <?description Sample CustomerPaymentStatusReport to acknowledge that the message passed validation and was accepted based on the customer profile.?> <Document xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:iso:std:iso:20022:tech:xsd:pain.002.001.04"> <CstmrPmtStsRpt> <GrpHdr> <MsgId>BBBB/120928-PSR/001</MsgId> <CreDtTm>2013-09-28T14:09:00</CreDtTm> <InitgPty> <Nm>Smith</Nm> <PstlAdr> <StrtNm>Virginia Lane</StrtNm> <BldgNb>3</BldgNb> <PstCd>W12 8QT</PstCd> <TwnNm>London</TwnNm> <Ctry>GB</Ctry> </PstlAdr> </InitgPty> <DbtrAgt> <FinInstnId> <BICFI>AAAAUS29</BICFI> </FinInstnId> </DbtrAgt> </GrpHdr> <OrgnlGrpInfAndSts> <OrgnlMsgId>CAVAY1234</OrgnlMsgId> <OrgnlMsgNmId>pain.008.001.03</OrgnlMsgNmId> <OrgnlCreDtTm>2013-09-28T14:07:00</OrgnlCreDtTm> <OrgnlNbOfTxs>1</OrgnlNbOfTxs> <OrgnlCtrlSum>985</OrgnlCtrlSum> <GrpSts>ACCP</GrpSts> </OrgnlGrpInfAndSts> </CstmrPmtStsRpt> </Document>
Depending on the scenario, an initiation request may also be rejected, and this is specified in the response message, which includes additional details and a transaction status of 'Reject.'
All positive and negative scenarios (business flows) are documented in the ISO 20022 payments model.
The above message flow and sample messages describe only the process of receiving and responding to messages sent in an ISO 20022 format. Typically, the message needs to be transformed into an internal system format and mapped to a data model. The process of performing this mapping so that the payment initiation request can be processed is described in the next section.
ISO data mapping
Although ISO messages can be used for message exchanges between the sender and receiver of a message both externally and internally in the financial institutions, it is likely that a mapping from ISO to other message formats or to an internal data store is required at some point. The ability to map different messaging standards is an important aspect of interoperability across the industry. It allows for the seamless execution of a business process by various counterparts with different levels of automation and time-to-market requirements or capacity.
The process for the data mapping exercise maps the source data elements to the target. In this article, we discuss the data mapping process between ISO 20022 payment initiation messages and a sample transactional system data model.
You can use InfoSphere® Data Architect as a tool to help model, relate, and integrate data into a design based on the data mapping for ISO 20022. InfoSphere Data Architect can create logical, physical, and domain models, where elements can be visually represented in standard UML diagrams. It leverages data model value across the data lifecycle. In this article, InfoSphere Data Architect is used as the tool to demonstrate the data mediation and data relationship steps that are part of the overall data mapping exercise.
The high-level steps in the data mapping workflow are:
- Define data mediation between two sources (ISO message and the transactional system data model)
- Run data analysis to identify possible relationships and data types
- Follow-up analysis to identify indirect relationships and re-purpose data attributes for complete data flow
- Define system interface between data source and destination
The first step for the data mapping process is the data mediation stage, where you are mapping from internal data sources in the client's transactional systems to ISO messages. When the data mapping is indirect using a mediating data model, the process is also called data mediation.
During the stage you identify the major data sources. Input data is referred to as data sources, and, typically, comes from a client's main transactional system. The aim is to map the data to the appropriate ISO XML message constructs.
In this article, it is assumed that the relevant data sources already exist and have been identified (including metadata). If there's an existing data warehouse, then the sources will already be known, and the project may not need anything beyond that.
On the other hand, there are scenarios where the data model may need revision. If the data model change is not possible, a data wrapping layer could be considered as an option where the data sources are aggregated into a single layer.
The example in Figure 2 shows data mediation between two entities, where the mediation data model that will facilitate the data mapping between the ISO messages and the transactional system has been identified and modelled. Built in InfoSphere Data Architect, the graphical designer enables users to easily mediate new data sources or to modify existing mediation jobs. In many cases, the creation of a job can be as easy as dragging and dropping existing stages into a connected series.
Figure 2. Data mediation
In this example, the data destination is the Payment Instruction entity, where all key data attributes are defined. A relationship between two entities demonstrates a flow during the data mediation step. Each data set is represented as a single data entity. The ISO Message Document entity contains the XML message header and the XML document payload. The Payment Instruction data entity contains data attributes defined in the transactional system data model that are used for payment processes scoped within the system.
The incoming XML message may contain multiple transactions within the XML payload. This is shown as a '0 or 1 to many' relationship with the Payment Instruction entity. The '0' covers a case when the message validation failed and resulted in no Payment Instruction defined in the incoming XML message.
In the previous example, the entire XML document payload will be stored in the ISO Message Document data entity for audit and validation purposes. The main XML fields from the data source, such as Message Name Identifier, Message Creation Date, and so on, are stored as data attributes for the entity. This facilitates any payment response message, where the attributes need to be sent back to the payment instruction originator. Depending on business requirements, a decision needs to be made which fields should be stored as a single attribute. Examples are message identification, number of transactions, and the group message header.
After the ISO message business rules validation is complete, the transactional system-defined fields are stored in the Payment Instruction entity. As part of the Data Mediation step, you define attributes that will be required to facilitate the payment process for the system. As part of this process, the following items need to be considered: business and functional requirements, local legal regulations for payment transactions, data retention and data compliance rules, transactional system auditing, the transaction response code or status (some data should be sent back to the data source originator as part a response), and so on.
The purpose of this task is to identify the relationships between the mediation data model and the ISO messages.
A relationship level represents a high-level view of data sources or the mediation data model and establishes the foundation from which data mapping will progress. You can use InfoSphere Data Architect to create the high-level view.
Note: Structural variations are variations in type or structure of the data entities. Different types typically have some common but also different additional attributes.
During this stage, the following steps are required:
- Identify data types, including transformation
- Identify variations in structure
- Identify commonality across structural variations
- Identify meta-model based on variations in structure
If the source and destination data types do not match, transformation logic needs to be implemented. This should also include validation rules based on the ISO message schema. Examples could be different string length, pre-defined string values, and so on.
You can use InfoSphere Data Architect to create a data relationship between the ISO 20022 message and the transactional system data model. Figure 3 (see larger image) demonstrates a relationship between pain.008 (direct debit instruction) and ISO Message Document and Payment Instruction attributes.
Note: From InfoSphere Data Architect's perspective, the data model is a centric source to external data entities for data mapping and does not reflect the data flow. In the following examples, the data model is considered to be the data source. However, from the data flow perspective, the data source is the ISO pain.008 message.
Figure 3. Data Mapping (ISO Message Document) for pain.008 message
Figure 4. Data Mapping (Payment Instruction) for pain.008 message
See a larger version of Figure 4.
Figure 5 (see larger image) shows how some data fields received as part of an original message can be communicated back to the message originator in the form of the message response or payment processing result (pain.002).
Figure 5. Data Mapping for pain.002 message
Data mapping represents a relationship between fields of the ISO message and the data model. This includes the attribute's type, length, and validation rules.
Table 1 shows data mapping for the pain.008 message and the transactional system data model. InfoSphere Data Architect can produce similar output in a CSV file format.
Table 1. Data mapping for pain.008 message
|ISO path||XML tag||ISO data type||Data entity||Data attribute||Data type|
|Document/@xmlns||Map the content of the namespace after xsd: to the Message Name (required for response) e.g. pain.008.001.03||ISO Message Document||Message Name ID||CHAR(35)|
|CstmrDrctDbtInitn/GrpHdr/MsgId||MsgId||Max35Text||ISO Message Document||Message Identifier||CHAR(35)|
|CstmrDrctDbtInitn/GrpHdr/CreDtTm||CreDtTm||ISODateTime||ISO Message Document||Message Creation Date||DATE|
|CstmrDrctDbtInitn/GrpHdr/NbOfTxs||NbOfTxs||Max15NumericText||ISO Message Document||Number of Transactions||NUMERIC(15,0)|
|CstmrDrctDbtInitn/GrpHdr/InitgPty/Id/OrgId/Othr/Id||Id||Max35Text||ISO Message Document||Initiating Generic Organisation ID||CHAR(35)|
|CstmrDrctDbtInitn/PmtInf/PmtMtd||PmtMtd||PaymentMethod2Code||N/A (Mandatory value of 'DD' for pain.008 transactions)|
|CstmrDrctDbtInitn/PmtInf/PmtTpInf/CtgyPurp/Cd||Cd||ExternalCategoryPurpose1Code||Payment Instruction||Payment Type Identifier||VARCHAR(3)|
|CstmrDrctDbtInitn/PmtInf/ReqdColltnDt||ReqdColltnDt||ISODate||Payment Instruction||Requested Date||DATE|
|CstmrDrctDbtInitn/PmtInf/CdtrAcct/Id/Othr/Id||Id||Max34Text||ISO Message Document||Creditor Account Identifier||CHAR(35)|
|CstmrDrctDbtInitn/PmtInf/CdtrAcct/Id/IBAN||IBAN||IBAN2007Identifier||Payment Instruction||Creditor Account IBAN||CHAR(34)|
|CstmrDrctDbtInitn/PmtInf/CdtrAgt/BrnchId/Id||Id||Max35Text||Payment Instruction||Creditor Account Branch Identifier||CHAR(35)|
|CstmrDrctDbtInitn/PmtInf/DrctDbtTxInf/InstdAmt||InstdAmt||ActiveOrHistoricCurrencyAndAmount||Payment Instruction||Instructed Amount||NUMERIC(23,5)|
|CstmrDrctDbtInitn/PmtInf/ DrctDbtTxInf/ DbtrAgt||DbtrAgt||BranchAndFinancialInstitutionIdentification5||N/A (Element required in XML but embedded detail not required)|
|CstmrDrctDbtInitn/PmtInf/DrctDbtTxInf/ Dbtr||Dbtr||PartyIdentification43||N/A (Element required in XML but embedded detail not required)|
|CstmrDrctDbtInitn/PmtInf/DrctDbtTxInf/ DbtrAcct/Id/Othr/Id||Id||Max35Text||N/A (Element required so Dummy Value expected. Value must be 'GCP Debtor Account')|
|CstmrDrctDbtInitn/PmtInf/ DrctDbtTxInf/ InstrForCdtrAgt||InstrForCdtrAgt||Max140Text||Payment Instruction||Product ID||CHAR(3)|
Table 2 shows the data mapping for the pain.002 message and the transactional system data model.
Table 2. Data Mapping for pain.002 message
|ISO path||XML tag||ISO data type||Data entity||Data attribute||Data type|
|CstmrPmtStsRpt/GrpHdr/MsgId||MsgId||Max35Text||ISO Message Document||Transaction Message ID||CHAR(35)|
|CstmrPmtStsRpt/GrpHdr/CreDtTm||CreDtTm||ISODateTime||ISO Message Document||Message Creation Date||DATE|
|CstmrPmtStsRpt/OrgnlGrpInfAndSts/OrgnlMsgId||OrgnlMsgId||Max35Text||ISO Message Document||Message Identifier||CHAR(35)|
|OrgnlGrpInfAndSts/OrgnlMsgNmId||OrgnlMsgNmId||Max35Text||ISO Message Document||Message Name ID||CHAR(35)|
|OrgnlGrpInfAndSts/OrgnlNbOfTxs||OrgnlNbOfTxs||Max15NumericText||ISO Message Document||Number of Transactions||NUMERIC(15,0)|
|OrgnlPmtInfAndSts/PmtInfSts||TxSts||TransactionIndividualStatus3Code||N/A (ACCP for all successful state transitions, RJCT for all unsuccessful state transitions)|
|OrgnlPmtInfAndSts/TxInfAndSts/TxSts||TxSts||TransactionIndividualStatus3Code||N/A (ACCP for all successful state transitions, RJCT for all unsuccessful state transitions)|
|OrgnlPmtInfAndSts/TxInfAndSts/StsRsnInf/Rsn/Prtry||Prtry||Max35Text||Payment Instruction Lifecycle Status||Payment Instruction State Type Id||NUMERIC(15,0)|
Note: Table 1 and Table 2 demonstrate some key fields and attributes for the purpose of showing the data mapping process. A complete list of XML fields and their presence in the message payload should be validated against the latest version of the ISO 20022 messages.
Following an introduction to ISO 20022 payment messages, this article walked you through the data mapping exercise for ISO payment initiation messages (pain.008 and pain.002). You can use the same method for other types of ISO messages based on the payment type and business requirements.
Any data retention or compliance requirements should be considered during the data mapping exercise. This may impact your data mapping method, as depending on the type of data and the business purpose, it may not be permissible to store some attributes. As an example, if the XML payload contains information that falls under the data compliance regulation, the data in the XML fields cannot be stored as part of the XML payload.
As a result of the ISO message data mapping exercise, we produced a process and rules for the data flow when the transactional system is able to receive and send back the messages defined by the ISO. The message gets translated into the transactional system data model and stored based on the system business rules.
- Learn more about the accessing ISO 20022 Universal financial industry message scheme.
- Learn more about the ISO Payments Dashboard.
- Learn more about ISO Payments Messages.
- Learn more about the European Payments Council SEPA Regulatory Framework.
- Learn more about InfoSphere Data Architect.
- Learn more about the International Organization for Standardization.
Dig deeper into Information management on developerWorks
Get samples, articles, product docs, and community resources to help build, deploy, and manage your cloud apps.
Keep up with the best and latest technical info to help you tackle your development challenges.
Software development in the cloud. Register today to create a project.
Evaluate IBM software and solutions, and transform challenges into opportunities.