Share this post:
Payment fraud detection has always had a bit more latitude than its counterparts in anti-money laundering, customer due diligence and even trade surveillance compliance. Unlike the latter areas, fraud prevention is an area not as heavily governed by regulations or specific rules of what a financial institution should or should not allow. It is obviously in the best interest of an institution, and its customers, to catch as much fraud as possible. But how zealous that approach should be was largely left to the fraud groups to determine.
However, something has changed. Fraud managers are being asked about the assumptions and limitations of models, the quality of the data used for their calibration, and the thoroughness and independence of the model validation process. In short – model governance has come to the fraud detection realm.
Model governance 101
Internal auditors expect financial institutions to understand the limitations of the models they use to analyze data and make important decisions. They want to clearly understand the methodological underpinning, the quality and availability of the data used for the calibration, limitation linked to the models’ implementation and limitation imposed by the context in which the models will be used. This represents a larger focus of the overall governance set around a model’s lifecycle – model development, implementation, calibration, and validation – at financial institutions, with emphasis on the robustness of the policies, processes and controls, tests, and the quality of the institutions’ documentation.
Internal auditors also expect financial institutions to continuously challenge the assumptions made in the design process and whether the models would be adequate in real-life situations to ensure the models are accurate and effective. In particular, model owners should be able to clarify under what circumstances the assumptions would no longer hold. This is especially critical when using data proxies, which may “break” during market or financial condition changes. Therefore, applying robust model governance is becoming of paramount importance for firms. Furthermore, as models can be a significant source of risk, institutions are setting dedicated teams to manage and minimize this risk.
Why does model governance matter for fraud?
While it’s clear from conversations with fraud managers that this expectation is growing, it’s not as apparent why now. In the US, both AML and fraud detection models were already subject to review under the Office of Comptroller of the Currency’s 2011 “Supervisory Guidance on Model Risk Management.” It seems that, while this is not entirely regulator driven, financial services organization could be applying their model governance criteria across all discipline to ensure consistency and reduce the risk of potential regulatory scrutiny.
A related cause could be that fraud models have some effect on other areas that do come under increased regulation, for example, if increased fraud on an account triggers a customer due diligence review or a lower risk threshold for AML transaction monitoring activity. A third possibility is that, financial institutions are trying to ensure their systems are contributing or displaying bias in their decision process.
Whichever the case may be, understanding how to incorporate model governance while accomplishing the goal of quick and efficient fraud prevention is essential.
Building a fraud model governance program
The three main components of a model governance program are determining what exactly what the organization considers a model to be, establishing model usage policies, and developing and maintaining a model inventory.
In general, a model inventory should cover: intended and approved use of the model, last calibration and calibration frequency, validation status (passed, failed, passed with caveats), date of model sign-off, significant manual overrides and their justification as well as dependencies on other models.
The way we’ve built the IBM Safer Payments real-time fraud prevention platform helps deliver on these requirements. In fact, model governance has already been built into IBM Safer Payments for the past 15 years, and it was based on legal guidelines for auditability, best industry practices, and industry rules such as set forth by the Basel Accord. As a result, the solution comprises all analytics and simulation tools needed to continuously monitor the business performance, using artificial intelligence, machine-learning and advanced analytics to suggest payment fraud detection models based on input data. The suggestion process – in a champion/challenger approach – is performed to optimize for explainability and controllability in order to comply with model inventory requirements.
IBM Safer Payments continuously monitors the efficacy of all defined fraud countermeasures and decision rules, highlighting the ones that demonstrate declining performance over time to the fraud experts for review. At the same time, artificial intelligence algorithms devise new fraud countermeasures and rules automatically from its internal database and presents them to the users for review.
For both ‘champions’ and ‘challengers’, the necessary model parameters (i.e. attributes, indicators, conditions) and keys intended or actual performance statistics (i.e. hit rate, false positives) to support the intended use of the models throughout the model lifecycle. All data can be produced in a reportable and auditable method and provided to regulators and internal auditors for model validation analysis as well as evidence of compliance with model inventory requirements.
Expect a trend in this direction
While some of these conversations on model governance for fraud models are happening within a handful of institutions, one should expect this trend to catch on. The financial industry tends to look to peers to fulfill regulatory expectations, and though regulators may not have been as stringent, your internal auditors aren’t waiting around for them to start. As your organization adds new technologies, whether to update existing capabilities or provide new products such as P2P payments, it’s best to build these types of capabilities into your selection process.