Case Study – AML Model Validation
© 2017 CRISIL Ltd. All rights reserved.
Case Study: AML Model Validation (1/2)
Background
• A US BHC wanted CRISIL to validate the anti money
laundering (AML) model (hereafter called as “the model”;
vendor model from Actimize) required as per the Bank
1. Model
Secrecy Act (BSA) /AML regulatory guidelines
documentation • The model helps identify the high risk entities who may be
process
© 2017 CRISIL Ltd. All rights reserved.
alignment partaking in money laundering and terrorist financing
activities via:
Customer due diligence (CDD) and know your
customer (KYC) for new customers
Real-time monitoring of the existing customers
4. False CRISIL
2. Data
positive GR&A Validation
identification Approach
Business Objective
3. Model • Ensure that the model is aligned with the regulatory
development
and requirements and effectively captures risk factors and
performance scores
tuning
• Data, process, and system validation
• Complete the exercise within the stipulated (rigorous)
time-frame
6
Case Study: AML Model Validation (2/2)
Validation Process 2. Data validation/integrity check
• Per SR 11-7 guidelines, “Vendor products should • Sufficiency of controls around the data mapping process
nevertheless be incorporated into a bank's broader model • Results of data mapping process done as part of UAT to
risk management framework following the same assess whether the model is capturing complete and
principles as applied to in-house models, although the accurate information from the source data
process may be somewhat modified.” The model • M&M plan to determine if the data mapping process will
validation consisted following activities be monitored on a regular basis in the future
© 2017 CRISIL Ltd. All rights reserved.
3. Model Tuning
1. Model documentation alignment w.r.t
• The Model Tuning done during implementation would
• BSA/AML regulatory guidelines. For instance, whether
determine the performance of the model and provide basis
the model identifies a PEP, gives a higher risk to the for further tuning.
geography which has been identified as high risk by
• Changes made, in various tuning processes, to the risk
certain sources.
factors, attributes of the risk factors, and scores given to
• Internal model documentation standards (based on SR various attributes of the risk factors
11-7 guidelines). For instance, whether the model
4. Evaluation of overrides (identified false positives)
documentation contains results for the User Acceptance
Testing (UAT), results for the system integration testing, • Made during the model production and whether the
overrides are being tracked and monitored on a regular
assumptions with their relevance, risk and limitations,
basis
roles and responsibilities for the model implementation,
validation, and use. Client Impact
• Values in the production system i.e. confirming that the
• The models were validated within a stringent deadline
risk factors, their attributes and sores, LOBs covered are and the multiple processes including tuning process were
same in the production system as provided in the model improved. Model documentation deficiencies were
documentation/other documents. identified and resolved.
