Zanders Model Risk Framework
As automation and digitalization are adopted more widely in the financial industry, the number of financial models used is also steadily growing. As a result, an institution’s success or failure depends increasingly on the accuracy and reliability of those models.
A real-life example of a model risk event is the 2012 JPMorgan Chase trading loss of more than USD 6 billion, caused by a risk evaluation model that mistakenly underestimated the risk by half – due to a spreadsheet error.
This did not go unnoticed: awareness in the industry is rising and some recent developments hint at an increasing focus on model risk from EU financial industry regulators.
Model risk is usually considered an operational risk and includes all risks that arise due to simplifications, inappropriate model choice, errors in a model or misuse of a model. Model risk management goes beyond model validation and concerns the entire model life cycle, from development and testing to validation and use.
A clear and comprehensive model risk framework is the basis for proper risk management. This article provides a high-level representation of the Zanders Model Risk Framework.
Zanders Model Risk Framework
The Zanders Model Risk Framework identifies four components:
- model risk governance
- model inventory and classification
- model life cycle management
- model risk quantification
1. Model risk governance
The model risk governance comprises the institution’s model risk definitions, appetite, policy and organization and describes how model risk is addressed. It should at least contain:
- definitions of a model and model risk;
- the institution’s ambition and risk appetite for model risk, which also contains a statement on prudency principles and mitigation techniques (e.g. when to apply a margin of conservatism)
- model risk policy with guidance on model development, testing, validation and use (see model life cycle) depending on the model risk tier;
- roles and responsibilities regarding the model and model risk, e.g. the three lines of defense of which a concise view is given in the picture to the right.
2. Model inventory and classification
The model inventory lists all models under the model definition in the model risk governance. The model risk department creates and maintains this model inventory list. If there is no model risk department, this role can be filled by the model validation department.
All models in the inventory are classified. This classification provides a view on the level of model risk in the institution’s business processes. The assessment can be qualitative or quantitative and is based on two dimensions: the likelihood of a model risk event and the potential impact on the business. See also section 4 on model risk quantification.
Classification can be done by assessing:
a) the level of simplification in the model ;
b) the level of expert judgement;
c) data deficiencies;
d) model use;
e) assumptions; and
f) model validation conclusions.
Models with a high likelihood of errors and high potential impact bear the largest model risk indicated in the figure as Tier 3 models. The consequences of a high risk tier, like more frequent model reviews, are described in the model risk governance.
3. Model life cycle management
To be in control of all models at different stages, clear model life cycle management is required. A schematic view of a model life cycle is provided in the picture below. The model life cycle should be applied to all models, but can be simplified when the complexity or materiality of the model is low. It results in clear segregation between different processes in the model life cycle, while flexibility remains. Furthermore, the model life cycle forces clear documentation, which is essential to keep track of intended use, restrictions, limitations and risks. Finally, it links back to the model inventory and classification with the ultimate goal to reduce the model risk.
4. Model risk quantification
For certain models, especially where a high level assessment resulted in a high model risk classification, it can be useful to carry out a more in-depth, quantitative model risk assessment. The applied techniques to quantify model risk depend on the specific model (nature and model risk classification of the model) and on the model risk assessment ambition as defined in the model risk governance. A quantitative model risk assessment should at least contain data deficiencies, model uncertainty and model use.
- Data deficiencies can be assessed by an input data quality review to get an understanding of the number of missing data and accuracy of available data. Impact of data deficiencies can be quantified, e.g. by comparing the model output based on the extremes of the possible data input for missing or incorrect data.
- Model uncertainty (i.e. the use of a wrong model form or the use of wrong parameters) be quantified by means of model sensitivity analysis, output distributions or confidence intervals, back testing and challenger models. In this regard, interaction and interdependencies1 between models must also be addressed. This is often ignored since it is hard to measure reliably due to the complexity of the model landscape in financial institutions and the sheer number of models used.2
- Model use risk can be assessed by experimenting with all manual steps in the daily process of the model. Next to that, different outcomes of the decision-making process should be simulated and quantified if feasible.
Ideally, the impact of the aggregated risk is quantified as well to get an understanding of the combined impact. The result of the model risk assessment provides an insight of the accuracy of the model and how to interpret the outcome of the model. Furthermore, it can serve as a base to determine the need for and/or the size of a capital reserve for model risk.
The Zanders approach to model risk management
The Zanders model risk management framework offers our clients an efficient way of dealing with model risks while gaining a better insight into their model landscape. Tailored to the needs of our clients we can set up or review a model risk framework based on our own research and insight into best industry practices. With broad experience in risk governance, model specification, development, testing and validation Zanders can advise and support financial institutions in all aspects of model risk management.
- Clear distinction of model risk versus other risk leads to better decision-making;
- Unknown risks can be detected and mitigated resulting in fewer and less impactful model risk ‘events/losses’;
- Effectiveness and efficiency in model development reduces costs and/or improves value from the model development spend;
- Improved reputation, e.g. towards supervisors; and
- Better resilience through model risk capital add-ons.
1. Interdependencies can e.g. occur if the output of one model is used by another model or if several models are using the same input data.