As we continue our series of blog posts examining the AICPA Practice Aid on preparing for the new current expected credit losses (CECL) standard, this post on modeling considerations is especially timely given the economic upheaval of the ongoing COVID-19 pandemic and the dramatic effect that it is having on loss models. Many of the assumptions built into January models were no longer valid by March of 2020. That has forced many institutions to adjust their models to this new reality.
As noted in a March 2020 Flash Report, regulators provided the option to delay implementation of the new CECL standard until the crisis is declared to be over by presidential decree. However, most financial institutions were so far along in the CECL implementation process that it would have been more work to revert back to the old incurred loss methodologies. We have added a section at the end of this post addressing these COVID-19 challenges.
Management Considerations
In selecting a model, management should consider the risks within its portfolio segments and choose a model for each loan pool that can provide a reasonable estimate of expected credit losses. Because multiple models could be appropriate, management must document the thinking behind both the initial model selection and any subsequent changes or re-thinking of the models.
Management is also required to establish and implement policies, procedures and controls over the models, including:
- A model validation process that includes an evaluation of appropriateness and integrity; ongoing monitoring to ensure consistency and ongoing suitability; and an outcome analysis
- Change of control policies and access controls
- Timely adjustment
- Periodic review, testing and calibration by an independent internal or external entity
- Inputs anchored in objective evidence
- Documentation of key parameters, required data, validation analysis and output adjustments
These considerations apply whether the model is developed in house, outsourced, purchased, or contracted through a service provider. And even when third parties are used, management remains responsible for understanding the model, including any proprietary “black box” calculations.
Audit Readiness Considerations
Auditors will be looking to determine how management came up with their CECL estimate. They will want to review the data, process, key assumptions and judgments upon which the estimate is based — including models and methodologies — as well as the validation process.
To increase the likelihood of a successful audit, management should ensure their rationale behind key model assumptions is adequately documented, including: the contractual lifetime of a loan or a pool of loans, amortized costs, recoveries, prepayments, troubled debt restructurings, unconditionally cancellable extension options/renewals, averaging methodology (simple or weighted), and incorporation of any forward-looking information.
The purpose of this documentation is to identify any risks of material misstatement that may draw auditor scrutiny. According to the AICPA Practice Aid, these risks may include:
- Risks related to the model’s conceptual soundness — Fundamental flaws may exist which may result in inaccurate outputs; mathematical theories may be misapplied; data and assumptions may not be properly evaluated or supported.
- Risks related to model development and implementation — Poorly documented or flawed model theories, assumptions and specific mathematical calculations; insufficient stress testing; a lack of model access controls; modeling limitations not assessed or addressed over time.
- Risks related to adjusting models over time — Failure to establish and maintain proper controls over model performance tracking, evaluation and approval; lack of timely and ongoing monitoring and validation procedures.
- Risks related to model suitability —Use of the same model to predict the effect that macroeconomic factors (for example, unemployment, housing price indexes) have on all loan segments, when the effects differ between segments; inappropriate application of a model originally created for a different purpose.
- Risks related to system integration — Incomplete or inaccurate information used to support the calculation; data flow is not properly controlled; model has not been accurately implemented, including aspects of system integration.
- Risks related to model output reviews — Not comparing model outputs to actual outcomes and failing to conduct appropriate model performance measurements; failure to review and investigate root cause of discrepancies between model outputs, expectations, external information and benchmarking; failure to translate estimates into useful business information.
- Risks related to objectivity and effective challenge — Insufficient separation between individuals designing and implementing the model and those reviewing and validating it; individuals reviewing the models do not have the necessary skills or expertise; senior management does not appropriately challenge the decisions made by the individuals responsible for modeling.
COVID-19 Considerations
Models that effectively forecast losses to estimate reserves under normal economic conditions may break in times of extreme duress. This was certainly true during the previous financial crisis and is true again today, amid the global economic upheaval caused by the COVID-19 pandemic. And while the government provided the option to delay CECL implementation due to the pandemic, the conditions and qualifiers were so difficult to interpret that most financial institutions opted to forge ahead.
Banks that adopted CECL as of year-end also had to incorporate the impact of the pandemic on economic conditions — such as over 40 million people filing unemployment claims; significant downturns in several industry sectors; changes in real estate collateral values, and principal and interest payment deferments, in addition to other forward-looking scenarios.
Based on a survey of 50 mid-sized and large public banks, first-quarter CECL estimates posted by public filers indicate that the day-one CECL adoption impact increased loss reserves by an average of 35-40% of 12/31/2019 year end-balances. Many of these filers also posted the COVID-19 impact on loss reserves, which averaged 45% of 12/31/2019 year-end and 25% of 3/31/2020 quarter-end allowance levels.
While some of the COVID-19-related economic changes can be accounted for in existing CECL models, many banks are also using outside-of-the-model qualitative adjustment frameworks to adjust for economic conditions where the relationships between model variables may no longer be valid due to the extreme values in variables or scenarios that exist.
The changing economic fallout from COVID-19 and the offsetting impact of government stimulus programs will continue to have a significant impact on credit losses. Management should monitor model performance and model outputs, and reassess the different scenarios impacting portfolio quality as institutions estimate their loss reserves.
As discussed in a previous blog post, Protiviti has been working hard to help our clients stay ahead of the knowledge curve on this issue. We recently published a brief on COVID-19 modeling challenges. Some of those challenges include:
- Models may fail due to negative economic impacts — As extreme macroeconomic factors become the actual data used for modeling and back-testing, models will need to be updated.
- Developing forecast scenarios to support pandemic response — As economic indicators fall, financial services companies need to develop multiple scenarios — baseline, adverse and severely adverse. CECL models may have very different results in Q2 compared to Q1.
- Getting an early view of portfolio impacts — Effective action plans driven by scenario analysis can minimize credit losses while maintaining liquidity and financial strength.
- Estimating the impact of government intervention — Government-mandated payment deferrals and loan forgiveness must be considered in estimating credit quality. This is a significant challenge for many reasons, but most notably because there is no established behavioral pattern for these programs.
Effective action plans driven by robust scenario analysis can help ensure that institutions have the financial strength they need to weather the storm. This applies whether an institution has already implemented CECL or has a future implementation date and thus still relies on incurred loss allowance methodologies. And whatever methods are used, auditors need to ensure that management is carefully documenting judgments, assumptions and model performance to help avoid material misstatements.
