Do virtual economists get virtual model validators?

Monday, November 19, 2018

Do virtual economists get virtual model validators?

In the novel on which the classic film Bladerunner is based, the prolific science fiction author Philip K. Dick asks whether androids dream of electric sheep. Perhaps in a science fiction universe, virtual economists could have their work validated by virtual model validators, have model risk managed by virtual CROs and have governance managed by a virtual board.  But here in this universe, under the current regulatory and legal framework, real live model validators must be used to validate models, real live CROs must be used to manage model risk, and real live boards must be used to provide governance.

CECL allows a wide range of methodologies. Regardless of the methodology, CECL requires an institution to quantify the relationship of current conditions and reasonable and supportable factors on the allowance. Perhaps shy from recent experience with qualitative adjustments, many smaller institutions who are not expressly choosing to use forward-looking models are nonetheless seeking ways to “quantify the qualitative.”

While this is understandable and desirable even from the point of view of many stakeholders, including external auditors, it is nonetheless potentially problematic. Using a correlation to determine if a factor is related to an institution’s loss history offers helpful information but does not address the unavoidable, fundamental challenge of CECL: quantifying this impact. Unfortunately, most anything that assists an institution in doing this quantification runs the risk of becoming a model. In particular, using trendlines is a sneaky way of accidentally creating a model.  Additionally, anything with the term “regression” is explicitly a model. Regardless of how a model enters into the CECL estimate, the accompanying risks must be managed.

Managing model risk is no joke, especially for an accounting estimate. There are a number of overseers who are directly interested in how an institution manages its CECL models, whether created intentionally by real economists or inadvertently by virtual ones. All three major safety and soundness regulators have issued guidance to institutions about managing model risk. The OCC and Federal Reserve issued comprehensive guidance in 2011 (OCC Bulletin 2011-12 and Fed Letter SR 11-7, respectively). The FDIC didn’t have consistent guidance with the other regulators until six years later when it issued FIL 2017-22, applicable to all institutions above $1 billion in assets.

Historically, the inherent risk of accounting estimates has been a major focus for auditing firms, and one can expect the focus to increase after CECL adoption. CECL will not only be a material estimate, but it is also a prominent one; it has gained a prominence in recent weeks that is unlike any other accounting standard in history. Controls around accounting estimates have also been important areas of review. It is important to remember that audit firms are themselves regulated, and in turn the focus of their regulator, the PCAOB, is on accounting estimates and their associated controls. Indeed, in the proposed update to the audit guidance, the PCAOB has explicitly added language to clarify their expectations in this important area. It has explicitly added clauses instructing auditors to:

  • Understand the methods used for the estimate, specifically mentioning “models” when identifying the risk of material misstatement (AS 2210)
  • Obtain written representations from management around the appropriateness of the method, consistency in application, accuracy and completeness of data, reasonableness of significant assumptions used in developing accounting estimates (AS 2805)

CECL does not require models; indeed, the FASB and the regulators have gone to great length to emphasize this point. However, it is of fundamental importance that institutions understand their methodologies and manage the risks, including model risks, appropriately. It is hard to see how having a virtual policy enforced by virtual risk managers to oversee the work of virtual economists is going to be an acceptable practice.

Asset Management, Regulation, Wealth Management

Theme picker