Stress Testing: A View from the Trenches - SAS · 2017-11-28 · Stress Testing: A View from the Trenches Since the global financial crisis of 2008, stress tests have taken on growing

Stress Testing:A View from the Trenches

Stress Testing:A View from the Trenches

Since the global financial crisis of 2008, stress tests have taken on growing importance and prominence in financial institution supervision and regulation. These tests, designed to measure an institution’s ability to maintain capital buffers and withstand extreme economic shocks, were imposed initially, and primarily, on the biggest multinational firms – those designated global significantly important banks (G-SIBs) or financial institutions (G-SIFIs). However, the circle for supervisory stress testing has widened to include a growing number of banks as defined by domestic jurisdictions – in the United States, for example, down to banks with $10 billion in assets under the Dodd-Frank Act Stress Test (DFAST) rule. What’s more, stress tests and their underlying scenarios can be of considerable value as a strategic management tool to a financial services company of virtually any type or size.

In the years since the crisis, regulators have continually modified and re-fined their approaches, which in turn have demanded increasing amounts of time and resources on the part of financial institutions even as they have progressed along the stress-test learning curve.

Survey of risk management professionals reveals disparity in maturity levels as thefinancial industry deals with a critical strategic and regulatory compliance function.

2

Introduction

GARP | SAS Institute Survey Report


To shed light on the state of stress testing as perceived by those who are “in the trenches,” the Global Association of Risk Professionals (GARP) and SAS Institute in the spring of 2015 surveyed 389 executives and practitioners directly or closely involved in the process at a globally distributed sample of financial institutions.*

The survey reflects the state of stress testing from the point of view of risk professionals. From these results, a picture emerges of the level of maturity and effectiveness of the processes and procedures that these organizations have put in place to implement stress tests.

There are a number of practices that the more mature or advanced institutions have in common, notably technology and data manage-ment proficiency and a working rapport with regulators in understanding and responding to stress-test requirements.

Meanwhile, various aspects of stress-test programs, including the marshaling of sufficient expertise and other resources, remain works in progress. Those that find themselves somewhat behind on the maturity curve can look to the more advanced and experienced organizations as a best-practices framework emerges from their experiences. Those practices may include, and will not necessarily be limited to:

• Adequate levels of management attention and resources to stress testing within the firm, and, externally, to relationships and communications with regulators. • Comprehensive, centralized data repositories and libraries of risk factors and scenarios.• Gaining added strategic value from stress testing in such areas as risk appetite, capital and liquidity management and underwriting.

The survey was conducted in May and June 2015. The 389 participants responded to invitations that were emailed to a targeted list of GARP members or posted on the GARP LinkedIn group. The respondents hold a variety of executive and administrative positions at finan-cial institutions headquartered in 68 countries; the largest numbers of responses came from the United States (21%), Canada (8%), India (7%) and the United Kingdom (7%). In terms of assets under management, 23% represent firms with at least $500 billion, 24% between $50 billion and $500 billion, and 53% less than $50 billion. Forty-one percent are employed by commercial banks, the rest by retail banks, investment banks, asset managers and diversified companies consisting of more than one of those types of institutions.

3 GARP | SAS Institute Survey Report

Methodology:

Toward Stress-Testing Maturity

- Financial institutions, on the whole, are rising to the challenges of supervisory stress testing.- Data management, modeling and scenario management – widely regarded as foundational requirements for an effective stress-test process – are reported as areas of significant progress by the majority of firms in the survey.- Performance gaps are most pronounced in the technology used in stress testing and in coordination of the reporting of test results among different parts of the organizations.

Respondents to the GARP/SAS survey assessed various components of their stress-testing processes, and they were divided into four catego-ries: highly mature, mature, immature and highly immature.

Certain practices are typical of highly mature organizations. These are, for example, much more likely than the overall population to have internal risk models that closely align with those of regulators. The vast majority of highly mature institutions have centralized libraries for both risk factors and scenarios, as well as unified data repositories.

Mature operations also differ from the norm in their evaluations of infor-mation provided by regulators: A slight majority (53%) of all in the survey said that such information is sufficient, but it was four out of five (79%) at institutions with highly mature stress-test programs.

Financial institutions have introduced changes to their business in re-sponse to test results, particularly in the area of risk. Those with highly mature stress-test processes were more likely to report that they have made changes based on test results.*

4

*Respondents were asked to what extent various strategic and decision-making functions were changed based upon stress testing results. Areas in which highly mature firms differed markedly from the overall sample included risk limits (41% of the highly mature group reported making significant changes, compared with 21% of all in the survey) and remuneration strategy (35% versus 5%).



Strengths andWeaknesses

- Self-assessments tend to highlight weaknesses in stress- test technology more often than in people and expertise.- Few are satisfied with the level of information reporting and coordination across the enterprise.

Respondents were asked to assess their organizations’ resources in four specific areas: data governance, modeling, scenario management and reporting. In each case, they rated their technology lower than people or expertise. In data governance, 42% agreed or strongly agreed that their technology had the right capabilities, versus 63% who said the same about their people and 56% about their insti-tutions’ expertise.

On modeling, 49% were satisfied with their technology, versus 68% with people and 69% with expertise. On scenar-io management, 45% were satisfied with their technology, 63% with people and 58% with expertise. On reporting, 45% were satisfied with their technology, 65% with their people and 63% with their expertise.

Highly mature financial institutions were most likely to agree or strongly agree that they have the right people, expertise and technology, but their technology still rated lower than the other two components. In modeling, 100% of highly mature institutions were satisfied with people and exper-tise, 93% with technology. In scenario management, there was a wider range: 97% satisfied with people, 93% with expertise, 83% with technology. In data management, 93% were satisfied with their people, 86% with both expertise and technology.

Technology is perceived as the weakest link regardless of institution type. Investment banks tend to have a higher level of satisfaction with their resources than do respon-dents overall, while retail banks’ satisfaction was lower than that of respondents overall.The technology challenge may be a function of an institution’s starting point.

“Over the last few years, many banks have relied on existing systems and internal development of models that are pieced together to tackle regulatory stress testing, as the need to address compliance has escalated faster than many banks’ ability to keep up,” said Wei Chen, director of stress testing solutions at SAS Institute. “The complexity and granularity needed to meet current regulatory and business challenges have now led the industry to further invest in technology and more comprehensive systems that not only can perform the required calculations, but can keep track

of key details and provide transparency to regulators.” Time constraints may also be affecting the industry’s response. Luther Klein, North America banking risk lead at Accenture, pointed out that the amount of time financial institutions devote each year to the stress testing process can interfere with attempts to upgrade technology. Meanwhile, regula-tors are pushing for a higher level of detail in test results, which complicates matters.

“Given the changing expectations from the regulators as well as the short timelines to turn around the results, it’s been a challenge to implement enterprise technology solutions that can support a stress test,” Klein said.Another concern relates to the way risk management, the capital planning function and other areas in financial institu-tions work together on reporting the stress-test results.In what is likely a reflection of large institutions’ organiza-tional silos, only 7% of respondents said their institution coordinated its reporting extremely well. Another 79% saw room for improvement, including 10% who cited “consider-able need for improvement.”

Underscoring the challenge of coordination amid tradition-ally siloed corporate structures are the needs for consistency and coherence of the data and assumptions that go into stress testing, said Fabienne Libert, head of ING Risk Services. “In general, stress testing exercises are quite confidential, which is inherent to the process. In line with the governance surrounding this process, people are involved in discussing the output and results on a need-to-know basis.”

Moreover, she said, with greater emphasis being placed on the qualitative dimension of stress testing, technologi-cal and data-collection impediments must be removed to ensure efficiency and effectiveness.


DifferingPerceptions

6

- Practitioners working directly on stress testing provide more favorable self-assessments than more-senior managers do.- Majorities of both populations say stress testing is getting about the right level of management attention.

Conventional wisdom would hold that senior executives have a more upbeat view of compli-ance and risk management performance than do the teams carrying out those functions day-to-day. In the GARP-SAS survey sample, consisting of 47% senior managers and 53% people involved in stress-testing projects, the practi-tioners tended to be more positive than the managers:

- 58% of practitioners said their data quality was mature or highly mature, versus 50% of managers. - 62% of practitioners said their institution’s expertise on data management was adequate or strong, compared with 49% of managers. - 63% of practitioners said their scenario management expertise was adequate or strong, while 52% of managers said the same.

“When it comes to stress testing, the prevail-ing sentiment we’ve heard from people in the trenches has been that management is wearing rose-colored glasses,” said Tom Kimner, head of the Americas risk practice at SAS Institute. “What we’re seeing in the survey results seems to be the opposite. People who self-identified as man-agers have a less optimistic, or perhaps more realistic, view of things, while the practitioners think more positively about where their firms are in stress testing.”

On some points, managers and practitioners were in accord. Approximately 45% of both groups considered their organizations’ integra-tion of data to be mature or highly mature; 54% of both groups said auditing and governance of internal cooperation on reporting results was mature or highly mature; and seven out of 10 managers and practitioners alike thought their firms had made some or significant progress on data management.

Managers and practitioners were also closely aligned on the question of whether stress testing is receiving the appropriate amount of management attention: 54% of both groups agreed with that statement.

ING’s Libert said the different perspectives may be explained by managers’ keen awareness that regula-tors are becoming increasingly demanding, and that the time frames for completing stress tests will be getting ever shorter.

She said those in the trenches “are very smart and know very well what they are doing. Nevertheless, it is crucial to be aware that the way you are performing the stress testing exercise nowadays is not the same as it will be in the future. This requires new capabilities from a data, process and tooling perspective that, from a strategic point of view, you need to invest in to embed the stress testing activities in your business as usual.”



In Libert’s view, regulators could be “more transparent and clear.” “This requires a lot of investments from the regulators,” the ING executive asserted. “And it requires that [regulators] listen to the bank, and that the bank also listens to the regulators. We should invest more time to better understand each other.”


- The more mature institutions are better in tune with regulators and their requirements.

Slightly more than half of the respondents said that regulators are providing them with enough infor-mation prior to and during stress testing exercises, and about the outcome of the tests.

Those at institutions with highly mature stress-test programs were much more likely to agree that regulators are providing enough information than were those with immature programs: 79% said that regulators are providing enough information prior to stress tests, 69% that they provide enough information during stress tests, and 72% that they share enough information after stress tests.

Conversely, only 44% of those at institutions that are highly immature felt they had enough infor-mation from regulators prior to stress tests, 38% during stress tests and 30% time after stress tests. Ingo Walter, professor of finance, corporate governance and ethics at New York University’s Stern School of Business, observed, “Those who are further along in their stress testing architecture know what to do with regulatory inputs or to come back to them with pertinent questions so the whole process is more seamless.”

Accenture’s Klein pointed out that in the United States, banks that are just starting to perform stress tests can find themselves at a disadvantage, because much of the information that regulators provide to banks “is between the regulator and the bank. Because of that, there is less public informa-tion available.”

RegulatoryTransparency

Nearly one in five (19%) reported making a significant change in risk appetite, and 36% made some change. Away from the risk function, only 5% (although 35% of those considered highly mature) said they had made a significant change in remuneration strategy, while 16% had made some change; and 8% made a significant change in underwriting, while 20% had made some change.

Further underscoring how highly mature financial institutions differentiate themselves, almost half (48%) of those in this category said they have made a significant change in their risk appetite (compared with 19% overall), and the same proportion have significantly increased the resources committed to risk control (versus 20% overall).There is still a long way to go before stress testing is universally seen as a competitive business tool.

8

- Changes in management and strategic functions as carry-overs from stress testing are only beginning to be noticed.

Expectations that financial institutions can leverage their investments in stress testing by using the data it gener-ates to inform their overall business strategy don’t seem to have panned out yet. But the survey shows firms have instituted changes in response to test results in some risk-related areas.

Roughly a fifth of respondents (21%) said they made a significant change in their risk limits, and more than a third (38%) made some change in risk limits. In addition, 20% significantly increased their risk control resources, and 34% made some increase in those resources.

Influence onStrategy



Progress andMaturity

9

- Strong improvement seen in data management, modeling and scenario management; reporting remains more of a challenge.- Centralized data and scenario repositories set the more mature institutions apart from the norm.

The survey results suggest that financial institutions continue to make improvements in stress-test process-es. About 70% of respondents said they have made some or substantial progress on the data management front, 65% some or substantial progress on modeling, and 60% some or substantial progress on scenario management. The numbers are quite different when it comes to the challenging reporting process: 27% cited significant progress, 55% little progress.

“Substantial progress has been made over the last several years with both design and implementation of stress testing programs at large, systemically import-ant financial institutions,” said Troy Haines, senior vice president of risk at SAS.

“However, building out long-run sustainable stress testing capabilities remains a significant challenge.”

Certain practices are more correlated with stress-test maturi-ty: 69% of highly mature institutions have internal risk models that closely align with those of regulators, versus 36% of all in the survey. Almost half (47%) of highly mature institutions cite the account level as the greatest degree of granularity their models achieve, versus 30% of institutions overall.

Three out of four (76%) highly mature institutions have a comprehensive, centralized library of risk factors, versus 38% overall, and 79% have a comprehensive, centralized library of scenarios, versus 38% overall. And 73% have a unified data repository that allows full integration and reconciliation, versus 17% of institutions overall.

Those whose processes are characterized as highly mature are more likely to say stress testing gets the appropriate amount of management attention (93%), versus just over half (55%) of respondents overall.


10

A Snapshot of Stress-Test Practices


63% say chief risk officers take prime responsibility for stress testing, followed by chief financial officers (14%), business unit heads (10%) and CEOs (7%).

46% require more than six hours to execute a set of stress testing models; 10% can do it in less than an hour.

29% of those dealing with multiple regulators’ stress tests will develop an integrated process; 36% will use separate processes but compare methodologies and results; 13% will manage them separately.

22% completed regulatory stress tests in less than a week; 45% took one to four weeks, 25% five to eight weeks and 9% more than eight weeks.


Conclusion

11

As the stress-testing survey highlights, new levels of rigor and expanding reporting requirements mandated by reg-ulators increasingly require banks, whatever their level of maturity, to evaluate not only what investments need to be made, but also how they will ensure the orchestration and transparency of the entire stress-testing process. Given requirements for more efficient data and modeling plat-forms, along with additional governance activities, it’s clear that meeting both the quantitative and qualitative aspects of regulatory compliance poses tough challenges for finan-cial institutions.

Banks with less mature stress-test processes can learn from the best practices followed by financial institutions that are more mature. Those best practices include strong governance around stress-testing processes; the use of shared, common, quality data for modeling and analysis; and a thorough inventory and understanding of the mod-els used throughout the process.


Also crucial are stress-test-related communications and coordination that break through siloed corporate struc-tures, as well as strong, positive interaction with regulators. While companies have invested in people and processes, they also need to invest in technology to help them deal with everything from the complexity of the exercise to the quick response times and turnaround that will be needed to address changing scenarios and regulatory require-ments.Organizations of all sizes and levels of maturity will need to manage change through a series of continuous improve-ments as regulators enhance their requirements and man-date that stress testing be performed within increasingly reduced time frames.

-- Managing data. Data management is an integral component of every initiative and strate-gic decision. Success depends on the bank’s ability to quickly and easily access and integrate the data required, while having confidence that all is correct, current and complete. Banks require comprehensive data management capabilities that include data quality, data lineage and metadata documentation in a transparent and readily searchable form through the entire stress-testing life cycle.

-- Monitoring model risk and performance. Banks require capabilities to support model de-velopment as well as model governance enabling independent review and validation of models used in internal capital planning. Requirements across model processes would include a gov-ernance system, a complete inventory of models, a model validation process and a champion/challenger assessment process.

-- Implementation. Model execution platforms need to perform huge numbers of calcula-tions, while executing a variety of models on large amounts of granular-level data. These plat-forms must also aggregate the results to any desired level for analysis and reporting, while including process management capabilities that ensure all necessary steps are completed, monitored and repeatable.

-- Coordination. Banks need to specify scenarios and consolidate modeling results into bal-ance sheets, financial statements and capital plans while orchestrating the various aspects of the stress-testing process and consolidating the results from various systems.

“Key Areas of Readiness for Future Changes in Stress Testing”

Creating a culture of risk awareness®

Global Association ofRisk Professionals

111 Town Square Place14th FloorJersey City, New Jersey 07310U.S.A.+1 201.719.7210

2nd FloorBengal Wing9A Devonshire SquareLondon, EC2M 4YNU.K.+44 (0) 20 7397 9630

www.garp.org

About GARP | GARP enables the risk community to make better informed risk decisions through creating a culture of risk awareness. We do this by educating and information at all levels, from those beginning their careers in risk, to those leading risk programs at the largest financial institutions across the globe, as well as, the regulators that govern them. www.garp.org

©2015 Global Association of Risk Professionals. All rights reserved. 7-15

SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Other brand and product names are trademarks of their respective companies. 107918_S142082.0915

Stress Testing: A View from the Trenches - SAS · 2017-11-28 · Stress Testing: A View from the Trenches Since the global financial crisis of 2008, stress tests have taken on growing

Documents