Kpmg AAA Updated Research on the NAIC Risk Based Capital (RBC) Formula Chris Nyce, FCAS, MAAA KPMG Senior Manager Research on the Effectiveness of a Trend.

kpmg

AAA Updated Research on the NAIC

Risk Based Capital (RBC) Formula

Chris Nyce, FCAS, MAAA

KPMG Senior Manager

Research on the Effectiveness of a Trend Test in the Property/Casualty

RBC Formula

Disclaimer

These results are based on research conducted by a subgroup of the American Academy P/C RBC Committee

Views expressed today are based on the research, but do not necessarily reflect the views of the Academy, CAS, KPMG, or the NAIC who of course makes all decisions about changes to the RBC formula

Examples used are illustrative, and not a reference to any specific company

Anyone who says otherwise is not only wrong, but is itching for a fight

The Mission of the AAA RBC Committee

Began the research with a charge- NAIC expressed concern that RBC was becoming more of a

lagging indicator, not useful in predicting companies that become weaker

- “Given the use of a trend test in the life RBC formula, is the application of a trend test in the Property/Casualty RBC formula a good idea?”

Our interpretation-- Not a “Yes/No” question

Instead-”What is the most effective way of differentiating between companies above the Company Action Level that are likely to fall below it, and those that are likely to remain above it.”

We approached this with a one year future time horizon, i.e. based on observable data this year, what will happen next year

Status of the Work

These ideas were presented in the Spring 2004 CAS meeting, prior to release of the Academy report

Academy issued a report on August 26, currently featured on the casualty cover page of the Academy website

- The Academy report did not recommend incorporating the finding in RBC formulas, instead concluded the one certain approach is a “best predictor” of those examined

NAIC PC RBC subgroup did recommend on August 31, 2004 that this trend test be incorporated into the RBC formula, and a company being flagged results in CAL

Exposure period ends November 11, 2004 NAIC considers comments, votes at December meeting

- Could vote yes, no, or further deliberation- If yes, likely implementation in 2005

Background

Life test currently uses a trend test Applies to companies with RBC between 250%

and the company action level (“CAL”) of 200%- RBC ratio is the ratio of capital to RBC required capital

Life test looks at past changes in RBC - Max of last year and the three year average decline in

margin for each company- Apply that change to the current RBC position- If below 190%, company is deemed to be at the CAL

Note that even before our work, the feeling of committee members was that the life trend test did not work well for P/C companies

- We quickly confirmed this to be true

Our Approach

Basic question-”What is the most effective predictor of decline in capital adequacy?”

In general terms, used “Hypothesis Testing” Examined specific cases of past company failures Formulated hypotheses on the causes of RBC

decline Tested the hypotheses using statistical tests on

annual statement data Conducted additional tests by examining the

effectiveness retrospectively Measured the results using a specific set of metrics Selected one approach that produced the best

metrics

Boundaries of our Study

Did not constrain ourselves to examining the life formula

Based on publicly filed data from the NAIC blank

- And used company level data, due to conclusions in the work leading to the original formula that this was appropriate

Outcome has to be intuitively correct, and simple

All research also from public data sources For NAIC data, company names remained

confidential Outcome had to be based on empirical data,

not on our preconceived opinions

Data Considerations

For “micro” analysis we used public data sources such as AM BEST and press reports

NAIC provided 5 year history of all requested data elements

- Confidential as to company identifier- About 2400 companies- Used data through 2002 for statistical tests,

updated through 2003 for retrospective test

We scrubbed the data, in general separately for each test to maximize data points utilized

- Screened out invalid entries and extreme values

Micro Results-Initial Hypotheses

Companies we examined could be characterized as experiencing trouble due to various causes, such as:

- High levels of reinsurance recoverables, causing high leverage in estimating reserves, and exposure to disputed balances

- High leverage of premiums and reserves to surplus

- Reserve inadequacies coming to roost - Poor operating results- Fraud and misrepresentation- Ill-liquid or incorrectly valued assets- Under-funded pensions: (usually a contributor,

not a cause)

What is the Best Early Indicator of Future Capital Declines?

Lack of LiquidityReserve

Inadequacies

Poor Profitabilit

y

Fraud

Leverage

Bad Assets Past Capital

Declines

Overall “Macro” Approach

Performed statistical tests on the NAIC database

- Explored the basic relationships behind each hypothesis

Performed retrospective tests on characteristics of companies just prior to falling to the CAL

Set up metrics to evaluate the outcome of the retrospective test

Determined recommendations based on all of the above

Statistical Tests

Explored relationships between hypothesized variables

Performed tests on the NAIC database of 2400 companies for 5 years ending 2002

Looked for statistical tendencies Generally used correlation and regression analysis

- Examined the percentage of variation explained- Calculated the measures of significance

Used to corroborate and explain retrospective result

Note that a poor result in our tests does not necessarily mean that the measure is not good for IRIS or other financial evaluations

- And high correlations don’t necessarily mean the hypothesis would form a good trend test

Statistical Test of Life Type Trend Test

Does a simple life type of trend test work?- Correlation between year to year changes in RBC

ratio for all companies= -23% (wrong sign)- For only companies near the CAL = 1%- In 2001 and 2002, the direction of the change in

subsequent years was only the same 41% of the time

Changes in market asset valuations dominated any characteristics of companies themselves

Implication: Life type of trend test is worse than random guessing for P/C Companies

What About Underwriting Results and Reserve Runoff?

Underwriting Results- Correlation between subsequent year combined

ratios= 25% to 34% between 2000-2002- For only companies near CAL correlation is 33%

to 75% (highly significant) Reserve Runoff

- Correlation between subsequent year runoff ratios=33% to 37% between 2000 and 2002

- For only companies near CAL correlation is 29% to 35%

This is good and bad news- Statistical relationship is strong- But still only predicts a portion of the subsequent

year outcome

What is the Predictive Power of Leverage?

Gross Leverage- Correlation between gross leverage and subsequent year

RBC ratio change= -1% to 1% between 2000-2002- For only companies near CAL correlation is –5% to –3%

(not significant)

Net Leverage- Correlation between net leverage and subsequent year

RBC ratio change= 3% to 4% between 2000-2002- For only companies near CAL correlation is 1% to 16%

(wrong sign)

This is not a good outcome- Statistical relationship is weak and sign is sometimes

wrong

Well then it must be Liquidity?

Correlation between liquid assets to surplus and subsequent RBC change is –4% to 1% over 2001 to 2002

Depending on sample, relationship is not significant, or sign is wrong

In 2002 and 2003, Portion of Companies Falling to CAL

RBC Ratio in Prior Year

Total Companies in Sample

Number of Companies Falling to

CAL

Percentage Falling to

CAL

200% to 300%

314 30 9.6%

300% to 350%

166 9 5.4%

350% to 400%

205 4 2.0%

400% to 450%

176 3 1.7%

Greater than 200%

3582 55 1.5%

Retrospective Tests

Performed on NAIC database of 2400 companies ending 2003

Generally “Yes/No”- Measured whether the hypothesis accurately predicted

the subsequent year outcome, or not- Therefore, scrubs were oriented toward invalids, but

not toward extremes

Measured on three metrics- Effectiveness-Percentage of overall correct

predictions- False alarms-Percentage of companies flagged that

did not deteriorate to CAL- Failing Companies Flagged-Percentage of

companies that subsequently declined to the CAL that were correctly flagged

Retrospective Approach

Started by setting a threshold such as leverage above industry average, or combined ratio above 110%, etc.

- Based on the threshold, companies were “flagged” or “not flagged”

Allowed for mixed approaches;- Leverage, reserve runoff, and combined ratio- Reserve runoff and combined ratio- Three year tests of reserve runoff and combined ratio

Adjusted the threshold to optimize the metrics- Based on trial and error

Understand, this test tells us not what causes RBC decline, but what best predicts it

- Although the implication for the cause is pretty clear

Retrospective Metrics

ThresholdRatio of failing

cos flaggedFalse

alarms/totalEffectiveness

RatioTotal # of

Companiestrend 62% 47% 50% 480-20% 67% 30% 67% 480-17% 64% 36% 61% 4807% 46% 43% 53% 4805% 46% 37% 59% 480

650% 56% 48% 48% 480350% 54% 46% 51% 480

Composite 64% 34% 63% 480Composite 69% 42% 56% 480-20%/34% 67% 26% 71% 480

Life Trend TestTest Year UW RatioThree Year Average UW Ratio

Composite UW and RunoffComposite uw/runoff/leverageTwo Tiered Underwriting Test

Test Year Runoff RatioThree Year Average Runoff RatioGross Leverage, End of Test YearNet Leverage, End of Test Year

An Effective Approach Based on Tests

RBC Ratio

Current Year Combined Ratio

Company Status

200%-300%

Greater than 120% More Likely to Decline to CAL-”flagged”

  Less than 120% Less Likely to Decline to CAL

300%-350%

Greater than 134% More Likely to Decline to CAL-”flagged”

  Less than 134% Less Likely to Decline to CAL

Above 350%

All Less Likely to Decline to CAL

Further Metrics from the Combined Ratio Trend Test

  Total Companies

% Flagged

% Of Companies Falling <200% in Subsequent Year

Flagged Not Flagged

Total

Two-Tiered CR Test for Companies 200%<RBC<250%

143 47% 17.9% 6.6% 11.9%

Two-Tiered CR Test for Companies 250%<RBC<300%

171 33% 17.9% 2.6% 7.6%

Two-Tiered CR Test for Companies 300%<RBC<350%

166 16% 14.8% 3.6% 5.4%

Total Population 200%<RBC<350%

480 31% 17.3% 3.9% 8.1%

Why not 100% Effective

Formula approach doesn’t account for capital changes (contributed, dividend)

Financial statements can always be subject to restatement

RBC ratio decline could involve fraud, or an other wise solid looking asset losing value

- Or pension funding

The statistical relationship is strong, but is not 100% predictive of direction and magnitude

Need to keep the test simple

Could a PC RBC Trend Test be Appropriate?

Hypothetical Formula Change Additional Companies at CAL Resulting

True Alarms, Those Falling <200%

False Alarm

True Alarms to Total Flagged

Change the CAL to 250% 143 17 126 11.9%

Implement a “Life” type trend test for Companies 200%<RBC<250%

117 13 104 11.1%

Implement a combined ratio trend test for Companies 200%<RBC<250%

67 12 55 17.9%

Implement a combined ratio trend test for Companies 200%<RBC<300%

123 22 101 17.9%

Implement a combined ratio trend test for Companies 200%<RBC<350%

150 26 124 17.3%