Russian banks sovereign ratings: a comparative study

Russian banks sovereign ratings: a comparative study

S.Smirnov, A. Kosyanenko, V. Naumenko,V. Lapshin, E. BogatyrevaHigher School of Economics, Moscow

2

IntroductionThe aim of the research was to assess credit ratings’ quality for the purpose of Bank Counterparties Credit Risk Assessment, in order to use them in credit risk models under IRB approach.

Presentation plan:General requirements to external credit ratingsProperties of migration matricesAssessment of conditional intervals for PDEntropy measuresMapping to model-based PDConclusions

3

Using credit ratings in modelsProbability of default (PD) is one of the major building blocks in credit risk management.

External credit rating can be used as an input variable in PD-models. Using credit ratings in addition to other sources of information about borrower's credit risk (e.g. financial ratios, market-based information) may improve the prediction power of credit risk models (see [Kealhofer, 2003], [Loffler, 2007])

By using credit ratings as input to credit risk models one should assess the uncertainty of these variables (see [Basel II]).

4

Basel II requirements for ratings

Under the IRB approach different exposures should be treated separately, e.g. corporate, sovereign, bank, retail, and equity (Basel II, §215).

“Irrespective of whether a bank is using external, internal, or pooled data sources, or a combination of the three, for its PD estimation, the length of the underlying historical observation period used must be at least five years for at least one source. If the available observation period spans a longer period for any source, and this data are relevant and material, this longer period must be used” (Basel II, § 463)

Ratings assigned by for external credit assessment institution (ECAI) should be recognized by regulator and satisfy 6 eligibility criteria: Objectivity, Independence, International access/Transparency, Disclosure, Resources, Credibility (Basel II, § 91).

:

5

National rating agencies in Russia

There are four largest national rating agencies recognized by the Russian Central Bank: RusRating, Expert RA, National rating agency (NRA) and AK&M.

Historical data for the purpose of the research was provided by RusRating and AK&M. Information about ratings assigned by NRA and Expert RA was taken from their web-sites.

Rating data contains monthly information about rating assigned to Russian banks from January, 2001 to May, 2010.

6

Dynamics of rating assignment

In October 2008 Russian Central Bank recognized ratings assigned by national rating agencies (RusRating, Expert RA, NRA and AK&M) for the purpose of granting unsecured loans.

As of May 1, 2010 there were the following numbers of efficient credit ratings in the banking sector:

RusRating – 51

AK&M – 33

NRA – 65

Expert RA - 113

7

Rating TransitionsRating agencies are not likely to revise their ratings: since 2001 there were only few rating downgrades. The number of rating upgrades is much more substantial.

Total Since 01.01.2008

Downgrades

Upgrades

Obs No.

Downgrades

Upgrades

Obs No.

RusRating 11 85 3375 2 34 1315

Expert RA 13 18 2365 13 18 2261

NRA 0 19 922 0 19 888

AK&M 1 4 611 1 4 601

There are several cases when ratings withdrawals due to termination of the contract followed rating downgrades in a month or two. In the beginning of 2010 Expert RA had 5 such facts.

Rating philosophies• Ratings Point in Time indicate the current

probability of issuer’s default. They are likely to change significantly during the bad times.

• Ratings Through the Cycle indicate the average probability of default during the long period of time. They are not likely to change during the economic cycle. However it’s likely that default probabilities associated with ratings grades do change.

Transition matrix computation

• Cohort approach takes into account only the initial and terminal states of the institution in question.

• Duration approach takes into account time spent in every rating grade.

• Under conditions of first order Markov process, time homogenous matrix structure for Russian agencies these approaches coincide.

10

Migration matrices: the case of S&PTypical S&P migration matrix (expressed in monthly transition probabilities for the purpose of comparison with Russian rating agencies):

Key features are:

Distinct diagonal line (taking into account aggregation of rating classes).

Existence of non-diagonal elements which gives evidence of rating upgrades and downgrades.

Source: S&P 2008 Global Corporate Default Study and Rating Transitions

11

Migration matrices: RusRating

A+ A A- BBB+ BBB BBB- BB+ BB BB- B+ B B- CCC+ CCCA+ 1,000 0 0 0 0 0 0 0 0 0 0 0 0 0A 0 1,000 0 0 0 0 0 0 0 0 0 0 0 0A- 0 0 1,000 0 0 0 0 0 0 0 0 0 0 0BBB+ 0 0 0 0,988 0,012 0 0 0 0 0 0 0 0 0BBB 0 0 0 0,013 0,981 0,006 0 0 0 0 0 0 0 0BBB- 0,003 0 0 0,003 0,011 0,980 0,003 0 0 0 0 0 0 0BB+ 0 0 0 0 0 0,019 0,977 0,002 0 0 0,002 0 0 0BB 0 0 0 0 0,003 0 0,028 0,969 0 0 0 0 0 0BB- 0 0 0 0 0 0,003 0,004 0,013 0,975 0,004 0 0 0 0B+ 0 0 0 0 0 0,002 0 0 0,036 0,959 0,002 0 0 0B 0 0 0 0 0 0 0 0,003 0,007 0,030 0,956 0 0,003 0B- 0 0 0 0 0 0 0 0 0,007 0 0,035 0,950 0,007 0CCC+ 0 0 0 0 0 0 0 0 0 0 0 0,063 0,937 0CCC 0 0 0 0 0 0 0 0 0 0 0 0 0,111 0,889CCC- 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Data on rating transitions had monthly frequency. Each number in a diagonal cell of a migration matrix shows probability of the fact that rating will not change in a month period of time. Numbers below the diagonal line show the probability of rating upgrades, above –downgrades.

RusRating migration matrix demonstrates sufficient amount of both upgrades and downgrades.

12

Migration matrices: Expert RA, AK&M

A++ A+ A B++ B+ B C++ C+ C DA++ 1,000 0 0 0 0 0 0 0 0 0A+ 0 1,000 0 0 0 0 0 0 0 0A 0 0,003 0,982 0,013 0 0 0 0 0 0B++ 0 0 0,004 0,975 0,006 0 0 0 0 0B+ 0 0 0 0,026 0,930 0 0 0 0 0,003B 0 0 0 0 0,033 0,945 0 0 0 0,011C++ 0 0 0 0 0 0 0 0 0 0C+ 0 0 0 0 0 0 0 0 0 0C 0 0 0 0 0 0 0 0 0 0D 0 0 0 0 0 0 0 0 0 0

Migration matrices for Expert RA and AK&M have much less non-diagonal elements than migration matrix for RusRating.

A++ A+ A A- B++ B+ B C++ C+ C DA++ 0 0 0 0 0 0 0 0 0 0 0A+ 0 0,975 0 0 0 0 0 0 0 0 0A 0 0 0,974 0 0,005 0 0 0 0 0 0B++ 0 0 0,007 0 0,975 0 0 0 0 0 0B+ 0 0 0 0 0,042 0,896 0 0 0 0 0B 0 0 0 0 0 0 0,909 0 0 0 0C++ 0 0 0 0 0 0 0 0 0 0 0C+ 0 0 0 0 0 0 0 0 0 0 0C 0 0 0 0 0 0 0 0 0 0 0D 0 0 0 0 0 0 0 0 0 0 0

Expert RA

AK&M

13

Migration matrices: NRA

AAA AA+ AA AA- A+ A A- BBB+ BBB BBB- BB+ BB BB- DAAA 0,949 0 0 0,026 0 0 0 0 0 0 0 0 0 0AA+ 0,023 0,977 0 0 0 0 0 0 0 0 0 0 0 0AA 0 0 0,964 0 0 0 0 0 0 0 0 0 0 0AA- 0 0 0 1,000 0 0 0 0 0 0 0 0 0 0A+ 0 0 0 0,008 0,992 0 0 0 0 0 0 0 0 0A 0 0 0 0 0,022 0,978 0 0 0 0 0 0 0 0A- 0 0 0 0 0 0,029 0,968 0 0 0 0 0 0 0BBB+ 0 0 0 0 0 0 0,047 0,922 0 0 0 0 0 0BBB 0 0 0 0 0 0 0,081 0 0,919 0 0 0 0 0BBB- 0 0 0 0 0 0 0 0 0 1,000 0 0 0 0BB+ 0 0 0 0 0 0 0 0 0 0,042 0,958 0 0 0BB 0 0 0 0 0 0 0 0 0 0 0 0,800 0 0,200BB- 0 0 0 0 0 0 0 0 0 0 0 0 0,909 0

Rating history of NRA has almost no downgrades.

14

Confidence intervalsConfidence interval is an interval with lower (L) and upper (U) bounds that covers the unknown true parameter, i.e. L < p < U with some predefined probability:

Prob{L < p < U} = 1 − α.

Confidence intervals is a standard industry tool to assess uncertainty of PD estimations (see, for example [OeNB, 2004]).

One of the major factors that influence the length of confidence intervals for PD is the amount of data available. There are research papers that show that to some extent it is impossible to statistically distinguish investment grade rating classes (see [Lawrenz, 2008]).

15

Confidence interval methodologyTo calculate confidence intervals for PD one should:Fit a priori unconditional PD distribution from external data

(Russian Deposit Insurance Agency PD model) as Beta distribution – very good agreement. Estimated parameters (a,b) = (1,16.3).

Regard each month for each bank with given rating as a trial: success if no default, failure if default. Form posterior distribution for PD: Beta (number of defaults + 1, number of non-defaults + 16.3).

Find 95% confidence interval for Beta distribution with estimated parameters and plot together with (number of observations)-1.

16

Confidence intervals: RusRating

17

Confidence intervals: Expert RA

18

Confidence intervals: NRA

19

Confidence intervals: AK&M

20

Conditional entropyConditional entropy measures new information (in bits) contained in each successive rating value (randomly selected).

To understand what happened to credit quality of the rating object (3 possibilities: whether it improved, deteriorated or remained the same) it is necessary to obtain data over the period of (months):

RusRating – 9; NRA– 11; Expert RA– 13; AK&M – 21.

Given migration matrix pi,j and unconditional probabilities pi (expected) conditional entropy is

, 2 ,1 1

logn n

i i j i ji j

p p p

21

Mapping to model assessed PDRatings were mapped to PD estimates derived from econometric model based on balance sheet data. This model is used by Deposit Insurance Agency to assess PD of banks –participants of Deposit Insurance System.

The following measures were calculated in order to estimate the accuracy of ratings:

average PD for each rating grade;

confidence intervals for PD according to each rating grade;

probability that PD associated with different rating grades will coincide.

22

Rating grades comparison methodology

Given PD samples for 2 different rating values, test a hypothesis: “these 2 samples really come from the same PD distribution”.

Non-parametric Kolmogorov-Smirnov test using

as test statistics.

Enter the p-value for each pair of rating values (including general population) in a table.

1 2ma ) ( )x (x

CDF x CDF x

23

Mapping: RusRating

All AAA BBB+ BBB BBB- BB+ BB BB- B+ B B- CCC+ CCCAll - 58% 56% 1% 0% 0% 11% 1% 1% 51% 0% 1% 0%AAA 58% - 92% 4% 27% 53% 91% 38% 16% 79% 0% 5% 0%BBB+ 56% 92% - 2% 37% 41% 47% 25% 8% 71% 0% 9% 0%BBB 1% 4% 2% - 0% 0% 0% 4% 1% 1% 2% 15% 0%BBB- 0% 27% 37% 0% - 65% 3% 0% 0% 10% 0% 0% 0%BB+ 0% 53% 41% 0% 65% - 9% 0% 0% 2% 0% 0% 0%BB 11% 91% 47% 0% 3% 9% - 0% 0% 15% 0% 0% 0%BB- 1% 38% 25% 4% 0% 0% 0% - 5% 3% 0% 20% 0%B+ 1% 16% 8% 1% 0% 0% 0% 5% - 3% 0% 51% 0%B 51% 79% 71% 1% 10% 2% 15% 3% 3% - 0% 1% 0%B- 0% 0% 0% 2% 0% 0% 0% 0% 0% 0% - 20% 0%CCC+ 1% 5% 9% 15% 0% 0% 0% 20% 51% 1% 20% - 0%CCC 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% -

Probability of PD coincidence

24

Mapping: Expert RA

All A++ A+ A B++ B+ BAll 100% 0% 0% 4% 12% 6% 6%A++ 0% 100% 3% 0% 0% 0% 0%A+ 0% 3% 100% 0% 0% 0% 0%A 4% 0% 0% 100% 1% 0% 0%B++ 12% 0% 0% 1% 100% 34% 34%B+ 6% 0% 0% 0% 34% 100% 100%B 6% 0% 0% 0% 34% 100% 100%

Probability of PD coincidence

25

Mapping: NRA

Probability of PD coincidenceAll AAA AA+ AA A+ A A- BBB+ BBB BB+ BB BB-

All - 13% 25% 0% 24% 9% 0% 17% 34% 1% 93% 6%AAA 13% - 32% 0% 8% 2% 0% 2% 17% 1% 82% 4%AA+ 25% 32% - 0% 5% 28% 3% 11% 51% 12% 72% 51%AA 0% 0% 0% - 0% 1% 3% 0% 0% 0% 36% 0%A+ 24% 8% 5% 0% - 5% 1% 38% 52% 0% 59% 5%A 9% 2% 28% 1% 5% - 5% 3% 15% 4% 100% 25%A- 0% 0% 3% 3% 1% 5% - 3% 13% 0% 67% 3%BBB+ 17% 2% 11% 0% 38% 3% 3% - 38% 0% 70% 3%BBB 34% 17% 51% 0% 52% 15% 13% 38% - 5% 74% 30%BB+ 1% 1% 12% 0% 0% 4% 0% 0% 5% - 11% 74%BB 93% 82% 72% 36% 59% 100% 67% 70% 74% 11% - 38%BB- 6% 4% 51% 0% 5% 25% 3% 3% 30% 74% 38% -

26

Mapping: AK&M

All A+ A B++ B+ BAll - 29% 0% 47% 3% 5%A+ 29% - 19% 43% 4% 1%A 0% 19% - 0% 0% 1%B++ 47% 43% 0% - 37% 6%B+ 3% 4% 0% 37% - 27%B 5% 1% 1% 6% 27% -

Probability of PD coincidence

27

Conclusions It is reasonable to use external credit rating as an input parameter in

credit risk models. Accuracy of these rating assessment should be taken into account.

However according to our findings we can not recommend to use ratings assigned by national rating agencies in credit risk models as the only source of information due to the lack of credibility:

rating are not likely to be downgraded;

sometimes there is no uniform dependence between rating grades and PD;

in most cases we can not differentiate between rating grades.

When new data will be accumulated it will be possible to estimate rating accuracy once more and probably use ratings as an alternative source of credit quality information.

28

References1. Basel Committee on Banking Supervision. International

Convergence of Capital Measurement and Capital Standards. A Revised Framework. Bank for International Settlements. June 2006 (Basel II).

2. Kealhofer, 2003. Quantifying Credit Risk I: Default Prediction. Finandal Analysts Journal, 59, pp. 30-44.

3. Loffler, 2007. The Complementary Nature of Ratings and Market-Based Measures of Default Risk. The Journal of Fixed income, pp. 38-47.

4. OeNB (Oesterreichische Nationalbank), 2004. Rating Models and Validation in Guidelines on Credit Risk Management.

5. Lawrenz J. Assessing the estimation uncertainty of default probabilities.// Kredit und Kapital. -2008.-Vol. 41 (2). pp. 217-238.

29

Thank you for your attention!