STAR Results and Analysis (24Nov)

8/10/2019 STAR Results and Analysis (24Nov)

1/46

STAR 2012 & 2013 Results

November 2014


2/46

Contents

Methodology

Portfolio Analyzed

Results

Analysis

Portfolio Outlook until 2020


3/46


4/46

STAR Principles

Sustainable Transport Appraisal Rating

Measures projects contribution to delivering economic,

social and environmental objectives

Sustainable transport objectives

Accounts for risk

Partly guided, qualitative

Ratings:

Outputs:

Aggregate rating of sustainability

Separate ratings by dimension of sustainability

Score Rating

7 to 10 Highly Sustainable

5 to 6 Sustainable

3 to 4 Moderately Sustainable

1 to 2 Marginally Sustainable

-1 to 0 Moderately Unsustainable

-2 to -4 Unsustainable

-5 to -10 Highly Unsustainable


5/46


6/46

Portfolio Rating Methodology

Evaluation choices:

Appraisal stage - based on public RRP documents

Projects approved in 2012 and 2013


7/46

Rating Process

Rating team:

One from each ADB transport division

One external auditor

Common score book

Two raters per project: ADB staff + auditor

Rater independence

Differences resolved in joint meeting

Specific guidance issued for safety and GHG


8/46

Portfolio Analyzed


9/46

Portfolio Analyzed: 46 Projects, $7BillionBy Subsector

!"#! !"#%!"#$%&'( ) * ) !"#$%&'( ) * )

+#,- /",0(1#"' 23 45) 67436789: ;2) 23 4 /",0(1#"' : :) : :) 6 8) 6


10/46

Portfolio Analyzed: 46 Projects, $7BillionBy Geographic Region

!"#! !"#%!"#$%&'( ) * ) !"#$%&'( ) * )

=29(0)*>?24( 8 5


11/46

Results


12/46

STAR Results

!"#! !"#% ! E2)04C,"D=0,>>LKM(',=0,B>%

#" N%>#O

C#-%",'%>LKM(',=0,B>%

KM(',=0,B>%#" @B#P%

C,"D=0,>>LKM(',=0,B>%

#" N%>#O

C#-%",'%>LKM(',=0,B>%

KM(',=0,B>%#" @B#P%

C,"D=0,>>LKM(',=0,B>%

#" N%>#O

C#-%",'%>LKM(',=0,B>%

KM(',=0,B>%#" @B#P%

NL KQ,"% #R S%0-=0D 58) 92) !"D 5;) 9L KM(',=0,B>%

KM(',=0,B>%

C#-%",'%>L KM(',=0,B>%

C,"D=0,>>L KM(',=0,B>%

C#-%",'%>L A0(M(',=0,B>%

:)

6:)

9:)

4:)


13/46

Sub-Ratings by Objective (2 years)

6)26)

34)

5>

KM(',=0,B>% #" @B#P%

C#-%",'%>L KM(',=0,B>%

C,"D=0,>>L KM(',=0,B>% #"N%>#O


14/46

Analysis


15/46

Ratings By Subsector: STAR

U=DQ>L KM(',=0,B>%KM(',=0,B>%C#-%",'%>LKM(',=0,B>%

C,"D=0,>>LKM(',=0,B>%

C#-%",'%>LA0(M(',=0,B>%

+#,- /",0(1#"' +,=> /",0(1#"' ?,'%" /",0(1#"'

@=" /",0(1#"' A"B,0 /",0(1#"' /",0(1#"' CD'E F !#>=&=%(


16/46


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Policy LoanAir Transport

Water

Railway

Metro

BRT

Ring Road

Road Maintenance

Rural Road

Trunk Road

Marginally Sustainable or Less Moderately Sustainable

Sustainable or Above

25

4

1

1

4

3

2

3

2

1

ROADS

URBAN

OTHER

57%

9%

1%

1%

7%

11%

4%

3%

2%

6%

No %lending


17/46


C#-%",'%>LA0(M(',=0,B>%

C,"D=0,>>LKM(',=0,B>%

C#-%",'%>LKM(',=0,B>%

KM(',=0,B>% U=DQ>LKM(',=0,B>%

N+/

?,'%"

+#,- C,=0'%0,0&%+M",> +#,-

C%'"#

+,=>O,L

@=" /",0(1#"'

!#>=&L S#,0+=0D +#,-

/"M0W +#,-


18/46

Ratings By Lending Instrument

U=DQ>L KM(',=0,B>%KM(',=0,B>%C#-%",'%>L

KM(',=0,B>%

C,"D=0,>>L

KM(',=0,B>%

C#-%",'%>L

A0(M(',=0,B>%

!"#$%&' S#,0 @--=X#0,> Y=0,0&=0D K%&'#" S#,0 CYY !#>=&L S#,0


19/46


20/46

Rating: Regional Projects

U=DQ>L KM(',=0,B>%KM(',=0,B>%C#-%",'%>L

KM(',=0,B>%

C,"D=0,>>L KM(',=0,B>%C#-%",'%>L

A0(M(',=0,B>%

+%D=#0,> T#0Z+%D=#0,>


21/46

Other findings from ratings

Economics input in project selection & design limited

Great diversity of practices with regard to road safety

Rather successful mainstreaming of climate resilience

Highest rated projects (STAR):

Best: BRTs, water, road maintenance & rural roads,

Second best: railway/metro and selected trunk road projects

Middle and upper middle-income countries


22/46

Issues with information available

MFF information on 2ndand other tranches

RRP information on project content and impacts/risks

Specific impacts not understood: safety, climate change

Economic analysis weak points: demand, benefit analysis

IPSA on social risks/ benefits

Risk analysis focus on implementation risks, not

development risks


23/46

Lessons on rating process

Ratings subjective, but dispersion of results limited

Good consistence achievable through team rating

Often information not in RRP / has to be inferred Time budget:

STAR desk-based rating process: 4-8 h/project

Initial training + team discussions needed to train raters

Basic self-rating by project officer: 10-30 min

Complete detailed report with indicators, justifications and

ratings possible as part of PPTA


24/46

Performance Outlook

until 2020


25/46

Performance Factors (1)

Portfolio composition:

2009-2013 actual

2014-2017 portfolio pipeline analysis

2018-2020 scenario to close gap with STI targets


26/46

Portfolio composition - ADB

96)

63)

5:)

:)

2:)

6:)

5:)

9:)

3:)

4:)

;:)


27/46

Portfolio assumptions by RD

Z

3::

27:::

273::

67:::

673::

57:::

573::

97:::

6::8Z6:22 6:26Z6:29 6:23Z6:2; 6:2 +#,- +#,- C,=0'%0,0&%

+,=>O,L A"B,0 "#,-( ,0- '",]& G,0,D%G%0' N+/

C%'"# @=" /",0(1#"' ?,'%"

Z

3::

27:::

273::

67:::

673::

57:::

573::

97:::

6::8Z6:22 6:26Z6:29 6:23Z6:2; 6:2


28/46


Z

3::

27:::

273::

67:::

673::

57:::

573::

97:::

973::

6::8Z6:22 6:26Z6:29 6:23Z6:2; 6:2 +#,- +#,- C,=0'%0,0&%

+,=>O,L A"B,0 "#,-( ,0- '",]& G,0,D%G%0' N+/

C%'"# @=" /",0(1#"' ?,'%"

Z

3::

27:::

273::

67:::

673::

57:::

573::

97:::

973::

6::8Z6:22 6:26Z6:29 6:23Z6:2; 6:2


29/46


Z

2::

6::

5::

9::

3::

4::

;::

+#,-

/"M0W +#,-


30/46

Portfolio composition - ADB

96)

63)

5:)

:)

2:)

6:)

5:)

9:)

3:)

4:)

;:)


31/46

STAR Performance Assumptions

2009-2011 : historic average ratings by subsector

2012-2013 : actuals

2014-2020: 2 scenarios

Historic average ratings by subsector

STI mainstreamingscenario: stronger action on maximizingsocial and environmental positive impacts+ moderately better

economics


32/46

Outlook on sustainability

6::8Z

6:22

6:26Z6:2

9

/'0J'*;' 36)9


33/46

Outlook With Mainstreaming

6::8Z

6:22

6:26Z6:2

9

/'0J'*;' 36)9


34/46

Scope for Change

%GD %+D

!!D

#"D

39) 35)

36)

32)

GD#%D

!.D

%QD

6::8Z6:22 6:26Z6:29 6:23Z6:2; 6:2L (M(',=0,B>%

C,"D=0,>>L (M(',=0,B>% ,0- >%((


35/46

S%((

KM(',=0,B>%

C,"D=0,>>L

KM(',=0,B>%

C#-%",'%>L

KM(',=0,B>%

KM(',=0,B>% U=DQ>L

KM(',=0,B>%

/"M0W +#,- +M",> +#,- +#,- C,=0'%0,0&%

+,=>O,L A"B,0 "#,-( ,0- '",]& G,0,D%G%0' N+/

C%'"# @=" /",0(1#"' ?,'%"

!""Q>!"##

%GD

GD

#"D

%QD

S%((

KM(',=0,B>%

C,"D=0,>>L

KM(',=0,B>%

C#-%",'%>L

KM(',=0,B>%

KM(',=0,B>% U=DQ>L

KM(',=0,B>%

Scope for change

5&R

!"#G>!"!"


36/46


37/46

Conclusions and next steps

Proposed discussion by division of results and outlook

STAR options

Annual reporting on new projects and portfolio

Gradual introduction in parallel to informal peer review

Use of framework during PPTA on voluntary basis

Review of future project pipelines by country or region?


38/46

Thank you


39/46

Additional Slides


40/46

Portfolio composition assumptions

!""Q>!"## !"#!>!"#+ !"#F>!"#- !"#G>!"!"S5&R &)0

!"!"

H'): 27;:: 6733: 6793: 675:: 2743:

H);* 53: 53: 4:: 33: 27:::

T0M)9 2:: 93: 3:: 27::: 276::A;0 3: 3: 2:: 2:: 3:

?)(20 3: 2:: 3: 2:: 2::

&'()* !,!"" %,F"" %,-"" +,""" +,"""

!""Q>!"## !"#!>!"#+ !"#F>!"#- !"#G>!"!"S

=?HI 8:: 27::: 272


41/46

Results profiles by sector

5(029=%0&%

=0&"%,(=0D>L

G,=0('"%,G

C,0L 1"#$%&'( Q,P% >#O %#G=&"%'M"0(` #P%"-%(=D07 "%D=#0,> 1"#$%&'(

T%D,XP% (,R%'L OQ%0 O=-%0=0D7 0#

(L('%G,X& ,11"#,&Q '# ,M-='

C#-%",'% 0%D,XP% =G1,&'( #0%0P="#0G%0'7 >=G='%- =G1"#P%G%0'(

1#((=B>%

T# (=-%",X#0 R#" '",0(1#"' (%"P=&%(

Y%O 1"#$%&'( ,--"%(( (%&'#"ZO=-% =((M%(

C,=0'%0,0&% "=(W

HC0)* 0'):4

K'"#0D (#&=,> B%0%a'(

_%0%",>>L 1#(=XP%

%0P="#0G%0',>

=G1,&'(

J#G=& B%0%a'( G,L B% Q=DQ #" >#O

-%1%0-=0D #0 ,11"#,&Q ,0- ,"%,

T# (=-%",X#0 R#" '",0(1#"' (%"P=&%(7

M(%" 0%%-(

C,=0'%0,0&%


42/46

Results profiles by sector (2)

5(029

B%0%a'(

K,R%'L =G1,&'( #c%0 0%D,XP% =R 0#'&,"%RM>>L G=XD,'%-

VH&4 K'"#0D %#G=&( =R ,'

D",-%

K'"#0D (#&=,>d%0P="#0G%0'

e","%>L G,$#" 'Q#MDQf

\G1>%G%0',X#0 G1>%g='L

L2(0' K=G=>," B%0%a'( ,( N+/1"#$%&'(7 BM' >%(( P,>M%Z

R#"ZG#0%L

K=G1>%" =G1>%G%0',X#0 'Q,0 N+/(7

(' #P%""M0(

H);*W)K4 K'"#0D (#&=,>d%0P="#0G%0'B%0%a'( =R G#-,> (Q=c

K'"#0D %#G=&( =R R"%=DQ'

J#G=&(d(M(',=0,B=>='L R#"

1,((%0D%" ",=>O,L(

+=(W( 'Q,' G#-,> (Q=c 0#' ,&Q=%P%-d

(M(',=0%-


43/46

Results profiles by sector (3)

='PP29(4

?)(20

(0)94X'0(

\0>,0- O,'%" '",0(1#"'` %0P="#0G%0',> =G1,&'( G,L B% G=g%-7(#&=,> =G1,&'( >=G='%-7 %#G=&( -%1%0- #0 G#-,> (Q=c 1#'%0X,>

\0'%"Z=(>,0- e"%G#'% ,"%,(f` Vh %#G=&( e%(1E "%>=,B=>='Lf7 ('"#0D

(#&=,> B%0%a'( e(,R%'L7 ,&&%((f7 1#(=XP% %0P="#0G%0' e%G=((=#0(7

"%(=>=%0&%fA;0X'0(4 K'"#0D %#G=&(

T# -="%&' 1#(=XP% =G1,&'( #0 1#P%"'Ld(#&=,>

T%D,XP% #0 %0P="#0G%0'7 %(1%&=,>>L _U_7 BM' =G1"#P%- "%(=>=%0&%

H;9< 0'): C#-%",'% 1#(=XP% %#G=&(

K'"#0D 0%D,XP% #0 (#&=,>7 %0P="#0G%0'H);*W)K4 K'"#0D (#&=,>d%0P="#0G%0' B%0%a'( =R G#-,> (Q=c

K'"#0D %#G=&( =R R"%=DQ'

J#G=&(d(M(',=0,B=>='L R#" 1,((%0D%" ",=>O,L(

+=(W( 'Q,' G#-,> (Q=c 0#' ,&Q=%P%-d(M(',=0%-


44/46

STAR Scoring: Step 1


45/46

Step 2


46/46

Rating HighlyUnsustainable Unsustainable

ModeratelyUnsustainable

MarginallySustainable

ModeratelySustainable Sustainable

HighlySustainable

Score -5 to -10 -2 to -4 -1 to 0 1 to 2 3 to 4 5 to 6 7 to 10

STAR Scoring: Step 3

STAR Results and Analysis (24Nov)

Documents