8/10/2019 STAR Results and Analysis (24Nov)
1/46
STAR 2012 & 2013 Results
November 2014
8/10/2019 STAR Results and Analysis (24Nov)
2/46
Contents
Methodology
Portfolio Analyzed
Results
Analysis
Portfolio Outlook until 2020
8/10/2019 STAR Results and Analysis (24Nov)
3/46
8/10/2019 STAR Results and Analysis (24Nov)
4/46
STAR Principles
Sustainable Transport Appraisal Rating
Measures projects contribution to delivering economic,
social and environmental objectives
Sustainable transport objectives
Accounts for risk
Partly guided, qualitative
Ratings:
Outputs:
Aggregate rating of sustainability
Separate ratings by dimension of sustainability
Score Rating
7 to 10 Highly Sustainable
5 to 6 Sustainable
3 to 4 Moderately Sustainable
1 to 2 Marginally Sustainable
-1 to 0 Moderately Unsustainable
-2 to -4 Unsustainable
-5 to -10 Highly Unsustainable
8/10/2019 STAR Results and Analysis (24Nov)
5/46
8/10/2019 STAR Results and Analysis (24Nov)
6/46
Portfolio Rating Methodology
Evaluation choices:
Appraisal stage - based on public RRP documents
Projects approved in 2012 and 2013
8/10/2019 STAR Results and Analysis (24Nov)
7/46
Rating Process
Rating team:
One from each ADB transport division
One external auditor
Common score book
Two raters per project: ADB staff + auditor
Rater independence
Differences resolved in joint meeting
Specific guidance issued for safety and GHG
8/10/2019 STAR Results and Analysis (24Nov)
8/46
Portfolio Analyzed
8/10/2019 STAR Results and Analysis (24Nov)
9/46
Portfolio Analyzed: 46 Projects, $7BillionBy Subsector
!"#! !"#%!"#$%&'( ) * ) !"#$%&'( ) * )
+#,- /",0(1#"' 23 45) 67436789: ;2) 23 4 /",0(1#"' : :) : :) 6 8) 6
8/10/2019 STAR Results and Analysis (24Nov)
10/46
Portfolio Analyzed: 46 Projects, $7BillionBy Geographic Region
!"#! !"#%!"#$%&'( ) * ) !"#$%&'( ) * )
=29(0)*>?24( 8 5
8/10/2019 STAR Results and Analysis (24Nov)
11/46
Results
8/10/2019 STAR Results and Analysis (24Nov)
12/46
STAR Results
!"#! !"#% ! E2)04C,"D=0,>>LKM(',=0,B>%
#" N%>#O
C#-%",'%>LKM(',=0,B>%
KM(',=0,B>%#" @B#P%
C,"D=0,>>LKM(',=0,B>%
#" N%>#O
C#-%",'%>LKM(',=0,B>%
KM(',=0,B>%#" @B#P%
C,"D=0,>>LKM(',=0,B>%
#" N%>#O
C#-%",'%>LKM(',=0,B>%
KM(',=0,B>%#" @B#P%
NL KQ,"% #R S%0-=0D 58) 92) !"D 5;) 9L KM(',=0,B>%
KM(',=0,B>%
C#-%",'%>L KM(',=0,B>%
C,"D=0,>>L KM(',=0,B>%
C#-%",'%>L A0(M(',=0,B>%
:)
6:)
9:)
4:)
8/10/2019 STAR Results and Analysis (24Nov)
13/46
Sub-Ratings by Objective (2 years)
6)26)
34)
5>
KM(',=0,B>% #" @B#P%
C#-%",'%>L KM(',=0,B>%
C,"D=0,>>L KM(',=0,B>% #"N%>#O
8/10/2019 STAR Results and Analysis (24Nov)
14/46
Analysis
8/10/2019 STAR Results and Analysis (24Nov)
15/46
Ratings By Subsector: STAR
U=DQ>L KM(',=0,B>%KM(',=0,B>%C#-%",'%>LKM(',=0,B>%
C,"D=0,>>LKM(',=0,B>%
C#-%",'%>LA0(M(',=0,B>%
+#,- /",0(1#"' +,=> /",0(1#"' ?,'%" /",0(1#"'
@=" /",0(1#"' A"B,0 /",0(1#"' /",0(1#"' CD'E F !#>=&=%(
8/10/2019 STAR Results and Analysis (24Nov)
16/46
Ratings By Subsector: STAR
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Policy LoanAir Transport
Water
Railway
Metro
BRT
Ring Road
Road Maintenance
Rural Road
Trunk Road
Marginally Sustainable or Less Moderately Sustainable
Sustainable or Above
25
4
1
1
4
3
2
3
2
1
ROADS
URBAN
OTHER
57%
9%
1%
1%
7%
11%
4%
3%
2%
6%
No %lending
8/10/2019 STAR Results and Analysis (24Nov)
17/46
Ratings By Subsector: STAR
C#-%",'%>LA0(M(',=0,B>%
C,"D=0,>>LKM(',=0,B>%
C#-%",'%>LKM(',=0,B>%
KM(',=0,B>% U=DQ>LKM(',=0,B>%
N+/
?,'%"
+#,- C,=0'%0,0&%+M",> +#,-
C%'"#
+,=>O,L
@=" /",0(1#"'
!#>=&L S#,0+=0D +#,-
/"M0W +#,-
8/10/2019 STAR Results and Analysis (24Nov)
18/46
Ratings By Lending Instrument
U=DQ>L KM(',=0,B>%KM(',=0,B>%C#-%",'%>L
KM(',=0,B>%
C,"D=0,>>L
KM(',=0,B>%
C#-%",'%>L
A0(M(',=0,B>%
!"#$%&' S#,0 @--=X#0,> Y=0,0&=0D K%&'#" S#,0 CYY !#>=&L S#,0
8/10/2019 STAR Results and Analysis (24Nov)
19/46
8/10/2019 STAR Results and Analysis (24Nov)
20/46
Rating: Regional Projects
U=DQ>L KM(',=0,B>%KM(',=0,B>%C#-%",'%>L
KM(',=0,B>%
C,"D=0,>>L KM(',=0,B>%C#-%",'%>L
A0(M(',=0,B>%
+%D=#0,> T#0Z+%D=#0,>
8/10/2019 STAR Results and Analysis (24Nov)
21/46
Other findings from ratings
Economics input in project selection & design limited
Great diversity of practices with regard to road safety
Rather successful mainstreaming of climate resilience
Highest rated projects (STAR):
Best: BRTs, water, road maintenance & rural roads,
Second best: railway/metro and selected trunk road projects
Middle and upper middle-income countries
8/10/2019 STAR Results and Analysis (24Nov)
22/46
Issues with information available
MFF information on 2ndand other tranches
RRP information on project content and impacts/risks
Specific impacts not understood: safety, climate change
Economic analysis weak points: demand, benefit analysis
IPSA on social risks/ benefits
Risk analysis focus on implementation risks, not
development risks
8/10/2019 STAR Results and Analysis (24Nov)
23/46
Lessons on rating process
Ratings subjective, but dispersion of results limited
Good consistence achievable through team rating
Often information not in RRP / has to be inferred Time budget:
STAR desk-based rating process: 4-8 h/project
Initial training + team discussions needed to train raters
Basic self-rating by project officer: 10-30 min
Complete detailed report with indicators, justifications and
ratings possible as part of PPTA
8/10/2019 STAR Results and Analysis (24Nov)
24/46
Performance Outlook
until 2020
8/10/2019 STAR Results and Analysis (24Nov)
25/46
Performance Factors (1)
Portfolio composition:
2009-2013 actual
2014-2017 portfolio pipeline analysis
2018-2020 scenario to close gap with STI targets
8/10/2019 STAR Results and Analysis (24Nov)
26/46
Portfolio composition - ADB
96)
63)
5:)
:)
2:)
6:)
5:)
9:)
3:)
4:)
;:)
8/10/2019 STAR Results and Analysis (24Nov)
27/46
Portfolio assumptions by RD
Z
3::
27:::
273::
67:::
673::
57:::
573::
97:::
6::8Z6:22 6:26Z6:29 6:23Z6:2; 6:2 +#,- +#,- C,=0'%0,0&%
+,=>O,L A"B,0 "#,-( ,0- '",]& G,0,D%G%0' N+/
C%'"# @=" /",0(1#"' ?,'%"
Z
3::
27:::
273::
67:::
673::
57:::
573::
97:::
6::8Z6:22 6:26Z6:29 6:23Z6:2; 6:2
8/10/2019 STAR Results and Analysis (24Nov)
28/46
Portfolio assumptions by RD
Z
3::
27:::
273::
67:::
673::
57:::
573::
97:::
973::
6::8Z6:22 6:26Z6:29 6:23Z6:2; 6:2 +#,- +#,- C,=0'%0,0&%
+,=>O,L A"B,0 "#,-( ,0- '",]& G,0,D%G%0' N+/
C%'"# @=" /",0(1#"' ?,'%"
Z
3::
27:::
273::
67:::
673::
57:::
573::
97:::
973::
6::8Z6:22 6:26Z6:29 6:23Z6:2; 6:2
8/10/2019 STAR Results and Analysis (24Nov)
29/46
Portfolio assumptions by RD
Z
2::
6::
5::
9::
3::
4::
;::
+#,-
/"M0W +#,-
8/10/2019 STAR Results and Analysis (24Nov)
30/46
Portfolio composition - ADB
96)
63)
5:)
:)
2:)
6:)
5:)
9:)
3:)
4:)
;:)
8/10/2019 STAR Results and Analysis (24Nov)
31/46
STAR Performance Assumptions
2009-2011 : historic average ratings by subsector
2012-2013 : actuals
2014-2020: 2 scenarios
Historic average ratings by subsector
STI mainstreamingscenario: stronger action on maximizingsocial and environmental positive impacts+ moderately better
economics
8/10/2019 STAR Results and Analysis (24Nov)
32/46
Outlook on sustainability
6::8Z
6:22
6:26Z6:2
9
/'0J'*;' 36)9
8/10/2019 STAR Results and Analysis (24Nov)
33/46
Outlook With Mainstreaming
6::8Z
6:22
6:26Z6:2
9
/'0J'*;' 36)9
8/10/2019 STAR Results and Analysis (24Nov)
34/46
Scope for Change
%GD %+D
!!D
#"D
39) 35)
36)
32)
GD#%D
!.D
%QD
6::8Z6:22 6:26Z6:29 6:23Z6:2; 6:2L (M(',=0,B>%
C,"D=0,>>L (M(',=0,B>% ,0- >%((
8/10/2019 STAR Results and Analysis (24Nov)
35/46
S%((
KM(',=0,B>%
C,"D=0,>>L
KM(',=0,B>%
C#-%",'%>L
KM(',=0,B>%
KM(',=0,B>% U=DQ>L
KM(',=0,B>%
/"M0W +#,- +M",> +#,- +#,- C,=0'%0,0&%
+,=>O,L A"B,0 "#,-( ,0- '",]& G,0,D%G%0' N+/
C%'"# @=" /",0(1#"' ?,'%"
!""Q>!"##
%GD
GD
#"D
%QD
S%((
KM(',=0,B>%
C,"D=0,>>L
KM(',=0,B>%
C#-%",'%>L
KM(',=0,B>%
KM(',=0,B>% U=DQ>L
KM(',=0,B>%
Scope for change
5&R
!"#G>!"!"
8/10/2019 STAR Results and Analysis (24Nov)
36/46
8/10/2019 STAR Results and Analysis (24Nov)
37/46
Conclusions and next steps
Proposed discussion by division of results and outlook
STAR options
Annual reporting on new projects and portfolio
Gradual introduction in parallel to informal peer review
Use of framework during PPTA on voluntary basis
Review of future project pipelines by country or region?
8/10/2019 STAR Results and Analysis (24Nov)
38/46
Thank you
8/10/2019 STAR Results and Analysis (24Nov)
39/46
Additional Slides
8/10/2019 STAR Results and Analysis (24Nov)
40/46
Portfolio composition assumptions
!""Q>!"## !"#!>!"#+ !"#F>!"#- !"#G>!"!"S5&R &)0
!"!"
H'): 27;:: 6733: 6793: 675:: 2743:
H);* 53: 53: 4:: 33: 27:::
T0M)9 2:: 93: 3:: 27::: 276::A;0 3: 3: 2:: 2:: 3:
?)(20 3: 2:: 3: 2:: 2::
&'()* !,!"" %,F"" %,-"" +,""" +,"""
!""Q>!"## !"#!>!"#+ !"#F>!"#- !"#G>!"!"S
=?HI 8:: 27::: 272
8/10/2019 STAR Results and Analysis (24Nov)
41/46
Results profiles by sector
5(029=%0&%
=0&"%,(=0D>L
G,=0('"%,G
C,0L 1"#$%&'( Q,P% >#O %#G=&"%'M"0(` #P%"-%(=D07 "%D=#0,> 1"#$%&'(
T%D,XP% (,R%'L OQ%0 O=-%0=0D7 0#
(L('%G,X& ,11"#,&Q '# ,M-='
C#-%",'% 0%D,XP% =G1,&'( #0%0P="#0G%0'7 >=G='%- =G1"#P%G%0'(
1#((=B>%
T# (=-%",X#0 R#" '",0(1#"' (%"P=&%(
Y%O 1"#$%&'( ,--"%(( (%&'#"ZO=-% =((M%(
C,=0'%0,0&% "=(W
HC0)* 0'):4
K'"#0D (#&=,> B%0%a'(
_%0%",>>L 1#(=XP%
%0P="#0G%0',>
=G1,&'(
J#G=& B%0%a'( G,L B% Q=DQ #" >#O
-%1%0-=0D #0 ,11"#,&Q ,0- ,"%,
T# (=-%",X#0 R#" '",0(1#"' (%"P=&%(7
M(%" 0%%-(
C,=0'%0,0&%
8/10/2019 STAR Results and Analysis (24Nov)
42/46
Results profiles by sector (2)
5(029
B%0%a'(
K,R%'L =G1,&'( #c%0 0%D,XP% =R 0#'&,"%RM>>L G=XD,'%-
VH&4 K'"#0D %#G=&( =R ,'
D",-%
K'"#0D (#&=,>d%0P="#0G%0'
e","%>L G,$#" 'Q#MDQf
\G1>%G%0',X#0 G1>%g='L
L2(0' K=G=>," B%0%a'( ,( N+/1"#$%&'(7 BM' >%(( P,>M%Z
R#"ZG#0%L
K=G1>%" =G1>%G%0',X#0 'Q,0 N+/(7
(' #P%""M0(
H);*W)K4 K'"#0D (#&=,>d%0P="#0G%0'B%0%a'( =R G#-,> (Q=c
K'"#0D %#G=&( =R R"%=DQ'
J#G=&(d(M(',=0,B=>='L R#"
1,((%0D%" ",=>O,L(
+=(W( 'Q,' G#-,> (Q=c 0#' ,&Q=%P%-d
(M(',=0%-
8/10/2019 STAR Results and Analysis (24Nov)
43/46
Results profiles by sector (3)
='PP29(4
?)(20
(0)94X'0(
\0>,0- O,'%" '",0(1#"'` %0P="#0G%0',> =G1,&'( G,L B% G=g%-7(#&=,> =G1,&'( >=G='%-7 %#G=&( -%1%0- #0 G#-,> (Q=c 1#'%0X,>
\0'%"Z=(>,0- e"%G#'% ,"%,(f` Vh %#G=&( e%(1E "%>=,B=>='Lf7 ('"#0D
(#&=,> B%0%a'( e(,R%'L7 ,&&%((f7 1#(=XP% %0P="#0G%0' e%G=((=#0(7
"%(=>=%0&%fA;0X'0(4 K'"#0D %#G=&(
T# -="%&' 1#(=XP% =G1,&'( #0 1#P%"'Ld(#&=,>
T%D,XP% #0 %0P="#0G%0'7 %(1%&=,>>L _U_7 BM' =G1"#P%- "%(=>=%0&%
H;9< 0'): C#-%",'% 1#(=XP% %#G=&(
K'"#0D 0%D,XP% #0 (#&=,>7 %0P="#0G%0'H);*W)K4 K'"#0D (#&=,>d%0P="#0G%0' B%0%a'( =R G#-,> (Q=c
K'"#0D %#G=&( =R R"%=DQ'
J#G=&(d(M(',=0,B=>='L R#" 1,((%0D%" ",=>O,L(
+=(W( 'Q,' G#-,> (Q=c 0#' ,&Q=%P%-d(M(',=0%-
8/10/2019 STAR Results and Analysis (24Nov)
44/46
STAR Scoring: Step 1
8/10/2019 STAR Results and Analysis (24Nov)
45/46
Step 2
8/10/2019 STAR Results and Analysis (24Nov)
46/46
Rating HighlyUnsustainable Unsustainable
ModeratelyUnsustainable
MarginallySustainable
ModeratelySustainable Sustainable
HighlySustainable
Score -5 to -10 -2 to -4 -1 to 0 1 to 2 3 to 4 5 to 6 7 to 10
STAR Scoring: Step 3