Statistical evaluation of model uncertainties in Copert III, by I. Kioutsioukis & S. Tarantola (JRC, I)

Statistical evaluation of model uncertainties in Copert III,

by

I. Kioutsioukis & S. Tarantola (JRC, I)

Sensitivity analysis tests performed using COPERT III

Interpretation of the results

Objectives of the statistical analyses

Need to check robustness of emission estimates to poorly known parameters and model assumptions.Reflect our poor knowledge on input parameters by means probability distributions and apply Monte Carlo analysis to estimate probability distributions of emissions.

Representation of a Monte Carlo simulation

Objectives

Precision of emission estimates depends on the assumptions made in the definition of the various model input parameters.

Uncertainty should always accompany an estimate, as it is a measure of the quality of the estimate.

Representation of the Monte Carlo simulation

Objectives

Objective is to apply up-to-date sensitivity analysis to identify theparameters mainly responsible for uncertainty in the emissions

Help us improving the quality of emission estimates if we direct efforts to improve our knowledge of the important parameters

Estimates (and related uncertainties) can then be used 1. to adopt traffic policy measures 2. for inventory systems3. as input to air quality models

Objectives

Statistical analyses

• Description of sources of uncertainty (input):

• Description of the set up of the analyses

• Results (Figures and Tables)

- Uncertainty in traffic parameters (how to model them)

- Uncertainty in average speed

- Uncertainty in emission factors

Country-specific mileage data taken from MEET deliverable #22

All the categories of vehicles considered

FBM–INFRAS used for decomposition of fleet into sub-categories

Model uncertainty in traffic parameters

τ: steers the technology stage percentages;

τ = –1, 0 and +1 represent fleet with 'low/medium/high' amount of new technology vehicles, respectively.δ: steers the diesel share of PC and LDV;

δ = –1, 0 and +1 represent fleet with 'low/medium/high' amount of diesel vehicles, respectively.σ : steers the size (weight class) distribution of HDV;

σ = –1, 0 and +1 represent fleet with 'low/medium/high' amount of heavy-weight HDV's, respectively.

Uncertainty in traffic parameters

FBM (expensive) is only executed at selected points

τ τ

δ σ

We feed COPERT with a representative configuration of fleet breakdown at each Monte Carlo run i.

We sample a point τi, δi, σi over the square and interpolating the FBM runs we obtain the configuration of fleet breakdown f (τi, δi, σi )

Uncertainty in average speed

Currently described with rather rough statistical distributions

Exploratory analyses have shown that average speed is rather an important parameter. Perform more refined analyses…

0,000

0,020

0,040

0,060

0,080

0,100

0,120

0,140

40 60 80 100 120

velocità media [km/h]

f(x)

■ Average speed in rural road ● average speed in motorway

More reliable pdf’s using Goodness of fit tests based on driving cycles

Uncertainty in emission factors

Very low regression coefficients

1

1

2

1

i

i

i

ij N

sN

sS

)1(*)1,0(* 22 RNSRefEF jreg

)1,0(NSefEF jreg

regef

Not sufficient

Uncertainty in load factorsPdf=Normal; mean=50%, std = 10%(questionnaire - expert opinion)

Uncertainty in meteo conditions(statistical model - INFRAS)

Uncertainty in average trip length Pdf=Log-Normal; mean=12Km, std=3Km(questionnaire - expert opinion)

Variable Description Units Distribution (,)

PPC population of PC - Normal (34799160, 347992)

PLDV population of LDV - Normal (2142083, 21421)

PHDV population of HDV - Normal (1293357, 12934)

PUB population of UB - Normal (83851, 838)

P2W population of 2-wheel vehicles - Normal (2000000, 20000)

MPC Annual mileage of PC Km Normal (10059, 1006)

MLDV Annual mileage of LDV Km Normal (17706, 1771)

MHDV Annual mileage of HDV Km Normal (38741, 3874)

MUB Annual mileage of UB Km Normal (41800, 4180)

MPW Annual mileage of PW Km Normal (5000, 1000)

UPC driving share (urban) of PC % Normal (35, 10)

HPC driving share (highway) of PC % Normal (15, 2.5)

ULDV driving share (urban) of LDV % Normal (40, 12)

HLDV driving share (highway) of LDV % Normal (30, 4.5)

UHDV driving share (urban) of HDV % Normal (30, 9)

HHDV driving share (highway) of HDV % Normal (50, 7.5)

UUB driving share (urban) of UB % Normal (75, 10)

HUB driving share (highway) of UB % Normal (15, 2.25)

UPW driving share (urban) of PW % Normal (30, 9)

HPW driving share (highway) of PW % Normal (40, 6)

VU velocity profile (urban) Km/h Normal (20, 3)

VRPC velocity profile (rural) of PC Km/h Normal (65, 9.75)

VHPC velocity profile (highway) of PC Km/h Normal (100, 15)

VRLDV velocity profile (rural) of LDV Km/h Normal (60, 9)

VHLDV velocity profile (highway) of LDV Km/h Normal (90, 13.5)

VRHDV velocity profile (rural) of HDV Km/h Normal (50, 7.5)

VHHDV velocity profile (highway) of HDV Km/h Normal (80, 12)

VRUB velocity profile (rural) of UB Km/h Normal (50, 7.5)

VHUB velocity profile (highway) of UB Km/h Normal (85, 12.75)

VRPW velocity profile (rural) of PW Km/h Normal (65, 9.75)

VHPW velocity profile (highway) of PW Km/h Normal (100, 15)

Ltrip Average trip length Km Log-Normal (12, 3)

LP load factor % Normal (50, 10)

slope slope category - Normal (0, 1)

A lowest minimum temperature C Normal (3.4, 0.35)

H highest minimum - lowest minimum temperature C Normal (14.9, 0.51)

D highest maximum - (A+H) temperature C Normal (13, 0.48) traffic parameter - Uniform (-1, 1) traffic parameter - Uniform (-1, 1) traffic parameter - Uniform (-1, 1)

eEF amplitude Emission Factor - Normal (0, 1)

Variable Description Units Distribution (,)

PPC population of PC - Normal (34799160, 347992)

PLDV population of LDV - Normal (2142083, 21421)

PHDV population of HDV - Normal (1293357, 12934)

PUB population of UB - Normal (83851, 838)

P2W population of 2-wheel vehicles - Normal (2000000, 20000)

MPC Annual mileage of PC Km Normal (10059, 1006)

MLDV Annual mileage of LDV Km Normal (17706, 1771)

MHDV Annual mileage of HDV Km Normal (38741, 3874)

MUB Annual mileage of UB Km Normal (41800, 4180)

MPW Annual mileage of PW Km Normal (5000, 1000)

UPC driving share (urban) of PC % Normal (35, 10)

HPC driving share (highway) of PC % Normal (15, 2.5)

ULDV driving share (urban) of LDV % Normal (40, 12)

HLDV driving share (highway) of LDV % Normal (30, 4.5)

UHDV driving share (urban) of HDV % Normal (30, 9)

HHDV driving share (highway) of HDV % Normal (50, 7.5)

UUB driving share (urban) of UB % Normal (75, 10)

HUB driving share (highway) of UB % Normal (15, 2.25)

UPW driving share (urban) of PW % Normal (30, 9)

HPW driving share (highway) of PW % Normal (40, 6)

VU velocity profile (urban) Km/h Normal (20, 3)

VRPC velocity profile (rural) of PC Km/h Normal (65, 9.75)

VHPC velocity profile (highway) of PC Km/h Normal (100, 15)

VRLDV velocity profile (rural) of LDV Km/h Normal (60, 9)

VHLDV velocity profile (highway) of LDV Km/h Normal (90, 13.5)

VRHDV velocity profile (rural) of HDV Km/h Normal (50, 7.5)

VHHDV velocity profile (highway) of HDV Km/h Normal (80, 12)

VRUB velocity profile (rural) of UB Km/h Normal (50, 7.5)

VHUB velocity profile (highway) of UB Km/h Normal (85, 12.75)

VRPW velocity profile (rural) of PW Km/h Normal (65, 9.75)

VHPW velocity profile (highway) of PW Km/h Normal (100, 15)

Ltrip Average trip length Km Log-Normal (12, 3)

LP load factor % Normal (50, 10)

slope slope category - Normal (0, 1)

A lowest minimum temperature C Normal (3.4, 0.35)

H highest minimum - lowest minimum temperature C Normal (14.9, 0.51)

D highest maximum - (A+H) temperature C Normal (13, 0.48) traffic parameter - Uniform (-1, 1) traffic parameter - Uniform (-1, 1) traffic parameter - Uniform (-1, 1)

eEF amplitude Emission Factor - Normal (0, 1)

first stage: screening analyses (Morris and Standardised Regression Coefficients (SRC)) to identify the non-influential input parameters.

Results: total emissions in Italy for years 2000 and 2010

40 parameters 15 parameters

Identified 25 parameters that do not influence the variability of the emission estimates (eg meteo variables)

0 0.5 1 1.5 2 2.5 3

x 105

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2x 10

5

MU

SIG

MA

HDV

VU

LDVPCLP

ltrip

AHdUBVH126

VR126

VR34VH34

MPC

MUBMLDVMHDVMPW

UPC

UHDVULDVUUBHPCHLDVHHDVHUBUPWHPW

delta

tau

sigma

eEF

PW

VOC

Results of the screening technique – yr 2000

Region of the non-

influential parameters

0 1 2 3 4 5 6

x 104

0

2000

4000

6000

8000

10000

12000

14000

16000

MU

SIG

MA

HDV

VU

LDVPC

LP

ltrip

AHdUBVH126

VR126

VR34VH34

MPC

MUBMLDV MHDVMPW

UPC

UHDV

ULDVUUBHPCHLDV

HHDV

HUBUPWHPW

delta

tau

sigma

eEF

PW

VOC

Results of the screening technique – yr 2010

Region of the non-

influential parameters

VOC Emissions - Italy

0

100,000

200,000

300,000

400,000

500,000

600,000

700,00019

81

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

2000

2005

2010

2015

2020

[t]

Passenger Cars Light Duty Vehicles Heavy Duty Vehicles Buses

LAT dataUncertainty analysis on 15 parameters

Uncertainty analysis - 2010

VOC

0

20

40

60

80

100

120

140

160

180

8.1E

+04

1.0E

+05

1.3E

+05

1.5E

+05

1.7E

+05

1.9E

+05

2.1E

+05

2.4E

+05

2.6E

+05

2.8E

+05

3.0E

+05

3.2E

+05

3.5E

+05

3.7E

+05

3.9E

+05

Annual Emissions (tonnes)

Fre

qu

ency over-estimation of VOC:

probably l-trip is overestimated

LAT value

NOx Emissions - Italy

0

100,000

200,000

300,000

400,000

500,000

600,000

700,000

800,000

900,000

1,000,000

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

2000

2005

2010

2015

2020

[t]

Passenger Cars Light Duty Vehicles Heavy Duty Vehicles Buses

LAT data

NOX

0

20

40

60

80

100

120

140

160

180

200

2.1E

+05

2.4E

+05

2.6E

+05

2.9E

+05

3.1E

+05

3.4E

+05

3.7E

+05

3.9E

+05

4.2E

+05

4.4E

+05

4.7E

+05

5.0E

+05

5.2E

+05

5.5E

+05

5.8E

+05

Annual Emissions (tonnes)

Fre

qu

ency

Uncertainty analysis - 2010

LAT value

PM Emissions - Italy

0

5,000

10,000

15,000

20,000

25,000

30,000

35,000

40,000

45,000

50,00019

81

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

2000

2005

2010

2015

2020

[t]

Passenger Cars Light Duty Vehicles Heavy Duty Vehicles Buses

LAT data

PM

0

50

100

150

200

250

1.1E

+04

1.3E

+04

1.6E

+04

1.9E

+04

2.1E

+04

2.4E

+04

2.6E

+04

2.9E

+04

3.1E

+04

3.4E

+04

3.6E

+04

3.9E

+04

4.1E

+04

4.4E

+04

4.6E

+04

Annual Emissions (tonnes)

Fre

qu

ency

Uncertainty analysis - 2010

LAT value

CO2 Emissions - Italy

0

20,000,000

40,000,000

60,000,000

80,000,000

100,000,000

120,000,000

140,000,000

160,000,000

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

2000

2005

2010

2015

2020

[t]

Passenger Cars Light Duty Vehicles Heavy Duty Vehicles Buses

LAT data

CO2

0

20

40

60

80

100

120

140

160

180

200

9.1E

+07

9.8E

+07

1.1E

+08

1.1E

+08

1.2E

+08

1.2E

+08

1.3E

+08

1.4E

+08

1.4E

+08

1.5E

+08

1.6E

+08

1.6E

+08

1.7E

+08

1.8E

+08

1.8E

+08

Annual Emissions (tonnes)

Fre

qu

ency

Uncertainty analysis - 2010

LAT value

METHOD Morris FAST

No RUNS 350 5499

YEAR 2000 2010 2000 2010

Mean VC (%) Mean VC (%) Mean VC (%) Mean VC (%)

VOC 639 21 213 13 641 31 213 22

NOX 740 11 377 12 740 13 379 15

PM 39 22 21 21 40 25 21 26

CO2 117,621 7 136,406 6 113,852 9 133,350 9

Summary of Uncertainty Analysis

second phase: quantitative sensitivity analysis technique (extended-FAST) to apportion variance of emission estimates back to input parameters.

VOC

delta

ltrip

eEF

VU

MPC

1-SUM

2000 2010

68% of VOC variance explained by the top-three parameters

increase of ltrip and decrease of VU becomes important in 2010

PM

1-SUM

VU

MHDV

ltrip

eEF

delta

2000 2010

Uncertainty in diesel share of PC and LDV is important

The differences with the run conducted for 2000 are in the vehicle Populations, fleet breakdown and in the use of new fuel.

NOX

1-SUM

VU

ltrip

MPC

MHDV

delta

eEF

2000 2010

important variables are eEF and MPC becomes important

CO2

1-SUM

MHDV

UPC

VUltrip

MPC

2000 2010

CO2 emissions are mostly influenced by MPC (SMPC=37%)

and ltrip. Situation remains unchanged in 2010 VU becomes important in 2010

Output variability for each pollutant IS described by three most influential input parameters.ltrip, eEF, VU and are common to almost all the pollutants.

Technological and fuel improvements will result in reduced emissions for VOC, PM and NOX (2000 2010).

Interpretation and conclusions

Quality of emission estimates can be enhanced if we direct efforts to improve our knowledge on average trip length, emission factors, diesel share between PC and LDV and the annual mileage of passenger cars

Importance of emission factors , with the current statistical model, increases 2000 2010.

Uncertainty in emission factors should be explained by a set of kinetic parameters (not only average speeds).

Acknowledge uncertainty in the emission factorsat the level of driving cycles

When driving cycles are combined to build TS,it is straightforward to calculate uncertainty bounds for TS.