2 WP9/JRA3: ESM Evaluation: Developing an Infrastructure Participants: WP Leader: George Tselioudis, Academy of Athens Co-leader: Michael Schulz, IPSL.

2

WP9/JRA3: ESM Evaluation: Developing an Infrastructure

Participants: WP Leader: George Tselioudis, Academy of Athens

Co-leader: Michael Schulz, IPSLWP Partners:

Veronika Eyring, DLRPierre Friedlingstein – U. of Exeter

Stefan Bakan, MPI-MRalph Doscher, SMHI

3

OBJECTIVECreate an interdisciplinary infrastructure that will facilitate the scientific evaluation of complex ESMs

TASKS

Task 1. Survey of methods and problems in ESM evaluation workTask 2. Creation of support structure to facilitate model evaluation testsTask 3. Provision of cataloguing and access services to observational datasets and evaluation toolsTask 4. Pilot study on the implementation of evaluation toolkits on ESM output

4

FIRST YEAR PROGRESS

T.1 Survey of methods and problems in ESM evaluation workQuestionnaires were sent to all major European ESM groups. ESM evaluation areas have been divided among the partners as follows:

AA: Clouds, radiation and precipitationCNRS-IPSL: Aerosols and paleoclimateMPI-MET: Surface fluxesDLR: Atmospheric chemistryU. Exeter: Carbon cycle

87% of questionnaires have been received

CMCC CNRM COSMOS Hadley IPSL

Aerosols X X X X

Carbon Cycle X X X X X

Chemistry X X X XClouds X X X X

Precipitation X X X X

Radiation X X X X

Surface Fluxes X X X X

Evaluation Dataset TypesSatellite Ground based Air-borne (aircrafts, ozone-sondes etc.)(Re-)analysis Sat-Gr Sat-Gr-Air Sat-Gr-Analysis

Aerosol 8 58 0 0 0 0 0Carbon Cycle 10 39 0 2 0 0 0

Chemistry 43 6 8 4 2 1 0Clouds 35 9 3 0 3 0 0

Precipitation 2 2 0 3 8 0 0Radiation 37 6 0 1 0 0 0

Surface Fluxes 11 11 0 8 3 0 2

Some preliminary remarks

Aerosol and carbon cycle properties are evaluated using primarily ground based observations

Cloud, radiation, and chemistry properties are evaluated using primarily satellite observations

Precipitation and surface flux properties are evaluated using a mix of the two dataset types

Evaluation Datasets

Aerosols Carbon Cycle Chemistry Clouds Precipitation Radiation Surface Fluxes

EMEP 11 GLODAP 8 MIPAS 12 ISCCP-D 9 GPCP 4 ERBE 13 NCEP-NCAR 6

IMPROVE 11 NOAA ESRL GMD 4 UARS 8 MODIS 7 CMAP 3 CERES 11 Southampton climatology 5

EANET 9 TRANSCOM 4 HALOE 4 CALIPSO 6 NCEP-NCAR 1 ISCCP-FD 4 SMD94 4

AERONET 9 4 TOMS 3 MISR 3 GPCC 1 PARASOL 3 ISCCP-FD 4

GAW 4 EUROFLUX 3 MOZAIC 3 PARASOL 3 CRU 1 GEBA 2 ERBE 2

SKYNET 4 AMERIFLUX 3 SCIAMACHY 3 TOGA-COARE 3 HOAPS 1 BSRN 2 GEBA 2

AEROCE 3 NCEP/DOE AMIP-II 2 ODIN 3 AMMA 3 TRMM 1 MODIS 2 CMAP 2

MISR 3 ISLSCP 2 ILAS 3 SIRTA 3 NCEP/DOE AMIP-II 1 1 ERA-40 2

MODIS 3 2 MOPITT 2 BOMEX 3 ERA-Interim 1 NCEP-NCAR 1 HOAPS 2

3 CARBOOCEAN 2 SHADOZ 2 ARM 3 AMMA 1 1 OAFLUX 2

CASTNET 2 2 Logan 2 2 ISCCP-D 1 BSRN 2

U of Miami 1 2 TES 2 ERBE 1 SIRTA 1 GPCP 1

POLDER 1 MODIS 2 TRMM 2 NOAA Interpolated OLR 1 TOGA-COARE 1 MODIS 1

CALIPSO 1 1 ERA-40 2 AIRS 1 ARM 1

SEAREX 1 1 UKMO analysis 2 TES 1

VGPM 1 NIWA 1 SCIAMACHY 1

EMDI 1 WOUDC 1

1 NOAA ESRL GMD 1

IGBP 1 EMEP 1

GEOLAND 1 CASTNET 1

1 IMPROVE 1

1 NDEP 1

1 EANET 1

AVHRR/NOAA 1 AIRS 1

GPCP 1

Fluxnet

SMD94 (da Silva)

Takahashi DB

Amsterdam Island & Cape Grim

NOAA Interpolated OLR

Zinke et al. CloudSat

Olson et al.

SeaWifs

ClimPP

Norby et al.

Sabine et al.,2004

Carr et al. 2006

Kettle et al. 1999

Evaluation MethodsStatistical analysis Process simulation Satellite simulator

Aerosols 100.0% 0.0% 0.0%Carbon Cycle 50.0% 50.0% 0.0%Chemistry 73.3% 26.7% 0.0%Clouds 7.7% 7.7% 84.6%Precipitation 100.0% 0.0% 0.0%Radiation 50.0% 25.0% 25.0%Surface Fluxes 75.0% 0.0% 25.0%

Frequently reported problems

Diversity of observational data formats reported as the main problem

HDF format of satellite data not compatible with model output analysis tools

NetCDF (CMOR comliant if possible) preferred data format

9

FUTURE STEPS

Task 2. Creation of support structure to facilitate model evaluation tests – Put emphasis on process-based evaluation

Task 3. Provision of cataloguing and access services to observational datasets and evaluation tools – Include peripheral datasets and tools not commonly used by ESMs

Task 4. Pilot study on the implementation of evaluation toolkits on ESM output

10

ESM Evaluation Portal

26th, 27th & 28th May 2010

IS-ENES General Assembly Barcelona, Spain

11

Aerosols CMCC COSMOS Hadley IPSL EC-Earth

Carbon Cyc CMCC CNRM COSMOS Hadley IPSL

Chemistry CMCC CNRM COSMOS Hadley IPSL EC-Earth

Cloud CMCC CNRM Hadley

Ocean CMCC

Precipitation CMCC CNRM

Radiation CMCC CNRM

Sfc Fluxes CMCC

A Framework for Process-Oriented Evaluation of Earth System Models

V. EYRING, P. Friedlingstein, O. Boucher, W. J. Collins, C. JONES, N. de Noblet-Ducoudre, H. DRANGE, T. Fichefet, C. Heinze, W. Knorr, J.-F. Lamarque, M. G. Lawrence, I. C. Prentice, M. Scholze, M. Schulz, J. TEIXEIRA, G. Tselioudis, M. Vancoppenolle, AND P. F. J. van Velthoven  AFFILATIONS: EYRING – Deutsches Zentrum für Luft- und Raumfahrt (DLR), Institut für Physik der Atmosphäre, Oberpfaffenhofen, Germany; Friedlingstein – University of Exeter, Exeter, U.K.; BOUCHER – Met Office Hadley Centre, Exeter, U.K.; COLLINS – Met Office Hadley Centre, Exeter, U.K.; JONES – Swedish Meteorological and Hydrological Institute (SMHI), Norrköping, Sweden; de Noblet-Ducoudre - Commissariat à l'Énergie Atomique (CEA), Gif-sur-Yvette, France; DRANGE – University of Bergen, Norway; Fichefet - Université catholique de Louvain, Louvain-la-Neuve, Belgium; Heinze - University of Bergen, Geophysical Institute and Bjerknes Centre for Climate Researc, Norway; Knorr - University of Bristol, U.K.; Lamarque - NCAR, Boulder, USA; Lawrence - Max Planck Institut für Chemie, Mainz, Germany; Prentice - University of Bristol, U.K.; Scholze - University of Bristol, U.K.; Schulz - Commissariat à l'Énergie Atomique (CEA), Gif-sur-Yvette, France; TEIXERA – NASA Jet Propulsion Laboratory (JPL), USA; Tselioudis – University of Athens, Greece; M. Vancoppenolle - Université catholique de Louvain, Louvain-la-Neuve, Belgium; van Velthoven - Koninklijk Nederlands Meteorologisch Instituut (KNMI), Netherlands. CORRESPONDING AUTHOR: Veronika Eyring, Deutsches Zentrum für Luft- und Raumfahrt (DLR), Institut für Physik der Atmosphäre, Oberpfaffenhofen, Germany, Phone: +49-8153-28-2533, E-mail: [email protected]. Capsule. Evaluating ESMs with the presented framework will increase our confidence in climate projections.

J. Climate Paper in Preparation

13

ESM Evaluation Portal

14

ESM Evaluation Portal

15

ESM Evaluation Portal

16

ESM Evaluation Portal

17

ESM Evaluation Portal

18

ESM Evaluation Portal

19

ESM Evaluation Portal

20

ESM Evaluation Portal

21

ESM Evaluation Portal

22

ESM Evaluation Portal

23

ESM Evaluation Portal

Aerosols CMCC COSMOS Hadley IPSL EC-Earth

Carbon Cyc CMCC CNRM COSMOS Hadley IPSL

Chemistry CMCC CNRM COSMOS Hadley EC-Earth

Cloud CMCC CNRM

Ocean CMCC

Precipitation CMCC CNRM

Radiation CMCC CNRM

Sfc Fluxes CMCC