GMES Marine Service MY OCEAN Cross Cutting Cal/Val Activities & Product Quality Working Group Scientific Assessment of External Products in MyOcean Second.

GMESMarineService

MY OCEAN

Cross Cutting Cal/Val Activities & Product Quality

Working GroupScientific Assessment of

External Products in MyOcean

Second annual meeting

F. Hernandez, chair PQCV-WG, QuARG

Marine Core Service

MyOcean : Second Annual Meeting / 27-28 April 2011 / Roma

Cal/Val activities for the second MyO year

Cal/Val teams in Production Centres (X.5) strongly involved in:

– Test and implementation of Cal/Val tools for V1

– Design Calibration and Validation strategies and plans (ScCP, ScVP)

– Perform scientific calibration, routine scientific validation, and report (ScCR, ScVR, ScVR-Rea) for reviews

Prepare ScCP, ScVP…

….. V1S1 devlpt.

PC Acceptancereview of ScCP, ScVP,

ScCR

June 2010 End September 2010

VAlid. Read. Rev.Start V1 S1 operation

15/12/2010

Test op. validation

V1S1 scientific assessment ….. V1S1 pre-operated ….. V1S1 in operation

Operate routinely PQ validation: reg. reporting

29/4/2011

P. Read. Rev.Review V1 S1 ScVR

Calibration

….. V1S2 devlpt. / REA V1S2 & REA scientific assessment

PC Acceptancereview of ScCP, ScVP, ScCR, ScVR-Rea

March 2011 June 2011 Sept. 2011

VARRStart V1S2 operation

PRRV1S2 ScVR

….. V1S2 in operation

Validation

Marine Core Service


Cal/Val activities for the second MyO year

Product Quality addressed at the project level:

– Reviews, where ScCR, ScVR are examined

– Link with users: feedbacks from User Forum (Stockholm, April 2011)

– Designing MyOcean-2 Product Quality activities: focusing on cross-cutting integrated actions and methods, regular monitoring, link between Production Centres and Users

Marine Core Service


Organising Product Quality for V1:plans and reporting for reviews

PC Acceptance Stream1 V1, Roma, September 2010

VARR Stream1 V1, Brussels, December 2010

PC Acceptance Stream2 V1, Copenhagen, March 2011

User Forum, Stockholm, April 2011

PRR Stream1 V1, Roma, April 2011

VARR Stream2 V1, June 2011

PRR Stream2 V1, September 2011

Prod. Centre©

ScCP/ScVP ScCR ScVR-Rea ScVR (PRR)

GLO x Stream 1 5 docs 1 doc + 2 docs

ARC x Stream 1 1 doc 1 doc

BAL x Stream 1

NWS x Stream 2 2 docs

IBI x Stream 2

MED x Stream 1 1 doc

BLS x Stream 2 1 doc

SL x Stream 1 1 doc

OC x Stream 1/2 1 doc 1 doc

SST no Stream 1 in 2 OSTIA doc 1 doc

SIW no for Wind Stream 1 7 docs no Wind

INS x NO (in prep?) 1 doc

Marine Core Service


Organising Product Quality for V1:plans and reporting for reviews

Overall good level of information in ScCR and ScVR

Permanent improvements of reporting, from stream1 to stream2:– Upgrading and extending Cal/Val methodologies

– Analysing benefit of V1 compared to V0

– Analysing added values of regional products

– Starting to address « estimated accuracy » values compared to user requirements (if exists)

From V1 to V2, plans to upgrade cal/val strategies and associated documentation with more consistency among TACs and MFCs

Plans to use existing scientific cal/val to provide usefull information for users

Marine Core Service


Flavour of Product Quality assessment, from reporting

Different difficulties, management, history, heritage, and many positive aspects:

– SST TAC reporting: not all docs, but validation strategies already described in existing operational activities. ScVR, really complete, and usefull for users (estimated confidence level per products)

– INS TAC slightly different from other PC: provide Real Time Quality Control, instead of Cal/Val phasing, and in a continous framework

– BAL MFC: a complete reporting, plus a long annex (+550p)

– BLS MFC: enormous lack of observations, but alternative use of climatology

– ARC MFC: A strong dedication to Sea Ice scientific assessment.

– …..

Marine Core Service


Flavour of Product Quality assessment, from reporting:V1 vs V0

From IBI ScCR

Marine Core Service


Flavour of Product Quality assessment, from reporting:V1 vs V0

From NWS ScCR

Marine Core Service


Flavour of Product Quality assessment, from reporting: proposing new approach: OC TAC and MED MFC validating Chl-a

From MED ScVR

Marine Core Service


Flavour of Product Quality assessment, from reporting:organize and standardize cal/val (metrics) definitions

ScCP, ScVP provide description of metrics Next phase: harmonisation

Marine Core Service


Flavour of Product Quality assessment, from reporting: organize product quality synthesis

From ARC ScVR, ScVR-REA

Marine Core Service


Tranverse Cal/Val at MYO level

Organize more homogeneous Cal/Val procedures btw PC/PU:

– X.5 experts asked to review ScCR: « share good ideas »

– Neighboring dynamics:

• WP6 / WP7 working already together add WP8

• WP8 / WP9 and WP4 / WP9 in contact

• WP10 / WP9 starting

• WP4 / WP8 « natural contacts »

• Existing inside each TACs

– Indirect benefit from Lisbon 1st PQCV-WG meeting: lot’s of similarities in ScCR and ScVR metrics

Second PQCV-WG meeting yesterday to strengthen transverse Cal/Val

Marine Core Service


PQCV-WG 2nd meeting outcomes

Improvements of ScCP, ScVP, technical definitions of metrics

– Upgrade definitions before V2, start with moorings

Marine Core Service


Upgrade definition: adopt common names of moorings…..

Marine Core Service





Product Quality expected accuracy level (static numbers to be associated to products):

– Starting from list of V1 products (eg EPST excel sheet)

– End of June 2011: every PC provide numbers to PQCV-WG

– End of August 2011: After PQCV-WG consistency check, numbers provided to WP17

Marine Core Service


Last year: PQCV-WG proposition for link with Service Desk

information V1 V2

Static:

“precision” level given as meta-data

Dynamic

“flags”

Error bars

Extreme values

? YES

<e>

no YES

ENSTaf BQMMPP

Marine Core Service









QUID: Quality Information Document for users:

– Baseline: provide ScCR, ScVR

– Instead provide dedicated documents from ScCR, ScVR

– Template discussed at PQCV-WG meeting

– Late September 2011 (phased with end of Stream2 Validation Period) QUID draft delivered by each PC

– Mid-October 2011: after a 2 week PQCV review, QUID provided to PMO, WP17

Marine Core Service









QUID: Quality Information Document for users:

– Baseline: provide ScCR, ScVR

– Instead provide dedicated documents from ScCR, ScVR

– Template discussed at PQCV-WG meeting

– Late September 2011 (phased with end of Stream2 Validation Period) QUID draft delivered by each PC

– Mid-October 2011: after a 2 week PQCV review, QUID provided to PMO, WP17

Transverse Cal/Val activities: internal website:

– Start with selected metrics for SL, SST, Sea-Ice, Chl-a, Water masses

– First integration aspects (tbd) with WP2 in September 2011

Marine Core Service


Marine Core Service


Cal/Val activities in MyOcean, summary

All PC/PU involved

Project rules (document, reviews) adopted, with obvious benefits

Non expected yet, but User request for quality (strongly reminded during the User Forum) can start to fullfiled due to the Cal/Val work achieved

The Cal/Val transverse strategy is accepted, even if additional work is requested

This strategy and difficulties encountered have been a direct input for MYO-2 Cal/Val definition and organisation

GMES Marine Service MY OCEAN Cross Cutting Cal/Val Activities & Product Quality Working Group Scientific Assessment of External Products in MyOcean Second.

Documents