CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006 XFDU & SAFE - ESA return from experience ESA return from experience & f Stéphane Mbaye [email protected].

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

ESA return from experienceESA return from experience

&

f

Stéphane [email protected]

Mathias [email protected]



Context

Support to XFDU Specifications

Historical Archive Rationalization (HARM)



Support to XFDU Specifications

(Among other requirements)• Edit/Review « XFDU Construction Rules » releases• Edit/Review « XFDU Green Book » releases• Critical analysis of the standard regarding ESA needs• Provide an early implementation of the standard• Produce sample packages• Perform interoperability testing with other agencies



Historical Archive Rationalization (HARM)

• ESA Project: Historical Archive Rationalization and Management (HARM)

• Consortium: ACS, GAEL, Werum, DLR• (Among other requirements)

• Develop a Standard Archive Format for Europe (SAFE)• Support all ESA archived products (ESA and third party missions)• Conform to modern standards including OAIS RM and W3C.

• Develop an API for handling SAFE products• Java and C++ API• Read/Write SAFE Products• Convert native formats to SAFE formats



Mutual Interests of the activities

• CCSDS/MOIMS/IPR XFDU• HARM may provide a testbed for improving « XFDU

Construction Rules »• SAFE I/O API or any of its component may provide an alternate

implementation of XFDU standard as required by the CCSDS• SAFE test data may provide IPR working group with meaningful

sample data

• HARM Project• SAFE may benefit from studies and international consensus

conducted by CCSDS agencies• SAFE may conform to XFDU and, therefore, tackle with the

CCSDS recommendation: « Each Agency is, however, strongly encouraged to direct planning for its new standards and implementations towards the later version of the Recommendation »



Decision: Joint Activities

• SAFE shall follow the « XFDU Construction Rules »• SAFE sample and test data should be proposed to the

IPR as XFDU sample data• SAFE I/O API shall be split in two separate layers:

• An independent XFDU I/O API (only those features required by SAFE will, however, be implemented)

• A top level API implementing SAFE specific features and converters

• « SAFE Control Book Volume 2 – Recommendation for Specialisations » shall be written to help the editing of « XFDU Green Book »



SAFE Core Specifications



Information Model



Logical Model



Physical Model



Manifest - Internal Constraints



Manifest - Identification of Product Types

<xfdu:XFDU xmlns:xfdu="http://www.ccsds.org/xfdu/2004"

xmlns:safe="http://www.esa.int/safe/1.0"

xmlns:xs="http://www.w3.org/2001/XMLSchema"

xmlns:envisat="http://www.esa.int/safe/1.0/envisat"

xmlns:xlink="http://www.w3.org/1999/xlink"

xmlns:gml="http://www.opengis.net/gml"

version="esa/safe/1.0/envisat/gomos/nl/level-0">

[...]

</xfdu:XFDU>



Manifest - Reference Informations

« SAFE requires Representation Information of each Data Object »



Manifest – Multiple Rep. Info’s

« Representation Information may be composed of several components »



Manifest – Binary Metadata

« XFDU Recommends to refer to Binary Metadata Components through a Data Object »

« XFDU requires that all Data Objects should be referenced from the Information Package Map »

« The second version of XFDU may simplify the mechanism for describing object to object associations »



SAFE Specializations



Specialization - Definition



Abstract specializations



Abstract specializations - Example



Redefinitions and Imports



Example of SAFE Product’s Manifest

• Refer to « SAFE Control Book Volume 2 – Recommentation for Specialisations », Appendix A



ESA/XFDU I/O Java/C++ API



Overview

JNI Interface



Use cases

• Create of an XFDU package

• Open an XFDU package

• Remove an XFDU package

• Move an XFDU package considering referenced objects

• Validate an XFDU package including the object contents

• Verify fixty i.e. Components checksums

• Add/Remove/Browse Content Units of the Information Package Map

• Add/Remove/Retrieve Metadata Objects

• Add/Remove/Retrieve Data Objects

• Apply Transformation Objects



Java Packages

• esa.xfdu General XFDU packages management.

• esa.xfdu.map XFDU Information Package Map Section

• esa.xfdu.object Classes handling all XFDU object types.

• esa.xfdu.object.metadata XFDU meatadata object types.

• esa.xfdu.object.data Classes handling XFDU data object types.

• esa.xfdu.object.behavior Classes handling XFDU behavior object types.

• esa.xfdu.object.behavior.impl Behavior Implementations



Main Java Interfaces

• esa.xfdu.object.Object• esa.xfdu.object.metadata.MetadataObject

• esa.xfdu.object.data.DataObject

• esa.xfdu.object.behavior.BehaviorObject

• esa.xfdu.object.behavior.Mechanism• esa.xfdu.object.behavior.Transformer

« The second version of XFDU may gather the means for referencing Metadata/Data/Behavior objects in a common object list. »



C++ Wrapper



Current Version

xfdu-1-0-rc-10(April 28, 2006)



Code Statistics



Further information

http://www.gael.fr/xfdu



Connected Application

• CNES has developped a GUI on top the XFDU I/O Java API: the « XFDU View »

• C.f. http://www.gael.fr/xfdu-view



ESA/SAFE I/O Java/C++ API



Overview



Use cases

• Create of a SAFE product

• Open a SAFE product

• Remove a SAFE product

• Move a SAFE product considering referenced objects

• Validate a SAFE product including the object contents

• Verify fixty i.e. Components checksums

• Add/Remove/Browse Content Units of the Information Package Map

• Add/Remove/Retrieve SAFE Specialized Metadata Objects

• Add/Remove/Retrieve SAFE Data Objects

• Browse XML and Binary Data Object Contents

• Accelerate Content access using SAFE Indexes



Java Classes

esa.safe.object.metadata.AcquisitionPeriod

esa.safe.object.metadata.CloudVoteNotationSystem

esa.safe.object.metadata.Container

esa.safe.object.metadata.CorruptedCause

esa.safe.object.metadata.CorruptedElements

esa.safe.object.metadata.Facility

esa.safe.object.metadata.Frame

esa.safe.object.metadata.FrameSet

esa.safe.object.metadata.FrameTile

esa.safe.object.metadata.GridReference

esa.safe.object.metadata.Instrument

esa.safe.object.metadata.Line

esa.safe.object.metadata.Location

esa.safe.object.metadata.MissingElementsesa.safe.object.metadata.OrbitNumberesa.safe.object.metadata.OrbitReferenceesa.safe.object.metadata.Platform esa.safe.object.metadata.Point esa.safe.object.metadata.Processing esa.safe.object.metadata.QualityInformation esa.safe.object.metadata.Resource esa.safe.object.metadata.Softwareesa.safe.object.metadata.Time

esa.safe.Safeesa.safe.SchemaValidator



C++ Wrapper



Current Version

safe-1-0-rc-13(June 10, 2006)



Code Statistics



Further information

ESA Official Web:

http://earth.esa.int/SAFE

GAEL’s Project Web:

http://www.gael.fr/safe



Conclusions

• SAFE Specifications verified that XFDU is suitable for some amount of EO data sets (59)

• An alternate XFDU implementation is available (C++/Java) and interoperates with NASA API

• Up to 59 EO sample packages have been generated• SAFE Volume 2 is ready to be conveyed to the XFDU

Green Book• An XFDU v2 may improve the v1, at least, by:

• Simplifying the association mechanism between Objects• Generalizing Metadata/Data/Behavior Object references

• Future of XFDU and SAFE within ESA: c.f. ESA Agency Report

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006 XFDU & SAFE - ESA return from experience ESA return from experience & f Stéphane Mbaye [email protected].

Documents

xfdu safe esa return

safe formats

ccsds agencies safe

experience esa return

ccsds moims springs

ccsds safe test data

xfdu sample data safe

europe safe support