WGISS 24 October 2007 Slide 1 Interoperable Catalogues from HMA - FedEO Slides assembled from various sources, with thanks to the contributers to HMA

Slide 1WGISS 24 October 2007

Interoperable Catalogues fromInteroperable Catalogues fromHMA - FedEOHMA - FedEO

Slides assembled from various sources,with thanks to the contributers to HMA / FedEO

A precursor for GMES Data Access and Integration and an overall contribution to GEOSS

Jolyon Martin, ESA


Enterprise View: Context within GMESEnterprise View: Context within GMES

Service Access Layer

Comm. PublicPublic Public Public ScienceScience Comm.

Access

Non-Europeanagenciesground

segments

GEOSS Access

ES

A

Satellite/ Missions

otherEuropeanMissionground

segment

Na t

EO Data Access Integration Layer

EU

ME

TS

AT

Data integration

GMES Services

ESAMissions Ground

Segment

In-situStatic Geo-spatial


Information Viewpoint:Information Viewpoint:OverviewOverview

Information Viewpoint main components

• ISO19115 Collection Metadata

• ISO 19119 Service Metadata

• GML Schema for Earth Observation Products


Information Viewpoint:Information Viewpoint:OGC 06-080 Earth Observation ProductsOGC 06-080 Earth Observation Products

A base schema (eop) with specializations for Optical Sensors (opt), Synthetic Aperture Radar (sar), Atmospheric Instruments (atm) and mission specific (e.g. phr)

GML payload aligned with tag names from 19115-part II draft

Based on GML observation

Currently available as best practice paper on OGC web site

sar.xsd opt.xsd atm.xsd

eop.xsd

smXML - gmd (ISO 19139)

Catalogue metadata for mission type

Catalogue metadata for EO products

gml

Schema/namespace for radar missions

Schema/namespaces for optical missions

Schema/namespaces for

atmospheric missions

Generic and

geometric catalogue metadata

phr.xsdxyz.xsdCatalogue metadata for specific mission


GML observationGML observation

Property Description within Earth Observation context

Content

metadaProperty Describe general properties such as the data identifier, the downlink and archiving information

eop:EarthObservationMetadata

validTime Describe the acquisition duration, i.e. the start and end date of acquisition

gml:TimePeriod/gml:beginPositiongml:TimePeriod/gml:endPosition

using Describe the Platform/Instrument/Sensor used for the acquisition and the acquisition parameters (i.e. pointing angles, etc.)

eop:EarthObservationEquipment

target Describe the observed area on ground i.e. the footprint of acquisition

eop:Footprint

resultOf Describe the Earth Observation result composed of the browse, mask and product description

eop:EarthObservationResult


Information ViewpointInformation Viewpoint

GML Viewers EO Product GML Prospects

• Radar Altimetery specialisation currently being added CNES

• Meteo specialisation proposed by Eumetsat

• Use of the metadata as part of product format within GMLJP2 being explored


Service Viewpoint: OverviewService Viewpoint: Overview

Overall Goal: a standardized, harmonized set of interfaces that support a service oriented view of the heterogeneous mission ground segments, which can be implemented in a cost-effective manner; enabling the construction of compound / value added services from basic interfaces.

• Discovery (collection, services)

• Catalogue (earth observations)

[ Order (archive, subscription, on-line access) ]

[ Programming (order for acquisitions, mission planning feasibility) ]

[ Data Access (wcs, wms) ]

[ User Management, Service Orchestration, Monitoring & Control ...]


ISO 19119

Interface Standards OverviewInterface Standards Overview

19139

GML Appl Schema

OGC CAT

ISO extensionpackage

(CIM)

Order Programming

OGC SPSISO 19115

part2

ISO 19115

Discovery

Catalogue

ebRIM profile

EO extensionpackage(ebRIM)

OGC WMS

OGC WCS


Service Viewpoint: Service Viewpoint: 06-131 Catalogue06-131 Catalogue

ebRIM Application ProfileebRIM Application Profile ebRIM Model

CSW (Catalogue Service/Web) CSW (Catalogue Service/Web) HTTP Protocol

OGC Catalogue Services 2.0OGC Catalogue Services 2.0 Catalogue Interface (Abstract)

EO Products Extension PackageEO Products Extension Package Mapping

• Discussion paper at the OGC: (06-131r1_EO_Products_Extension_Package_for_ebRIM_Profile_of_CSW_2.0.doc)

• Defines a unique way to map the EO metadata type (EO GML) onto the ebRIM model.

• Necessary for EO ebRIM Catalogue federations and interoperability.


GML Appl. Schemafor EO Products

OGC 06-080

OGC Filter Encoding

OGC 04-095ISO 19143

RIM extension package

for EO2

OGC 06-131Refers to

Catalogue 2.0.1

OGC 04-021

CSW ebRIM AP

OGC 05-025

GML 3.2

ISO 19136

OASIS RIM

ISO/TS 15000-3

Uses

Specification StructureSpecification Structure


Platforminformation

Taxonomy

Acquisitioninformation

Additional Information

Archivinginformation

Productinformation

EO ebRIM Extension PackageEO ebRIM Extension Package

Browseinformation

Maskinformation


Possible ArchitecturesPossible Architectures

OGCEO CAT

OGCEO CAT

ProxyImpl.

XCAT

LegacyCatalog

A

Get

Reco

rds

GetRecords

GetRecords

Legacynativequery

Legacynativeresponse

response response

response

resp

onse

OGCEO CAT

ProxyImpl.

YCAT

LegacyCatalog

BReplicate

Catalogue federation

Harvest

OGCEO CAT

StandaloneImpl.

ZEOGML

request


EO Proxy Framework

LegacyCatalogue

● IONIC HMA-T objectives:

● build an EO proxy framework to allow the implementation of EO CS-W ebRIM proxies in front of legacy catalogs

● Productize EO Proxy framework developments (released as library set based on IONIC RedSpider Aug 07)

legacyconnector

EO CS-W Proxy FrameworkEO CS-W Proxy Framework

EOebRIMCS-WEO GML

user

ebXML


Users


EO Proxy FrameworkEO Proxy Framework

request module

responsemodules

Decoderlegacy format

to GML EO

EncoderOGC Filter tolegacy Filter

Decoderlegacy format

to ebRIM

legacy catalog

CSW ebRIMoperations

-GetCapabilities

-GetRecords

-GetRecordById

-GetRepositoryItem

legacyconnector

• Web Service• Database• Proprietary API• Any Service


EO Proxy Framework

LegacyCatalogue

IONIC EO CS-W Proxy Framework:● proposes simple interfaces● hides ebRIM structure to concentrate on EOProduct structure

● manages automatically ebRIM parsing/creation and EO GML creation

legacyconnector


EOebRIMCS-WEO GML

user

ebXML

/** add EOP level metadata */m_EOProductOHR.setAcquisitionType("NOMINAL");m_EOProductOHR.setStatus("ACQUIRED");m_EOProductOHR.setImageQualityDegradation(0);m_EOProductOHR.setAcquisitionStation("TLS");m_EOProductOHR.setOrbitNumber(12);…


To implement a legacy connector, the developer should build three modules to handle following functionalities:

1. Manage Legacy catalog connection

2. Transform OGC filter into one or more valid legacy queries (request module)

request module

EO Frameworkdecode

OGC filterinto

java classes

translateto legacyquery(ies)

legacyrequest

GetRecords , GetRecordById or GetRepositoryItem

response

legacyconnector

manage connection

legacy catalog


3. Populate “EOProduct” Java objects with legacy responses.

These “EOProduct” Java objects will then be automatically translated into a compliant ebXML response or EO GML response. (response module)

Those modules will have to integrate the IONIC EO CS-W Proxy Framework by implementing well documented interfaces and extending abstract classes.

response module

EO Framework encode

ebXML orEO GML response

legacy catalog

legacyconnector

manage connection

PopulateEOProduct

with response

GetRecords, GetRecordById or GetRepositoryItem

response

legacyresponse


● Currently OPT/SAR product level metadata are well defined in specifications, but other will come, so …

● IONIC EO Proxy Framework is designed to allow extensions:

● Ability to define a custom product-level classification (similar to PHR)

● Ability to add custom product-level metadata in EOProducts


/** create and initialize a Classification of type PHR, child of OHR Classification node */EOClassification phrclassification = m_EOExtensionPackage.createClassification(PHR_CONCEPT_NAME, PHR_CONCEPT_KEY ,ClassificationId, m_EOProductOHR.getIdentifier(), EOClassification.CLASSIFICATION_TYPE_OHR);

/** attach the classification to the EOProduct */m_EOProductOHR.setEOClassification(phrclassification);

/** add PHR level metadata */m_EOProductOHR.addSlot("cloudCoverQuotationMode", "AUTOMATIC");m_EOProductOHR.addSlot("imageQualityCodeQuotationMode", "AUTOMATIC");m_EOProductOHR.addSlot("cloudCoverAssessmentConfidence", 12.34);


Proxy AlternativesProxy Alternatives

Stylesheet translation

• Examples prototyped up to now are

EOLI-XML (ESA-MUIS)

ECHO (NASA-EOSDIS)


Supporting ToolsSupporting Tools

Currently stand-alone implementations also have been developed (or are in development)

• IONIC RedSpider (COTS)

• GALDOS IndiCio (COTS)

• OpenSource TBC

Main Issue is conformance testing: HMA-T

• Permit consolidation/evolution of HMA standards in parallel with

implementation.

• Permit conformance testing of HMA standards

• Permanent testbed based on SOA architecture, located at ESRIN


Conformance testsConformance tests

Context

• OGC migrates to open-source Test Engine (http://sourceforge.net/projects/teamengine)

• Compliance Test Language (CTL), Draft 0.2, Northrop Grumman IT, TASC

• OWS-4 reimplemented (some) existing conformance tests


Conformance tests (cont'd)Conformance tests (cont'd)

CITE test scripts development for

• CSW ebRIM Application Profile (Galdos, OWS-4): OGC 07-049

• ebRIM extension package for EO (Galdos) Planned delivery soon

Deployment of CITE test environment at ESRIN


Conformance testsConformance tests


EXAMPLESEXAMPLES


Spot Image - DALISpot Image - DALI


Spot - GetRecordByIDSpot - GetRecordByID


EumetsatEumetsat


ESA - MUISESA - MUIS


NASA ECHONASA ECHO


Multi-catalogue searchMulti-catalogue search


Service Viewpoint: Service Viewpoint: 07-038 Discovery07-038 Discovery

Use of UDDI for service configuration management, and for COTS tools oriented schema interrogation and service chaining

CSW ISO profile found broadly applicable for collection and service metadata discovery

• providing association between collection and service

• providing geographic extent to services

ESA supporting development of ebRIM extension pacakage for 19115/19119

• OGC 07-038


Service DiscoveryService Discovery

• OASIS UDDI v2


Collection DiscoveryCollection DiscoveryPrototypePrototype



• CSW ISO AP & OGC 07-025



MUIS2WMSMUIS2WMS“Present”“Present”


Reference analysisReference analysis

Metadata standards

• Elements to be included in the registry model

• Based on the ISO AP queryables and returnables

• 2 main options Define a simple registry structure with all

elements in a single extrinsic object– Valid option, but:– Management of the semantics not

compliant with ebRIM CS-W AP– Extensions cannot be processed

smoothly– Interest in defining/testing a more

complex registry structure

Define a registry structure closer to the ISO structure

– More complex queries

ISOMetadata

nameexternalIdentifier<<slot>> title<<slot>> created<<slot>> modified<<slot>> issued<<slot>> responsiblePartydescription<<slot>> type[0..*]<<slot>> typeDescription[0..*]<<classification>> KeywordScheme[0..*]<<slot>> source[0..*]<<slot>> format[0..*]<<slot>> references[0..*]<<slot>> date<<slot>> identifier[0..1]<<slot>> metadataLanguage[0..1]<<classification>> CharacterSet[0..1]<<slot>> conformsTo[0..1]<<classification>> SpatialRepresentationType[0..*]<<slot>> resolution[0..*]<<slot>> scale[0..*]<<slot>> dataLanguage[0..*]<<classification>> TopicCategoryCode[0..*]<<slot>> temporal[0..*]<<slot>> envelope[0..*]<<slot>> coverage[0..*]<<classification>> ServiceType[0..*]<<classification>> CouplingType[0..1]<<slot>> serviceOperationName[0..*]<<classification>> DCPList[0..*]<<slot>> operationReferences[0..*]<<slot>> rights[0..*]<<slot>> referenceSystemIdentifier[0..*]<<slot>> referenceSystemCodeSpace[0..*]<<slot>> referenceSystemCodeSpaceVersion[0..*]<<slot>> graphicOverviewName[0..*]

<<ExtrisicObject>>

CitedItem<<ExtrinsicObject>>

IdentifiedItem<<ExtrinsicObject>>

11

Authority

MetadataInformation

<<slot>> date : DateTime<<slot>> identifier [0..1] : InternationalString<<slot>> language [0..1] : Language<<classification>> CharacterSet[0..1]<<classification>> conformsTo[0..1]

<<ExtrinsicObject>>Rights

(from Basic Extension Package)

<<ExtrinsicObject>>

Image(from Basic Extension Package)

<<ExtrinsicObject>>

ResourceMetadata

description : InternationalString<<classification>> ObjectType[0..*]<<classification>> KeywordScheme [0..*] : KeywordType [0..*]<<slot>> source[0..*] : InternationalString<<slot>> format[0..*]<<slot>> references[0..*] : URI<<slot>> type[0..*] : InternationalString

<<ExtrinsicObject>>

0..*0..*

ResourceReferenceSystem

0..10..1

ResourceMetadataInformation

0..10..1

ParentMetadataInformation0..*0..*

ResourceConstraints0..*0..*

GraphicOverview


OrientationsOrientations Based on the analysis of the reference documents, the team’s

orientations were to:• Information Model

Repository contains the ISO 19139 instances; Registry contains a subset of ISO 19139 metadata elements.

Keep the registry structure close to the ISO standards Support the OGC Core queryables and returnables Support the ISO AP additional queryables and returnables at the summary

level Support INSPIRE Metadata elements (except Conformance) Reuse and extend the extrinsic objects, associations and slots defined in the

Basic Package Apply the rules defined in the OWS-4 CSW ebRIM Modelling Guidelines IPR

• Service Provide capabilities equivalent to those of the ISO AP Offer discovery-level search capabilities Support and refine the service interface semantics defined in the ebRIM AP


ImplementationsImplementations

First implementation during FedEO pilot (Ionic)

OpenSource implementation now being planned


Standardisation StatusStandardisation Status

CSW 2.0.2 – approved standard

CSW ebRIM profile – v1.0 draft almost ready for internal e-vote (needs alignment to 2.0.2 following final update from public comment period on v0 draft)

CSW EO extension package and CIM (Core Iso Metadata) extension package ready for e-vote, both discussion paper status, but corresponding “standard working group” will be set up to deal with comments towards

Push to do final presentation to next OGC TC (Ispra, December)


Various other catalogue Various other catalogue initiativesinitiatives


GEOSS Registry & GEOSS Registry & ClearinghouseClearinghouse


Other ebRIM examplesOther ebRIM examples

NASAESG


A possible OGC/ISO view on A possible OGC/ISO view on catalogablescatalogables

ISO 19110 for feature catalogues

ISO 19111 and ISO 19112 for coordinate systems

ISO 19114 for quality evaluation procedures

ISO 19117 for portrayal information

ISO 19126 for data dictionaries

ISO 19127 for geodetic codes and parameters

ISO 19131 for data product specifications

ISO 19135 for registers

ISO 19138 for data quality measures

… + SENSORS, MODELS, COMPONENTS?? …

ISO 19115-2 Catalogue Extension

Package

ebRIM CS-W Basic Extension

Package

ISO 19110 Catalogue Extension Package


Community-specificISO 19115 Catalogue Extension Package


ISO 19135 RegisterExtension Package



Register branchCatalogue branch


…

…

…ISO 19115-2

Catalogue Extension Package

ebRIM CS-W Basic Extension

Package



Community-specificISO 19115 Catalogue Extension Package





Register branchCatalogue branch


…

…

…

With differences between catalogue and registry:

governance and versioning


DISCUSSIONDISCUSSION

WGISS 24 October 2007 Slide 1 Interoperable Catalogues from HMA - FedEO Slides assembled from various sources, with thanks to the contributers to HMA

Documents

eo products ogc

eo metadata type eo

gmes slide

z eo gml request slide

esa slide

gml eo encoder ogc filter

eo ebrim catalogue federations

specific mission slide