Page 1 CSISS Center for Spatial Information Science and Systems CWIC Development Team Meeting, 2014 CWIC OpenSearch Design and Implementation Yuanzheng.

Page 1

CSISS Center for Spatial Information Science and Systems

CWIC Development Team Meeting, 2014

CWIC OpenSearch Design and Implementation

Yuanzheng Shao, Lingjun Kang, Archie Warnock{yshao3, lkang3}@gmu.edu

warnock@awcubed.com

CWIC Workshop, Jan 28-30, 2014Greenbelt, MD

Page 2

CSISS Center for Spatial Information Science and Systems

Outline

• CWIC OpenSearch Design– Standards– OSDD– Request– Response– Error Handling

• CWIC OpenSearch implementation– Scenario– Integrated catalog

• INPE• GHRSST• USGS/LSI

– Next

Page 3

CSISS Center for Spatial Information Science and Systems

• OGC 10-032r8– OGC OpenSearch Geo and Time Extensions– Latest version, to be approved. – Voting page (https://portal.opengeospatial.org/?

m=projects&a=view&project_id=82&tab=5)• OpenSearch Draft 5

– http://www.opensearch.org/Specifications/OpenSearch/1.1/Draft_5

• ATOM Syndication Format– http://tools.ietf.org/search/rfc4287

CWIC OpenSearch Design - Standards

Page 4

CSISS Center for Spatial Information Science and Systems

• Aim– generating OpenSearch description document dynamically

based on the dataset identifier. Instruct the granule-level querying

– Customized response for each dataset, hard-coded the dataset identifier in the query template

• Parameter– datasetId: used to specify dataset identifier (DIF Entry ID)

• Sample OSDD request (for Landsat_8 dataset)– http://cwicdev.wgiss.ceos.org/opensearch/datasets.atom?dat

asetId=Landsat_8

CWIC OpenSearch Design - OSDD

Page 5

CSISS Center for Spatial Information Science and Systems

• Sample Response <?xml version="1.0" encoding="UTF-8"?>

<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/" xmlns:geo="http://a9.com/-/opensearch/extensions/geo/1.0/" xmlns:time="http://a9.com/-/opensearch/extensions/time/1.0/" xmlns:cwic="http://cwic.wgiss.ceos.org/opensearch/extensions/1.0/"> <ShortName>CWIC OpenSearch</ShortName> <Description>CEOS WGISS Integrated Catalog OpenSearch</Description> <!-- Template --> <Url type="application/atom+xml” template="http://cwicdev.wgiss.ceos.org/opensearch/granules.atom?datasetId=INPE_CBERS2B_CCD&amp;startIndex={startIndex?}&amp;count={count?}&amp;timeStart={time:start?}&amp;timeEnd={time:end?}&amp;geoBox={geo:box?}"/> <!-- Example Queries --> <Query role="example" cwic:datasetId="INPE_CBERS2B_CCD" startIndex="1" count="10" geo:box="-180,-90,180,90" time:start="2010-01-01T00:00:00Z" time:end="2011-01-31T00:00:00Z"/> <Tags>CWIC,CEOS,WGISS,OpenSearch</Tags> <Contact>exec@wgiss.ceos.org</Contact> <Image>http://www.ceos.org/favicon.ico</Image> <Developer>CWIC Development Team</Developer> <Attribution>INPE</Attribution> <SyndicationRight>open</SyndicationRight> <Language>en-us</Language> <OutputEncoding>UTF-8</OutputEncoding> <InputEncoding>UTF-8</InputEncoding></OpenSearchDescription>

CWIC OpenSearch Design - OSDD

Page 6

CSISS Center for Spatial Information Science and Systems

• Issue– How to generate the response without passing dataset

identifier?• Error output• General response (See http://cwicdev.wgiss.ceos.org/opensearch/datasets.atom?)

CWIC OpenSearch Design - OSDD

Page 7

CSISS Center for Spatial Information Science and Systems

• Dataset Identifier (Mandatory)– The dataset identifier (datasetId) parameter is used to

specify the dataset identifier which could be retrieved from IDN as DIF Entry ID. This parameter is a required parameter. The client can not specify more than one dataset in single request

– Required namespace: • xmlns:cwic="http://cwic.wgiss.ceos.org/opensearch/extensions/1.0/"

• Error handling for non-exist Dataset Identifier– Set HTTP Status code to 400– Generate user-friendly exception message in <subtitle>– Send the response to client

CWIC OpenSearch Design – Request (datasetId)

Page 8

CSISS Center for Spatial Information Science and Systems

• Temporal Extension (Optional)– Adopted OGC 10-032r8– Required namespace:

• xmlns:time="http://a9.com/-/opensearch/extensions/time/1.0/"• Parameter Values

– time:start - a string describing the start of the temporal interval to search (bigger or equal to)

– time:end - a string describing the end of the temporal interval to search (smaller or equal to).

• Format– Character string with the start of the temporal interval

according to RFC-3339• date only: YYYY-MM-DD • date-time: YYYY-MM-DDTHH:MI:SSZ

CWIC OpenSearch Design – Request (temporal)

Page 9

CSISS Center for Spatial Information Science and Systems

• GEO Extension (Optional)– Adopted OGC 10-032r8– Required namespace:

• xmlns:geo="http://a9.com/-/opensearch/extensions/geo/1.0/"• Parameter Value

– geo:box - the bouding box specifying the area of interest• Format

– the box is defined by "west, south, east, north" coordinates of longitude, latitude, in an EPSG:4326 decimal degrees.

CWIC OpenSearch Design – Request (GEO)

Page 10

CSISS Center for Spatial Information Science and Systems

• Used namespace in ATOM Response

CWIC OpenSearch Design – ATOM Response

Namespace URL

xmlns http://www.w3.org/2005/Atom

xmlns:opensearch http://a9.com/-/spec/opensearch/1.1/

xmlns:dc http://purl.org/dc/elements/1.1

xmlns:georss http://www.georss.org/georss/10

xmlns:geo http://a9.com/-/opensearch/extensions/geo/1.0/

xmlns:time http://a9.com/-/opensearch/extensions/time/1.0/

xmlns:cwic http://cwic.wgiss.ceos.org/opensearch/extensions/1.0/

openSearch or opensearch

Page 11

CSISS Center for Spatial Information Science and Systems

• <feed> element

CWIC OpenSearch Design – ATOM Response

Element Value

title Fixed value: CWIC OpenSearch Response

updated Current system date/time

author Fixed value: <author><name>CEOS WGISS Integrated Catalog (CWIC) - CWIC Contact - Web: http://wgiss.ceos.org/cwic</name><email>cwic-help@wgiss.ceos.org</email></author>

id Fixed value: http://cwic.wgiss.ceos.org/opensearch/granules.atom

link Provide the “self” link corresponding to the request

link Provide the “search” link corresponding to the OSDD request

opensearch:totalResults The number of search results available for the current search.

opensearch:startPage Replaced with the page number of the set of search results desired by the search client.

opensearch:itemsPerPage The number of search results returned per page.

opensearch:Query Defines a search query that can be performed by search clients.

Page 12

CSISS Center for Spatial Information Science and Systems

• <feed>/<entry> element

CWIC OpenSearch Design – ATOM Response

Element Value

title granule name

id Fixed URL prefix followed by granule identifier. For example: http://cwic.wgiss.ceos.org/opensearch/granules.atom?uid=NAVO-L2P-AVHRR19_L.20091128-AVHRR19_L-NAVO-L2P-SST_s2348_e2352-v01.nc.bz2

updated granule update time

author Fixed value for each data provider

georss:box bounding box of specific entry

dc:date temporal range of specific entry

summary a text construct that conveys a short summary, abstract, or excerpt of an entry.

link see next slide

Page 13

CSISS Center for Spatial Information Science and Systems

• <feed>/<entry>/<link> element

CWIC OpenSearch Design – ATOM Response

Type of link

Definition rel value mime-type

data link representing a data file or other science data resource;

enclosure application/x-netcdf, application/x-hdf, etc.

metadata file with (usually) structured information about corresponding data files

describedBy (ESIP)via (OGC)

text/xml

browser image of the data typically used for making data request decisions

icon image/jpeg, image/pngetc.

order granule ordering url enclosure text/html

Page 14

CSISS Center for Spatial Information Science and Systems

• <feed>/<entry>/<link> for OPeNDAP

CWIC OpenSearch Design – ATOM Response

Type of link

Definition rel value

mime-type

Onlink urn:x-esri:specification:ServiceType:ArcIMS:Metadata:Onlink enclosure text/html

Thumbnail urn:x-esri:specification:ServiceType:ArcIMS:Metadata:Thumbnail icon image/png (for GHRSST)

Document urn:x-esri:specification:ServiceType:ArcIMS:Metadata:Document via text/xml

TDS urn:x-esri:specification:ServiceType:TDS enclosure application/x-bzip2

FTP urn:x-esri:specification:ServiceType:FTP enclosure application/x-bzip2

OPeNDAP urn:x-esri:specification:ServiceType:OPeNDAP enclosure text/html

LAS urn:x-esri:specification:ServiceType:LAS icon image/png (for GHRSST)

Download urn:x-esri:specification:ServiceType:Download enclosure application/x-bzip2

WMS/WCS urn:x-esri:specification:ServiceType:WMSurn:x-esri:specification:ServiceType:WCS

enclosure text/xml

Page 15

CSISS Center for Spatial Information Science and Systems

CWIC OpenSearch Design - Error Handling

Page 16

CSISS Center for Spatial Information Science and Systems

• CWIC OpenSearch uses the HTTP status codes for client (4xx) and server (5xx) errors.– Client-side issues

• 400 Bad Request: The request has an invalid syntax (i.e. badly formatted geometry)

• 413 Request Entity Too Large: The request originates too many returnable hits

– For server side errors• 500 Internal Server Error: Default code for the server side for an

execution error• 501 Not Implemented: When requesting an unimplemented feature

(e.g. relation operator not supported)• 503 Service Unavailable: When the search service is temporarily not

available (due to overload or other reasons)• 504 Gateway Timeout: When the search engine is a broker or

aggregator to other services that fail to produce a answer within a giving time frame

CWIC OpenSearch Design – Error Handling

Page 17

CSISS Center for Spatial Information Science and Systems

• Zero-result handling– Zero-result is not an error! HTTP status code sets to 200– Response shall contain a valid ATOM response without

<entry> element– Specify human-readable message in <subtitle> element– Set the value of <opensearch:totalResults> element to 0– Set the value of <opensearch:startIndex> element to 0– Set the value of <opensearch:itemsPerPage> element to 0

CWIC OpenSearch Design – Error Handling

Page 18

CSISS Center for Spatial Information Science and Systems

• Zero-result sample response

CWIC OpenSearch Design – Error Handling

<?xml version="1.0" encoding="UTF-8"?><feed xmlns="http://www.w3.org/2005/Atom" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/"> <title>CWIC OpenSearch Response</title> <subtitle>No results found for your specific dataset and query parameters</subtitle> <updated>2014-01-22T21:18:20Z</updated> <author> <name>CEOS WGISS Integrated Catalog (CWIC) - CWIC Contact - Web: http://wgiss.ceos.org/cwic</name> <email>cwic-help@wgiss.ceos.org</email> </author> <id>http://cwicdev.wgiss.ceos.org/opensearch/granules.atom</id> <opensearch:totalResults>0</opensearch:totalResults> <opensearch:startPage>0</opensearch:startPage> <opensearch:itemsPerPage>0</opensearch:itemsPerPage></feed>

Page 19

CSISS Center for Spatial Information Science and Systems

CWIC OpenSearch Implementation

Page 20

CSISS Center for Spatial Information Science and Systems

• Existing catalog analysis– Provide OpenSearch API, like ECHO and CCMEO

• Solution: Reuse native OpenSearch API, and convert the response to universal CWIC OpenSearch response

– Provide CSW API, like GHRSST• Solution: Convert CWIC OpenSearch request to CSW request, interact

with remote CSW catalog, parse CSW response and convert it to universal CWIC OpenSearch response

– Provide SOAP API, like USGS/LSI• Solution: Convert CWIC OpenSearch request to SOAP request, interact

with SOAP API, parse SOAP response and convert it to universal CWIC OpenSearch response

– Provide HTTP GET, like INPE• Solution: Issue GET request, retrieve XML response, parse the XML

and convert to universal CWIC OpenSearch response

CWIC OpenSearch Implementation – Scenario

Page 21

CSISS Center for Spatial Information Science and Systems

• Supported catalogs– INPE– USGS/LSI– GHRSST

• Ongoing integrated catalogs– ECHO– CCMEO

• Demo– CWIC-Start

CWIC OpenSearch Implementation

Page 22

CSISS Center for Spatial Information Science and Systems

Discussion