OBIS Portal Architecture Concepts plus potential for utilization as a basis for Regional OBIS Nodes Tony Rees, CSIRO Marine Research, Hobart (and OBIS.

OBIS Portal Architecture Conceptsplus potential for utilization as a basis for Regional

OBIS Nodes

Tony Rees, CSIRO Marine Research, Hobart (and OBIS Technical Subcommittee)

for: OBIS Nodes meeting, Halifax, September 2004

OBIS Architecture – Version 1 (2002)

data provider 1

data provider 2

data provider 3

(etc.)

www user 2

www user 3

(etc.)

Mapping tool 3

Mapping tool 2

all queries

OBIS Portal

C-squares mapper

= custom database wrapper

www user 1

Typical search sequence – OBIS version 1

Typical search result – OBIS version 1

OBIS Architecture – Version 1 (2002)

data provider 1

data provider 2

data provider 3

(etc.)

www user 2

www user 3

(etc.)

Mapping tool 3

Mapping tool 2

all queries

OBIS Portal

C-squares mapper

= custom database wrapper

www user 1

many searches return no data !!!(user does not know what is a viable query)

What is in the Index ...

OBIS Index

name index

• scientific “names with data” from data providers

• “names without data” from Catalogue of Life

• common names from Cat. of Life

• “near match” versions of scientific names

• allocation of all taxa to OBIS taxonomic categories

• synonyms transferred to current names

• metadata on each species (how many records, etc.)

• filters on marine vs. non-marine species, etc.

spatial index

• list of geographic units (squares) in which each species occurs – see next slide

New concept (1) developed in 2003/4 – the “OBIS Index”

Spatial indexing units used for OBIS: 0.5 x 0.5º squares (approx. 50 km resolution) – stored in “c-squares” notation (www.marine.csiro.au/csquares)

OBIS Index

75007500:47500:499

7500:499:4

Part of the spatial index (actually from another database, which employs finer resolution squares):

OBIS Index

New search options – current OBIS version

“Stage 1” searches – return lists of names, with metadata (nos. of records, etc.), taxonomic groups, common names, and “quick maps”

Other search options – scientific name + category search

Other search options – browse by taxonomic category

Other search options – clickable maps by taxonomic category

Example “Stage 1” search result – e.g. genus = “Lutjanus”

For each species ... (NB, all this information is from the Index) – example row:

from Catalogue of Life information

allocation to OBIS taxonomic category

metadata (from parsing the Cache content)

from the Spatial Index – species distribution by 0.5 x 0.5 degree squares

from Cat. of Life info

pre-formatted link to “Stage 2 search”

What is in the Cache ...

• One record per OBIS data point, with copy of key information fields for that point

• Cache content is built by crawling the data providers (and refreshing at intervals)

• Purpose is to provide insulation from providers being off line at any time, and to improve data retrieval speed

• Also as by-product, makes the task of keeping the Index up-to-date relatively simple (crawling of the provider content is already done).

New concept (2) developed in 2003/4 – the “OBIS Cache”

OBIS Cache

Pre-formatted link to “Stage 2” search (get OBIS data) ...

Typical search result – “Stage 2” search (2004)

OBIS Architecture – version 2

www user 1

www user 2

www user 3

(etc.)

Mapping tool 3

Mapping tool 2

“Quick maps”

OBIS Portal (search

application)

OBIS Cache

OBIS Index

“Stage 1”(get info) queries

C-squares mapper

“Stage 2” (get data) queries data provider 1

data provider 2

data provider 3

= DiGIR translation software

global names list (partially complete)

Cat. of Life

metadata refresh

(etc.)provider crawling

Index building

Implications ...

• Index is a self-contained guide to what data are available in the OBIS system (metadata layer) – makes the Portal “intelligent” (content aware)

• Can use the Index as a standalone tool to answer:

– For what species does OBIS have data (and how much), at any particular time

– What is the distribution of species “X” (by 0.5 x 0.5 degree squares) – displayed as “Quick Map” using the c-squares mapper

– Which species occur in region “Y” (0.5 degree square or larger – default query from main map entry point is currently set to 10 x 10 degrees)

– Browse OBIS content, e.g. by category, genus, alphabetical, plus show summary statistics (e.g. numbers of records by category)

– Auto-complete scientific names, correct misspellings, etc.

(above are “Stage 1” [Index] searches)

– Provide 2 entry points to “Get OBIS Data” (stage 2 search):

• Pre-formatted hyperlink to retrieve all data for a species (1-40,000 records), no additional typing required

• Click on any “Quick Map” to retrieve spatially filtered subset for a species.

Options for OBIS Nodes (under this model)...

Possible OBIS node – minimalist configuration

www user 1

www user 2

www user 3

(etc.)

(at remote location)

Mapping tool 3

Mapping tool 2

“Quick maps”

“Stage 2” (get data) queries

OBIS Portal (search

application)

OBIS Cache

OBIS Index

“Stage 1”(get info) queries

C-squares mapper

Possible OBIS node – expanded configuration

www user 1

www user 2

www user 3

(etc.)

Mapping tool 3

Mapping tool 2

“Quick maps”

“Stage 2” (get data) queries

OBIS Portal (search

application)

OBIS Cache

OBIS Index

“Stage 1”(get info) queries

C-squares mapper

The big picture ...

Common OBIS Data Set

OBIS search application – local entry point

www user

Regional Node (Standard

Configuration)

www user

Independent data provider 1

(etc.)

provider crawling

Index building

OBIS search application

(master)

OBIS Cache (master)

OBIS Index (master)

Independent data provider 2

Independent data provider 3

Catalogue of Life

“Quick maps”C-squares mapper

Mapping tool 2

Common OBIS / 3rd party tools (web accessible)

“OBIS Central” – Rutgers

OBIS search application

(copy)

OBIS Cache (copy)

OBIS Index (copy)

www user

Regional Node + full mirror

function

RON sourced data

Mapping tool 3

OBIS search application – local entry point

www user

Regional Node (Standard

Configuration)

OBIS search application – local entry point

www user

Regional Node (Standard

Configuration)