Repositories: Disruptive Technology or Disrupted Technology? Sandy Payette, Executive Director DORSDL Workshop at ECDL 2008 September 2008.

Repositories: Disruptive Technology or Disrupted Technology?

Sandy Payette, Executive Director

DORSDL Workshop at ECDL 2008September 2008

What does this talk have to say about “applications” which is the

theme of this track?

Social and Technical Forces Waves of Repository-Enabled Applications

• Institutional Repo and Digital Library Apps• IR: Scholars to deposit articles, etc.• DL: Digital library search and access; collections

• Collaborative “Web 2.0”• collaborative filtering• annotate; discuss scholarly materials

• E-Science, E-Research, Data-Intensive• Publications linked to data• Data aggregation from distributed sources• Fusion, simulation

Implications for Repositories

mor

e di

stri

bute

d

mor

e co

llab

orat

ive

mor

e w

eb-o

rien

ted

mor

e op

en

mor

e in

tero

pera

ble

We can build amazing private islands

But should we?

Repository Island

Infrastructure

Emergence of Infrastructure

Source: Understanding Infrastructure: Lessons for New Scientific Infrastructure, http://deepblue.lib.umich.edu/handle/2027.42/49353

Systems

Heterogeneous componentsCentral controlClosed, stableDedicated/improvised gateways

Heterogeneous systemsDistributed controlCoordinationGeneric gatewaysOpen, reconfigurable

Networks

Repositories as components of networked information

infrastructure

Which are repositories?

Internet Archive

Google Data

DSpaceFedora

E-Prints

S3

Repositories: Disruptive or Disrupted?

• Where’s the multi-institutional perspective?• Where’s the Web?• What constraints do we impose by starting with the

perspective of a single institution or “enterprise”?• What impact will “Cloud” storage and computing have?• What is the best path to real interoperability?• When do you need complex vs. simple and good enough?• How assert leadership in emerging value networks?

Right Direction …

Exposure and Re-Use of Digital Content

Object Re-Use and Exchange (OAI-ORE)

Data model for describing bounded aggregations in Web graph

http://www.openarchives.org/ore/

Hubble optical observationBaltimore, MD

Basic object informationStrasbourg, France

E-Science AggregationsAnd in digital scholarly communication, the single

container concept is obsolete.

text

2006 Astrophysics paper

X-MM-Newton X-ray observationVilspa, Spain

Chandra X-ray observationCambridge, MA

A1795

Fedora objects can map to ORE

Identified, bounded aggregations of related information units that form a logical whole.

Components of a digital object may vary according to:– Semantic type: book, article, software,

dataset, simulation, …

– Media type: text, image, audio, video, mixed

– Media format: PDF, HTML, JPEG, MP3, …

– Network location

– Relationships: internal, external

HTMLHTML

MP3 MP3

Identifier

PDFPDF

ORE Resource Map Exposure

ore:describes

Network of Digital Objects … aggregations related to aggregations

PID 4

PID 1

PID 3

hasPart hasPart

ore:describes

ore:describesore:describes

DCDC

DC

HTML

Another right direction…Repositories Embedded in Infrastructure

(Fedora as case study)

A History of Fedora Repository Project’s Technical Approaches to Interoperability

• 1998: Interfaces• Generic interfaces to access and manage digital objects in a repository• Extensible interfaces on digital objects (“Disseminators”) to provide uniform service access

points for heterogeneous underlying content

• 2001: Web Services• APIs to access and manage digital objects in a repository • SOAP and REST over HTTP

• 2002: XML-based digital object serialization formats• METS• FOXML (Fedora Object XML)

• 2005: Semantic Technologies and RDF (RDF-based “Resource Index”)

Now, make it fit better with Web in 2008+

• 2008: Atom Syndication Format • New with Fedora 3.0 via ingest/export operations

• 2008: OAI-ORE • Experiments completed• Work in progress for Fedora 3.0/3.1 support

• 2008-09: Adopt simple, common Web API(s) with wide appeal• Atom Publishing Protocol• SWORD• Other?

• 2008-09: Connect backend storage to “Cloud”

Fedora 2008-09: new Web exposures

New WebAPIs

FedoraAPIs

RegistrySearch

RDF Query

Ingest

Export

Manage

Validate

Access

RDF IndexStore Registry

File system RDBMS(Registry)

Disseminate

New Web APIs: Atom PublishingSWORDOther TBD

OAI-ORE (2008)Atom (2008) new formats

Triplestore

SPARQL/MulgaraLinked Data?

New:

Policy

Object serialization - Atomfe

eden

try

(1-n

)

New Akubra Project … backend abstraction

New WebAPIs

FedoraAPIs

RegistrySearch

RDF Query

Store

AkubraStorage Abstraction

Plug-in 1 Plug-in 2 Plug-in …

File System

Sun Honeycomb

Fedora Repository Service

registrydb

Cloud Storage:-Amazon S3 (now)-Google Data (next) - Other TBD

Other stores TBD…

Repositories Not-Disrupted

• Expose repositories in a global networked environment; not just as local, closed systems

• Make it easier to access and re-use content that is stored in digital repositories, especially on the Web

• Make repositories conform to common protocols, formats, and standards that are being used in the Web; get repositories in the game of emerging value networks

• Provide transformations that enable content to be reusable in different contexts.

• Create repository connectors for common authoring applications to position repositories to capture content at beginning of its life, not at just end of it.

More Info:http://fedora-commons.org/

http://fedora-commons.org/confluence

Questions and Discussion