D-Net Technology Paolo Manghi Istituto di Scienza e Tecnologia dellInformazione (ISTI) Italian National Research Council (CNR)

D-Net Technology

Paolo Manghi

Istituto di Scienza e Tecnologia dell’Informazione (ISTI) Italian National Research Council (CNR)

DRIVER production

220+ harvested repositories

29 countries from Europe and beyond

1,200,000+ open access publication metadata records

What’s behind?

Seems an Aggregative Digital Library Systems (ADLSs)

Aggregation system Portal

D-Net Software ToolkitService-Oriented Architecture

Enabling a repository infrastructure

Why D-Net and not an ADLS?DRIVER project requirements

Sustainability issues

Typical ADLS architectureOne institution, one community

BASE (Germany) DAREnet (Netherlands), OAIster (USA) Others…

OAI-PMHOAI-PMH OAI-PMH

…

Aggregator

Index Index

UI

…Store Store…

Search

Portal

Aggregation system

Info Space

The DRIVER project goals

Many institutions, many communitiesConstruction of the European Information Space for Open Access research publications Arbitrary number of repositories (data providers) Data curation

Arbitrary number of applications consuming the Information Space Community portals Community Information Spaces (comm. metadata format) Etc.

Highly evolving requirements!

Typical ADLSs Vs DRIVER requirements

OAI-PMHOAI-PMH OAI-PMH

…

Aggregator

Index Index

UI

…

Applications

Aggregation system

Store Store…

Search

OAI-PMH

New Institution Site

Manual maintenance

cost

New Fun

Info Space

New UI

Index

Store

New Info Space

AggregatorOAI-PMH

Typical ADLSs Vs DRIVER requirementsDrawbacks

Limited customizability E.g. pre-defined input and target metadata formats

High-cost software extensibility E.g, new functionality, new Information Spaces

“Manual” repository management Registration, harvesting, curation (XSLT), etc…

“Manual” administration for robustness and scalability

E.g., store and index replicas

Issue: sustainability in the long term

7

D-Net, an Infrastructural approach

Service-Oriented Architecture Web Service, service registration,

subscription&notification, component-oriented, etc.

System: distributed environment Administered by one responsible organization (RO) Used for by participating organizations (POs) to

collaboratively build ADLSs D-Net’s ADLSs are service-oriented applications

ALDS components (Lego approach) Enabling services: infrastructure middleware Data Management services: aggregation systems End-user Functionality services: portals

Building ADLSs in D-Net

Service distribution Components can be transparently distributed over

the network

Application “autonomicity” Services can be orchestrated automatically to

accomplish certain tasks

Service sharing Hardware and services

Service compositionality and openness Functionality “isolation”; e.g. harvest, index, storage

Service customizability e.g., parametric w.r.t. from metadata formats

9

Distribution and autonomicity

OAI-PMH

Institution Site

Index

Search

Index

UI

…

OAI-PMH

Institution Site

…OAI-PMH

Institution Site

OAI-PMH

Institution Site

…

Ena

blin

g La

yer

Mid

dlew

areUI

Search

Index

Store

Aggregator

User Profiling

…

Others

Aggregator

ServiceKits

Aggregator

Store

ContentResources

Dynamic, distributedRun-time Infrastructure

RO

PO

Sharing of functionality

OAI-PMH

Aggregator

Index

Search

Index

UI

…

OAI-PMH

Institution Site

…OAI-PMH

Institution Site

OAI-PMH

Institution Site

…

Ena

blin

g La

yer

Mid

dlew

areUI

Search

Index

Aggregator

User Profiling

…

Others

Aggregator

StoreStore

Institution Site

ContentResources

Dynamic, distributedRun-time Infrastructure

ServiceKits

RO

UI

POPO

Sharing of content

OAI-PMH

Aggregator

Index

Search

Index

UI

…

OAI-PMH

Institution Site

…OAI-PMH

Institution Site

OAI-PMH

Institution Site

…

Ena

blin

g La

yer

Mid

dlew

areUI

Search

Index

Aggregator

User Profiling

…

Others

Aggregator

Search

Store

Store

Institution Site

ContentResources

Dynamic, distributedRun-time Infrastructure

ServiceKits

RO

UI

POPO

Sharing of content

OAI-PMH

Aggregator

Index

Search

Index

UI

…

OAI-PMH

Institution Site

…OAI-PMH

Institution Site

OAI-PMH

Institution Site

…

Ena

blin

g La

yer

Mid

dlew

areUI

Search

Index

Aggregator

User Profiling

…

Others

Aggregator

Search

Index

Store

Store

ServiceKits

Institution Site

Dynamic, distributedRun-time Infrastructure

ContentResources

RO

UI

POPO

OpennessExample: DRIVER-II v2.0 extension

OAI-PMH

Harvester

Index Index

UI

…

OAI-PMH

Institution Site

…OAI-PMH

Institution Site

OAI-PMH

Institution Site

…

Ena

blin

g La

yer

Mid

dlew

areUI

Search

Index

Aggregator

User Profiling

…

Others

Search

Store

ServiceKits

Institution Site

Dynamic, distributedRun-time Infrastructure

ContentResources

Transformer

Store

RO

PO

Compositionality and customizability

HarvesterOAI-PMH

Index UISearch

Index UISearchStore

Store

Index

Transformer

Store Index

UI

Search

Metadata Formats

Index UISearch

D-Net’s Infrastructure benefitsSustainable business model for ADLS

Low ADLS realization costs By customizability, compositionality, sharing and

openness

Low administration costs By autonomic behaviour

Higher scalability and robustness By distribution and replicas

Repository management tools GUIs for harvesting and aggregation of arbitrary

numbers of repositories, from-to arbitrary metadata formats

D-NetSoftware packages Open Source Apache License Release v1.0 (production) and v1.2 (beta):

installable with technical support Release 2.0 (beta): Enhanced Publications, due

for the 31st of November 2009

Technical partners Istituto di Scienza e Tecnologia dell’Informazione,

National Research Council, Italy ICM Research Centre Warsaw, Poland National and Kapodistrian University of Athens,

Greece University of Bielefeld Library, Germany

DRIVER production infrastructureD-Net’s release v1.1

?

Enabling Layer

Data Layer

EU Open AccessRepositories

Functionality Layer

Adm

inis

trat

ors

End

use

rs

Advanced User InterfacesLight User Interfaces

PO

PO

RO

DRIVER production infrastructureCurrent status v1.1

Content220+ harvested repositories (more than twice to come)

29 countries from Europe and beyond

1,200,000+ OA publication metadata records

 ServicesD-Net: 30 different kinds of Services

36 service running instances

9 snodes at CNR, UNIBI, ICM and NKUAUp and running 99% for 14 months (August 2008)

D-Net’s current uptake

European Film Gateway EC projectwww.europeanfilmgateway.eu

OpenAIRE EC pilotwww.openaire.eu

Experimentation of deployment of new infrastructure instances

China, India, Portugal, Belgium, Spain, Slovenia

Credits

Paolo Manghi

ISTI - CNR

Speaker’s Contact [email protected]

DRIVER II Project

http://www.driver-community.eu/

Supported by European Commission