D-Net Technology
Paolo Manghi
Istituto di Scienza e Tecnologia dell’Informazione (ISTI) Italian National Research Council (CNR)
DRIVER production
220+ harvested repositories
29 countries from Europe and beyond
1,200,000+ open access publication metadata records
What’s behind?
Seems an Aggregative Digital Library Systems (ADLSs)
Aggregation system Portal
D-Net Software ToolkitService-Oriented Architecture
Enabling a repository infrastructure
Why D-Net and not an ADLS?DRIVER project requirements
Sustainability issues
Typical ADLS architectureOne institution, one community
BASE (Germany) DAREnet (Netherlands), OAIster (USA) Others…
OAI-PMHOAI-PMH OAI-PMH
…
Aggregator
Index Index
UI
…Store Store…
Search
Portal
Aggregation system
Info Space
The DRIVER project goals
Many institutions, many communitiesConstruction of the European Information Space for Open Access research publications Arbitrary number of repositories (data providers) Data curation
Arbitrary number of applications consuming the Information Space Community portals Community Information Spaces (comm. metadata format) Etc.
Highly evolving requirements!
Typical ADLSs Vs DRIVER requirements
OAI-PMHOAI-PMH OAI-PMH
…
Aggregator
Index Index
UI
…
Applications
Aggregation system
Store Store…
Search
OAI-PMH
New Institution Site
Manual maintenance
cost
New Fun
Info Space
New UI
Index
Store
New Info Space
AggregatorOAI-PMH
Typical ADLSs Vs DRIVER requirementsDrawbacks
Limited customizability E.g. pre-defined input and target metadata formats
High-cost software extensibility E.g, new functionality, new Information Spaces
“Manual” repository management Registration, harvesting, curation (XSLT), etc…
“Manual” administration for robustness and scalability
E.g., store and index replicas
Issue: sustainability in the long term
7
D-Net, an Infrastructural approach
Service-Oriented Architecture Web Service, service registration,
subscription¬ification, component-oriented, etc.
System: distributed environment Administered by one responsible organization (RO) Used for by participating organizations (POs) to
collaboratively build ADLSs D-Net’s ADLSs are service-oriented applications
ALDS components (Lego approach) Enabling services: infrastructure middleware Data Management services: aggregation systems End-user Functionality services: portals
Building ADLSs in D-Net
Service distribution Components can be transparently distributed over
the network
Application “autonomicity” Services can be orchestrated automatically to
accomplish certain tasks
Service sharing Hardware and services
Service compositionality and openness Functionality “isolation”; e.g. harvest, index, storage
Service customizability e.g., parametric w.r.t. from metadata formats
9
Distribution and autonomicity
OAI-PMH
Institution Site
Index
Search
Index
UI
…
OAI-PMH
Institution Site
…OAI-PMH
Institution Site
OAI-PMH
Institution Site
…
Ena
blin
g La
yer
Mid
dlew
areUI
Search
Index
Store
Aggregator
User Profiling
…
Others
Aggregator
ServiceKits
Aggregator
Store
ContentResources
Dynamic, distributedRun-time Infrastructure
RO
PO
Sharing of functionality
OAI-PMH
Aggregator
Index
Search
Index
UI
…
OAI-PMH
Institution Site
…OAI-PMH
Institution Site
OAI-PMH
Institution Site
…
Ena
blin
g La
yer
Mid
dlew
areUI
Search
Index
Aggregator
User Profiling
…
Others
Aggregator
StoreStore
Institution Site
ContentResources
Dynamic, distributedRun-time Infrastructure
ServiceKits
RO
UI
POPO
Sharing of content
OAI-PMH
Aggregator
Index
Search
Index
UI
…
OAI-PMH
Institution Site
…OAI-PMH
Institution Site
OAI-PMH
Institution Site
…
Ena
blin
g La
yer
Mid
dlew
areUI
Search
Index
Aggregator
User Profiling
…
Others
Aggregator
Search
Store
Store
Institution Site
ContentResources
Dynamic, distributedRun-time Infrastructure
ServiceKits
RO
UI
POPO
Sharing of content
OAI-PMH
Aggregator
Index
Search
Index
UI
…
OAI-PMH
Institution Site
…OAI-PMH
Institution Site
OAI-PMH
Institution Site
…
Ena
blin
g La
yer
Mid
dlew
areUI
Search
Index
Aggregator
User Profiling
…
Others
Aggregator
Search
Index
Store
Store
ServiceKits
Institution Site
Dynamic, distributedRun-time Infrastructure
ContentResources
RO
UI
POPO
OpennessExample: DRIVER-II v2.0 extension
OAI-PMH
Harvester
Index Index
UI
…
OAI-PMH
Institution Site
…OAI-PMH
Institution Site
OAI-PMH
Institution Site
…
Ena
blin
g La
yer
Mid
dlew
areUI
Search
Index
Aggregator
User Profiling
…
Others
Search
Store
ServiceKits
Institution Site
Dynamic, distributedRun-time Infrastructure
ContentResources
Transformer
Store
RO
PO
Compositionality and customizability
HarvesterOAI-PMH
Index UISearch
Index UISearchStore
Store
Index
Transformer
Store Index
UI
Search
Metadata Formats
Index UISearch
D-Net’s Infrastructure benefitsSustainable business model for ADLS
Low ADLS realization costs By customizability, compositionality, sharing and
openness
Low administration costs By autonomic behaviour
Higher scalability and robustness By distribution and replicas
Repository management tools GUIs for harvesting and aggregation of arbitrary
numbers of repositories, from-to arbitrary metadata formats
D-NetSoftware packages Open Source Apache License Release v1.0 (production) and v1.2 (beta):
installable with technical support Release 2.0 (beta): Enhanced Publications, due
for the 31st of November 2009
Technical partners Istituto di Scienza e Tecnologia dell’Informazione,
National Research Council, Italy ICM Research Centre Warsaw, Poland National and Kapodistrian University of Athens,
Greece University of Bielefeld Library, Germany
DRIVER production infrastructureD-Net’s release v1.1
?
Enabling Layer
Data Layer
EU Open AccessRepositories
Functionality Layer
Adm
inis
trat
ors
End
use
rs
Advanced User InterfacesLight User Interfaces
PO
PO
RO
DRIVER production infrastructureCurrent status v1.1
Content220+ harvested repositories (more than twice to come)
29 countries from Europe and beyond
1,200,000+ OA publication metadata records
ServicesD-Net: 30 different kinds of Services
36 service running instances
9 snodes at CNR, UNIBI, ICM and NKUAUp and running 99% for 14 months (August 2008)
D-Net’s current uptake
European Film Gateway EC projectwww.europeanfilmgateway.eu
OpenAIRE EC pilotwww.openaire.eu
Experimentation of deployment of new infrastructure instances
China, India, Portugal, Belgium, Spain, Slovenia
Credits
Paolo Manghi
ISTI - CNR
Speaker’s Contact [email protected]
DRIVER II Project
http://www.driver-community.eu/
Supported by European Commission