Update on the Fedora Project Common Solutions Group September 2005 Tim Sigmon University of Virginia Special thanks to the Fedora Team for these slides!

Update on the Fedora Project

Common Solutions GroupSeptember 2005

Tim SigmonUniversity of Virginia

Special thanks to the Fedora Team for these slides!

Fedora Development Team

Cornell University

• Sandy Payette (co-director)• Chris Wilper• Carl Lagoze• Eddie Shin

University of Virginia

• Thorny Staples (co-director)

• Ross Wayland• Ronda Grizzle• Bill Niebel• Bob Haschart• Tim Sigmon

Fedora Brief History• Cornell Research (1997-present)

– DARPA and NSF-funded research– First reference implementation developed– Interoperable Repositories (experiments with CNRI)– Policy Enforcement

• First Application (1999-2001)– University of Virginia digital library prototype – Technical implementation: adapted to web; RDBMS

storage– Scale/stress testing for 10,000,000 objects

• Open Source Software (2002-present)– Andrew W. Mellon Foundation grants– Technical implementation: XML and web services– Fedora 1.0 (May 2003)– Fedora 2.0 (Jan 2005)– Fedora 2.1 (coming soon!)

Known Use Cases for“Fedora Inside”

• Digital Library Collections• Institutional Repository • Educational Software

• Information Network Overlay • Digital Archives and Records Management• Digital Asset Management• File Cabinet / Document Management• Scholarly publishing

Fedora Repository 2.x

Manage AuthN AuthZ

Acce ss Validation Resource Index

Storage Dissemination Fie ldSearch

Fedora Repository Modules

M a n a g e A c c e s s S e a rc hR I S e a rc h

(R D F)

R E S T

C lie n tA pp

B a tchPro g ra m

O th e rS e rv ice

W e bB ro ws e r

R E S T S O A PS O A P R E S T S O A PR E S T

O A IP ro v id e r

R E S T

Web ServicesExposure

The FedoraDigital Object Model

Persistent ID ( PID)

Default Disseminator

System Metadata

Datastream (item)

Digital object identifier

Service Perspective: methods for disseminating “views” of content

Internal: key metadata necessary to manage the object

Item Perspective: Set of content or metadata itemsDatastream (item)

Datastream (item)

Your Extension

Your Extension

Fedora Repository 2.x

Manage AuthN AuthZ

Acce ss Validation Resource Index

Storage Dissemination Fie ldSearch

Fedora Repository Modules

M a n a g e A c c e s s S e a rc hR I S e a rc h

(R D F)

R E S T

C lie n tA pp

B a tchPro g ra m

O th e rS e rv ice

W e bB ro ws e r

R E S T S O A PS O A P R E S T S O A PR E S T

O A IP ro v id e r

R E S T

Web ServicesExposure

Fedora – what’s new(version 2.0)

• FOXML (Fedora Object XML)– Simple XML format directly expresses Fedora object model– Easily adapts to Fedora new and planned features– Easily translated to other well-known formats

• Enhanced Ingest/Export of objects– FOXML, METS (Fedora extension)– Extensible to accommodate new XML formats– Planned: METS 1.4, MPEG21 DIDL

Fedora 2.0 (continued)

• Object-to-object Relationships– Ontology of common relationships (RDF schema)– Relationships stored in special datastream (RELS-EXT)

• Resource Index (RI)– RDF-based index of repository (Kowari triple-store)– Graph-based index includes:

– Object properties and Dublin Core– Object Relationships– Object Disseminations

• RI Search (Search the repository as a graph)– Powerful querying of graph of inter-related objects– REST-based query interface (using RDQL or ITQL)– Results in different formats (triples, tuples, sparql)

Fedora 2.0 (continued)

• New Utilities• Batch Modify Utility • Repository Administrator Reporting

• Performance Tuning (~1 million objects)– Ingest testing (800K objects; 40 millisec/object)– Concurrency testing (access requests)

• Communications and Outreach• New Fedora Web Site• Improved Documentation• Tutorials

Preview Fedora 2.1 (Sept. 2005)

• ECL license• Support for SSL• Authentication plug-ins

– Tomcat realms and login modules• Plug-in #1 : Tomcat user/password file or database• Plug-in #2 : LDAP• Plug-in #3 : Radius Authentication

• Authorization module– XML-based policies using XACML– Repository-wide policies– Object-specific policies– Fine-grained policy enforcement

• API actions X subject attributes X object attributes

Authorization: Example Policies

• Repository Policy– Allow access to all API-M methods to “administrator” – Allow access to the deleteDatastream method to “author”

• Specific Object Policy– Allow access to object “uva:100” if user is Thorny.

• Group Object Policy– Allow access to the “getFullArticle” dissemination of objects

whose content model is “journal-article” if “faculty”– Allow access to the “secret” datastream if user is not “guest”

Authorization: Example Policies

• Time-oriented Policy– Permit students access to “answers” datastream of

learning object cs:125 after May 15, 2005– Deny all access to learning object cs:125 after June 15,

2005

• Backend Service Security Policy– Deny callback by external service represented by

Bmech:10

Preview of Fedora 2.1 (Sept. 2005)

• Enhanced OAI Provider Service (prOAI)– Harvest multiple metadata formats– Harvest datastreams and disseminations– Support for incremental harvest by modified date– Support for OAI sets– Highly configurable via queries against Resource Index

• Directory Ingest Service (and client tool)– Facilitate ingest of hierarchical directories of files– Submit files as .zip or .jar (with a METS manifest)– Automatically asserts parent-child relationships in RELS-EXT– Stages content and ingests as FOXML objects into

repository

• Policy Builder Client– Simple user interface to create access policies– Automatically generate XACML

• Handle Generation Plug-in (PIDs as Handles)

Fedora Service Framework(beginning Fedora 2.1)

F e d o r a R e p o s ito r yS e r v ic e

O A IP ro v id e rS e rv ic e

D ire c to ryIn g e st

S e rv ic e

Futur e

Ser v ice

Serv ices

Apps

Other

Ser v ice

Administrator

Other

Ser v ice

Futur e

Ser v ice

PolicyBuilde r

ZIP o r J A Rin p u t

Fedora Service Framework(2005-2006)

F e d o r a R e p o s ito r yS e r v ic e

Serv ices

Apps

P re se rv a tio nIn te g ri tyS e rv ic e

Ex te rn a lW o rk flo w

J H O V E

G D F R

B a sicW o rk flo wS e rv ic e

D i al og B ox N ame

O KText :

Text

Text

Text

Text

Text

Cancel

H elp

Sample Text Her e Sample Text Her e Sample Text

Her e Sample Text Her e Sample Text Her e SampleText Her e Sample Text Her e Sample Text Her e

Sample Text Her e Sample Text Her e

Sam ple Text Her e Sam ple Text Her e Sam ple Text Her e Sam ple Text Her e

Sam ple Text Her e Sam ple Text Her e Sam ple Text Her e Sam ple Text Her e

Sam ple Text Her e Sam ple Text Her e Sam ple Text Her e Sam ple Text Her e

Fedora-Web-IRAdministrator

O A IP ro v id e rS e rv ic e

D ire c to ryIn g e st

S e rv ic e

W e b -b a se dsu b m issio n a n db a sic w o rk flo w

D y n a m icD isse m in a to r

S e rv ic e

PolicyBuilder

F e d o raS e a rc hS e rv ic e

Other

Ser v ice

Fedora Service Framework(2006-2007)

Serv ices

Apps

P re se rv a tio nIn te g ri tyS e rv ic e

Ex te rn a lW o rk flo w

J H O V E

G D F R

B a sicW o rk flo wS e rv ic e

D i al og B ox N ame

O KText :

Text

Text

Text

Text

Text

Cancel

H elp

Sample Text Her e Sample Text Her e Sample TextHer e Sample Text Her e Sample Text Her e Sample

Text Her e Sample Text Her e Sample Text Her eSample Text Her e Sample Text Her e

Sam ple Text Her e Sam ple Text Her e Sam ple Text Her e Sam ple Text Her e

Sam ple Text Her e Sam ple Text Her e Sam ple Text Her e Sam ple Text Her e

Sam ple Text Her e Sam ple Text Her e Sam ple Text Her e Sam ple Text Her e

Fedora-Web-IRAdministrator

O A IP ro v id e rS e rv ic e

D ire c to ryIn g e st

S e rv ic e

W e b -b a se dsu b m issio n a n db a sic w o rk flo w

Pr e s e r vat io nM o n ito r in g

Se r vice

Eve n tNo tif icat io n

Se r vice

F e d o raS e a rc hS e rv ic e

D y n a m icD isse m in a to r

S e rv ic e

PolicyBuilder

Other

Ser v ice

FedoraRepository

Serv ice

F e d o r a R e p o s ito r yS e r v ic e

Fe d e r at io nPID Re s o lu t io n

Se r vice

FedoraRepository

Service

Fedora Web-based IR Client

Web-based client for “institutional repository”• Configurable• End-user submission• Object creation template for “content

models” • Basic Workflow• Search/Browse• Easy configuration of access policy

• Development to begin this fall.

More Dev-Team Priorities

• Federated Repositories• Federation with other repositories (DSpace,

aDORE)– note the Cornell/LANL NSF Pathways project.

• Fedora Showcase and News (on new website)• “Content Model” Specification Language• Advanced Object Creation Workbenches• Tools for RDF browse and graph traversal• Performance Tune – millions of objects• Web services security and Shibboleth• Code Refactoring

• Fedora as web app (.war)• MVC2 pattern for REST-based web exposures• Other misc.

VTLS

• Commercial support and value-add development– Similar to RedHat and Linux– Installation, training, support, hosting, etc.

• VITAL product (based on Fedora) contains VTLS-developed work flow extensions, management utilities, and enhanced searching capabilities

• In partnership with the ARROW project, VTLS is developing and contributing back to open source, e.g., – Handles integration– SRU/SRW interface to expose Fedora content– Metadata extraction and content validation via JHOVE– Automatic capture of technical metadata from images– Facilitate content exposure to web crawlers– Creating custom content models

“Fedora inside”

ARROW

• Australian Research Repositories Online to the World

• Intention of the project is to achieve wider access to Australian research by making it available on-line with appropriate discovery facilities

• “identify and test software or solutions to support best practice institutional digital repositories comprising e-prints, digital theses and electronic publishing”

• Selected Fedora and VITAL as its core repository solutions

• http://arrow.edu.au

“Fedora inside”

NSDL

• National Science Digital Library– Mission: improve Science, Math, and Engineering

education through digital libraries• First implementation was a metadata repository using

Oracle dbms to hold information about collections and items.

• Next generation NSDL will use Fedora to add value to digital content ... don't just provide access.

• Create an information network overlay that supports:– Rich and dynamic information objects– Information reuse/refactoring– Graph-based information model (ontology-based relationships)– Fine-grained access management

• http://www.nsdl.org

“Fedora inside”

AGU Digital Archive

• American Geophysical Union (a publisher) is developing a system for long-term preservation (20, 50, even 100 yrs)– Read the files– Understand the structure of the files– Ensure authentic copy of the work

• Selected Fedora based on an extensive list of requirements

• Emphasis on having good metadata– Descriptive (author, title, volume, ...)– Technical (formats, versions, ...)– Administrative (rights, events, audits, ...)

“Fedora inside”

Preservation of University Records

• Tufts and Yale received a grant from National Historical Publications and Records Commission (NHPRC)– To synthesize electronic records preservation

research with digital library repository research in an effort to develop systems capable of preserving university electronic records at both institutions

– To test the potential of Fedora to serve as the architecture for an electronic records preservation system.

“Fedora inside”

www.fedora.info