VITAL – an Open Source/Proprietary Source Solution for Institutional Repositories built on a unique funding and partnership model. The JISC/CNI Meeting.

VITAL – an Open Source/Proprietary Source VITAL – an Open Source/Proprietary Source Solution for Institutional Repositories built on a Solution for Institutional Repositories built on a

unique funding and partnership model. unique funding and partnership model.

The JISC/CNI Meeting The JISC/CNI Meeting July 6, 2006 York, UK July 6, 2006 York, UK

Vinod Chachra, PhDVinod Chachra, PhDCEO, VTLS Inc.CEO, VTLS Inc.

Visionary Technology in Library Solutions

VALET – How it began?

VTLS was participating in Open Source activities for a long time

In 2004: started Institutional Repositories Project – VITAL.

Decided to build on existing infrastructure and architecture.

VTLS examined all the available open source software and selected Fedora as a base to build on.

VTLS established a partnership with Fedora.

ARROW had similar ideas in 2004 – selected Fedora and VTLS to create the IR infrastructure with a very unique partnership.

This presentation discusses the unique partnership; the unusual funding model and the results achieved.


Presentation Outline

Who are the players and why they joined the game?

VITAL … a VTLS proprietary software product

FEDORA … open source software project

ARROW … Government funded higher education consortia

How was the project assembled?

What results were achieved?

Relationships between open source and proprietary source

Where do we go from here?


Who is VTLS?

VTLS is rooted at Virginia Tech, Virginia’s largest University

VTLS is the first spin-off corporation from Virginia Tech

VTLS in business for over 20 years

VTLS has agents and offices in countries around the world. VTLS does business over 40 countries

VTLS is opening an office in the UK in 2006

National Library of Wales and Oxford University are Virtua and VITAL customers.


Corporate Headquarters

Blacksburg, Virginia USA


VTLS Products Main VTLS Other In Beta

TestProducts VTLS Products

Related to Virtua: 1. FRBR2. Union Catalogs & Consortium Databases


VVTLS TLS IInstitutional Repository nstitutional Repository


What is Fedora™?

Flexible Extensible Digital Object Repository Architecture


History of Fedora™

1997-Present– DARPA and NSF-funded research project at Cornell

(Conceptual framework developed by Sandra Payette and Carl Lagoze)

– Reference implementation developed at Cornell

1999-2001– University of Virginia digital library prototype (Thornton

Staples and Ross Wayland)

2002-PresentAndrew W. Mellon Foundation gave Virginia and Cornell a large grant to develop a production-quality Fedora system

Fedora was developed and released 2003 as Open Source under the Mozilla public license agreement and continues to be enhanced.


Fedora™ Advantage

Extensible digital object model Repository exposed by Web services APIs

Management (Creation, Deletion, Maintenance, Validation)

Access (Search, Disseminations)

Scalable, persistent storage for content and metadata

Content can be local and/or remote Content versioning Open source solution


Fedora™ Digital Object Architecture

Globally Unique Persistent ID

Public View: access methods for obtaining “disseminations” of digital object content Internal View: metadata necessary to manage the objectProtected View: content that makes up the “basis” of the object

Persistent ID (PID)

Disseminators

SystemMetadata

Datastreams

EAD, TEI, DC, MARC,VRA Core, MIX, etc.

Images, E-books, E-journals, Music, Video,

etc.


Create Complex Digital Objects

Digital Object

DCDC

EADEAD

Datastreams

Datastreams

Admin

Metadata

Admin

Metadata

EAD

EAD


Fedora™Repositor

y

E x ter n a lC o n ten tS o u r c e

E x ter n a lC o n ten tS o u r c e

HT

TP

E x ter n a l C o n ten tR etr iev er

X M L F ile s

Re la t io n a l D B

S e s s io n M a n a g e me n tU s e r A u th e n t ic a t io n

P o l icies

U s ers /G ro u p s

H T T P

F T P

D atas tr eam s

D ig ita l O b jec tsS to rag e S u b s ys te m

S e c u rityS u b s ys te m

W e b Se r vi c eE xpo s ur eL aye r

SO

AP

R em o teS er v ic e

L o c alS er v ic e

M an ag e A c c e s s S e arc h O A I P ro v id e r

M an ag e m e n tS u b s ys te m

A c c e s sS u b s ys te m

HT

TP

FT

P

H T T PH T T P S O A P H T T P S O A P H T T P S O A P

C lie n tA pplica t io n

B a tchPro g ra m

S e rv e rA pplica t io n

W e bB ro ws e r

Co mp o n e n t M g mt

O b je c t M g mt

O b je c t Va lid a t io n

P ID Ge n e ra t io n

O b je c t D is s e min a t io n

O b je c t Re fle c t io n

P o lic y En fo rc e me n t

P o lic y M g mt

Co n te n t

Web Service Web Service Exposure Exposure LayerLayer


Reasons to base VITAL on Fedora™

The “Flexible” and “Extensible” aspects: Flexible – The development can be in any

programming language because the API is based on Web services

Extensible – We can write workflow tools that facilitate different types of workflows

XML Submission and Storage – Digital objects are stored as XML-encoded files that conform to an extension of the METS schema

Focuses on Object Repository model and not how the repository will be used

Native OAI-PMH support Open Source – VTLS is active in the open source movement


Reasons to base VITAL on Fedora™

Extensible digital object model (architecture) allowing for complex objects

Repository exposed by Web services APIs Management (Creation, Deletion, Maintenance, Validation) Access (Search, Disseminations)

Scalable, persistent storage for content & metadata

Content can be local and/or remote

Content versioning There is a community developing other add-ons to

Fedora that will add value and services for adopters Low cost procurement for customers

Visionary Technology in Library SolutionsVital Manager

FEDORA

VITAL Fedora Relationship

Valet

Access Portal

Indexes

Web services

Batch Loading Tool


Workflow tools that simplify the creation, modification, submission and dissemination of digital objects

Web Service Interfaces (API’s) Management Service (API-M)

Ingest – XML-encoded object submission Create – interactive object creation via API request Maintain – interactive object modification via API requests Validate – application of integrity rules to objects Identify – generate unique object identifiers Security – authentication and access control Preserve – automatic content versioning and audit trail Export – XML-encoded object formats

How does VITAL work with Fedora™?


Fedora is conveniently located 2 ½ hours north of VTLS at the University of Virginia.

VTLS needed some basic services (that belonged in Fedora but were not available) and started working together with Fedora with the understanding that some parts of VTLS developed modules would become open source like Fedora.

Fedora agreed to let VTLS provide installation and support services for Fedora users that needed these services.

VTLS announced “VITAL”

Enter Fedora- VTLS partnership


Enter ARROW - What is ARROW?

Australian Research Repositories Online to the World

Project funded by the Australian Commonwealth Department of Education, Science and Training, under the Research Information Infrastructure Framework for Australian Higher Education. (Funding ends December 31, 2006)

Consortium of Monash University (lead institution), National Library of Australia, the University of New South Wales, and Swinburne University of Technology.

“The ARROW project will identify and test software or solutions to support best practice institutional digital repositories comprising e-prints, digital

theses and electronic publishing.”


ARROW’s view… Why partner with VTLS?

Partnering for success, support and survivability

ARROW needed to partner with a developer who could not only produce the software but could provide ongoing user support and development after December 31, 2006

Why VTLS Inc.? VTLS wanted to be a development partner Had begun work on a repository solution

already Familiar with library sector Willing to produce a combination of a

proprietary solution, Fedora and other Open Source software


ARROW’s view… Why Fedora?

ARROW’s primary requirementsa robust, well architected underlying platformSupport for a flexible object-oriented data modelPersistent identifiers down to the level of

individual datastreams, accommodating its compound content model

Ability to version both content and disseminators (think of software behaviors for content)

Clean and open exposure of APIs with well-documented SOAP/REST web services.


ARROW/VTLS - Project ARROW/VTLS - Project TimelineTimeline

June 2004 -- Partnership contract signed.

January 2005 -- Version 1.2 delivered.

April 2005 -- Version 1.3 delivered.

June 2005 -- Contract extension signed.

September 2005 -- VALET for ETD’s put into open source. ARROW consortia install Fedora/VITAL. Count = 4

October 2005 -- VTLS contributes more open source to FEDORA.

November 2005 -- Version 2.0 delivered.

March 2006 -- Four universities implement Fedora/VITAL. Count = 8

May 2006 --Version 2.1 delivered and another university implements Fedora/VITAL; Count = 9

June 2006 -- Five additional universities join ARROW; Count = 14


ARROW/VTLS – Funding ARROW/VTLS – Funding ModelModel

ARROW started as a “demonstration project” to prove feasibility of IRs

ARROW partnered with VTLS-VITAL and ended as a “operational IR”

VTLS developed software in close co-operation with Fedora & ARROW.

The funding model:

ARROW provided funding to VTLS against some deliverables

The VTLS development project was required to coordinate and synchronize with Fedora developments.

As a part of the funding agreement, VTLS would place certain modules in “open source” for all Fedora users whether they were VITAL users or not.

The initial ARROW consortium would get the VITAL/Fedora software; but would be required to pay for installation, training and support services

Additional members could “buy in” at advantageous prices for a predefined period


VITAL / Fedora Open Source(through VITAL Version 4.0)


VITAL/FEDORA Platform(s)VITAL/FEDORA Platform(s)

ServerLINUX or SOLARISDatabases

MySQL (recommended)McKoiOracle or Oracle XE

Client PCWindows 2000 or XPBrowser

IEFirefox


Key Features/Benefits to VITAL (1)Key Features/Benefits to VITAL (1)Some key benefits of VITAL include:

Storage and management of any content format, including rich-media, due to VITAL's repository object architecture

Integration with existing systems through open, standards-based protocols

Search full-text content of PDF, DOC, RTF and other document formats

Display high resolution imagery, multi-page documents and specialized data formats (MARC, EAD, TEI, etc.)

Automatically capture preservation metadata and create long-term, citable DOIs

Storage of content in native XML provides support/validation against any metadata schema

Support for aggregation of like-content in collections

Automatic metadata extraction and validation (facilitates preservation)


Key Features/Benefits to VITAL (2)Key Features/Benefits to VITAL (2)Additional benefits of VITAL include:

Web-crawler indexing and exposure (Google, etc.)

Language support and UNICODE compliance

Support for custom index creation for resource discovery

SRW/SRU Interface for exposure of repository content

Tracking content changes through versioning

Robust facilities for batch ingest of content

Support for link resolving against an institution’s OpenURL server

Support for dynamic creation/export of content citations (EndNote, text)

Facilities for web-based self-submission of diverse content types (VALET)

Integration with Fedora™ repository architecture allows for extensibility


VITAL Components

VITAL ManagerVITAL

AccessPortal

VITAL Access Administration

VITAL AccessIndexing Options

&Content Models

Oracle9i,McKoi,

or MySQLDatabase

VITAL Advanced Server Fedora™ Server

Apache Web Server Apache Tomcat Web Server

VALETSelf-Submission Tool


editor

Ingesting using VALET

submitter

VALETweb – form

captures objects one by one


Ingesting using VALET

Web-based formFor self service or assisted submissionCan be customised for your look and feelCurrently 7 basic ARROW defined content

modelsCan be extended to include more models

User specified review processesNumber of review stages is fully configurableCan specify different reviewers for different stages

e.g. metadata, copyright

–


VALET for ETDsVALET for ETDs

A subset of VALET that is supplied with VITAL. A complete standalone application. Bundled with FedoraTM Digital Object Repository Is a free, open-source solution for web self-

submission of Electronic Theses & Dissertations. A ‘best of breed’ solution for web submission of ETDs

that builds upon our collaborative experience with: NDLTD Project at Virginia Tech

ADT Program

ARROW Project


Where does VALET for ETDs fit?

VITAL Manager

VITAL AccessPortal

VITAL Access Administration

VITAL AccessIndexing Options and

Content Models

Oracle9i,McKoi,

or MySQLDatabase

VITAL Advanced Server Fedora™ Server

Apache Web Server Apache Tomcat Web Server

VALET for VALET for ETDsETDs

VALET

Self-Submission Tool


VALET for ETDs ( Examples)VALET for ETDs ( Examples)


Login ScreenLogin Screen


Deposit FormDeposit Form


Deposit Form (cont.)Deposit Form (cont.)


Field Field Requirements/ValidationRequirements/Validation


Customizable Help FilesCustomizable Help Files


PreviewPreview


Preview (cont.)Preview (cont.)


Upload FilesUpload Files

Select FilesSelect Files

Access AuthorizationAccess Authorization


Verify DepositVerify Deposit


ETD Deposit CompletedETD Deposit Completed


Reviewer LoginReviewer Login


Review StagesReview Stages


Review Stage 1Review Stage 1


View/Modify dataView/Modify data


Add Reviewer NoteAdd Reviewer Note


Approval ConfirmationApproval Confirmation


Review Stage 3Review Stage 3


Publish ConfirmationPublish Confirmation


ETD Publish CompleteETD Publish Complete


Expose -- the VITAL Access Portal

ARROWDiscovery Service

OAI-PMH

SRU/SRW

Google, etc


Always work closely with the “source” of your “open source”

VTLS staff/customers are active in Fedora community:

Carl Grant, VTLS President sits on Advisory Group

ARROW Technical Architect sits on Advisory Group

Joe Liversedge, VITAL Lead Developer sits on Development Group.

ARROW staff sit on Development Group

VTLS/ARROW coordinating development of content models.

VITAL builds on XACML features of FEDORA.

Work together to minimize overlap and maximize coordination.

VTLS contributes back open source components.

Observation (1)


“Your selected support vendor needs to be able to demonstrate influence points within the open source community. This influence is critical since the vendor needs to have the ability to fix problems in such a way that changes are incorporated into the primary

source..” •Martin, Fink, The Business and Economics of LINUX and Open Source, Prentice-Hall, 2003

Observation (2)


Build add-on software (like VITAL) only on released versions of the open source platform (like Fedora)

All software projects take longer than expected. Fedora does not have “paying customers” and so software development schedules have a greater tendency to slip.

When VTLS builds VITAL releases on anticipated Fedora features and capabilities, which for some reason get delayed, then VTLS has to write the capability OR wait till it becomes available. New policy: build only on released versions of Fedora.

Observation (3)


There is an important role for “for profit” businesses to participate in “open source” solutions to provide continuity, service, and value over time.

ARROW started with a “demonstration project” goal and

ended with a “production IR system” due to the partnership.

ARROW protected itself and its users by involving VTLS in the project.

Observation (4)


It is possible to run a project among partners spread all over the globe, as long as everyone commits to good communications.

Regular phone (Skype) meetings.

Lots of documentation – e-mails, project plans, concept papers.

Frequent travel.

Commitment to a common goal.

Accountability for all.

Observation (5)


VITAL Release 3 and 4 available in 2006 with added functionality (see later slide)

25 committed users

In the UK – National Library of Wales & Oxford University

Seven content models supported, more coming

Project with a life and a future

Current Status – June 2006


ARROW Beyond 2006

– Funding request for ARROW2• 2007• Supporting the RQF (Research Quality

Framework)• Creative development of institutional repositories• Supporting Australian engagement with

institutional repositories• Building partnerships to further enhance

repositories• Identifier Management Infrastructure for e-

Research Resources


VITAL - What are you buying?

A product that builds on top of the open source foundation, a product that provides:

VITAL Manager VITAL Portal VITAL Access Portal VALET - Web Self-Submission Tool VTLS Batch Loader Tool Handles Server (CNRI) Google Indexing and Exposure SRU / SRW Support

Most importantly: Service Support Sustainability !


VITAL - What are you buying?

Service:New versions containing:

Enhancements Bug fixes

UpgradesDocumentationTraining24x7 emergency supportEmail and telephone supportVINES - A community of users and experts


VITAL - What else are you buying?

You’re buying into a concept, a product and a vendor that: Believes that Open Source software helps to provide a lower cost

solution for libraries. Openly and visibly demonstrates support for the Open Source

movement and has become part of the Open Source community. Contributes back to the Fedora™ open source solution as a result

of our customer partnerships. Provides commercial support, training, documentation & hosting

services for open source solutions - assuring long term sustainability of the solution.

Works with other commercial entities to provide essential tools/services needed to make repository services successful at your institution.


VITAL – What is coming?VITAL – What is coming?

Upcoming VITAL Versions: Improve Authentication/Authorization Services.

XACML (Policy enforcement)

Create Additional & Enhanced Content Models.

Provide Usage and access statistics.

Improve User configurable interfaces.

Move towards Web Based Staff Client -- replacing the present windows based client.

Support for OAI sets.

Integrate 3rd party components like federated searches.


VITAL – What else is VITAL – What else is coming?coming?

VTLS is working on the following – Provide a method to link your VITAL repository to bio pages for

staff & faculty of your institution. Allow you to “prime” your repository by loading metadata for

works published by your staff and faculty. In some instances, depending upon licenses, allow loading of

published papers & works. Provide a connection to a citation formatting tool that supports

100’s of formats.


Thanks

To the conference organizers for inviting me here

To all of you for being here

Parting thought

“Go as far as you can see for

you will be able to see farther

when you get there”

Source unknown

VITAL – an Open Source/Proprietary Source Solution for Institutional Repositories built on a unique funding and partnership model. The JISC/CNI Meeting.

Documents

visionary technology

library solutions valet

library solutions reasons

library solutions workflow

open source movement

selected fedora

countries vtls

years vtls