Www.apsr.edu.au 1 Cool mashups and new features with the latest Fez+Fedora media streaming, Solr powered search, Facebook integration Open Repositories.

www.apsr.edu.au

1

Cool mashups and new features with the latest Fez+Fedora

media streaming, Solr powered search, Facebook integration

Open Repositories 2008, Southampton

Christiaan Kortekaas

Open Sourcerer, Lead programmer, Fez and eScholarshipUQ Testbed

Australian Partnership for Sustainable Repositories

www.apsr.edu.au

2

The Australian Partnership for Sustainable Repositories

• Funded by the Australian Federal Government’s Department of Education, Science, and Training over 2004-2007 to establish a centre of excellence in digital collections management.

• APSR is a coordinated set of programs and projects that address strategic issues of digital sustainability, eResearch facilitation, repository management and system development.

www.apsr.edu.au

3

Left unsaid

• What Fez is• Why we developed it• Detailed technology• Application structure• Workflow system• Content Model Metadata Editing

system• Preservation services• ACML security system & Shibboleth SP

Search for “fez” on UQ eSpace

www.apsr.edu.au

4

On the menu

• Fez as UQ eSpace• Repository action verbs and

mashups• Fez developer community

contributions• Fez Roadmap

www.apsr.edu.au

5

UQ is using Fez for…

• Central research authority service• Theses management system (see

diagram)• Video, Audio streaming services• Image collections• HERDC• Research Assessment (RAE)• Datasets (possibly)

www.apsr.edu.au

6

UQ eSpace

• 1800+ Users have logged in• One main content administrator• Most content currently coming via

EndNote exported xml• For a large amount of academic work

metadata already exists, so only fulltext needs to be added (lowers the barrier to submission)

• Library working on getting all metadata only records to also have fulltext

www.apsr.edu.au

7

Innovative Actions in Fez

• Import• Deduplicate• Export & Feed• Delete, Undelete & Tombstone• Stream & Embed• Customise theme per

Community/Collection

www.apsr.edu.au

8

Import & Ingest

• ePrints ~5000 • Research Master ~45,000• Thompson ISI ~70,000• Endnote ~10,000

• New workflow for network ingest for bringing in very large files (greater than HTTP can handle)

www.apsr.edu.au

9

www.apsr.edu.au

10

De-duplicate

• 130,000 records de-duplicate down to 65,000

• ‘Fuzzy logic’ de-dupe workflow generates a report with GUI

• Many objects auto-de-duped and merged based on strong identifiers (ISI_LOC, ISSN, ISBN etc)

• Existing objects get merged with data from their duplicates, slave objects get ‘tombstoned’

www.apsr.edu.au

11

www.apsr.edu.au

12

www.apsr.edu.au

13

Export

• Business Objects/Crystal Reports SQL feed for central ah-hoc research report generation by Research Office

• Exported by Fedora in METS, converted to APSR METS Profile by the APSR RIFF Submission service

• CSV, Excel• EndNote soon

www.apsr.edu.au

14

Delete & Tombstone

• Different to purge – removes from Fez indexes and sets Fedora object status to ‘Deleted’

www.apsr.edu.au

15

Stream & Embed

• Automatic conversion of any uploaded or ‘network ingested’ video content into flash video (like YouTube)

• Uses a free open source streaming flash video player (looks like YouTube)

• UQ Library video competition winner watched over 1000 times

• Can be embedded into external websites (like facebook)

• Library man Embedded Library Man

www.apsr.edu.au

16

Custom Themes

• Using Fez Theme engine or Fez Manakin Emulation with Chameleon support

www.apsr.edu.au

17

Recent Community Contributions

• Solr Integration – Kai Jauslin @ ETH Zurich

• Origami Image Viewer Integration – Keith Maull @ Colorado Alliance of Research Libraries

• Fedora versioning viewing, Repository Bridge Application, DROID integration – Matt Painter @ Archives New Zealand

• User commenting on object – Jonathon Harker @ Catalyst New Zealand

www.apsr.edu.au

18

Integration

• Full featured modular search service Faceting, highlighting, rich API

• Excellent performance, with fast on-the-fly updates (uses lucene engine)

• Complete integration with the Fez authorisation index (FezACML)

• Direct easy mapping of search keys defined in the Fez GUI to ‘dynamic’ fields in the Solr index schema

• Fulltext indexing of PDF

www.apsr.edu.au

19

Future of Fez

• Community Driven• Fez 2 final release to include at

least Solr integration, other community contributed modules

• Simplification (One shell script to install all of Fedora and Fez in ONE command)

• Hosting Services - CARE Affiliates• Orbeon XFORMS optional module

www.apsr.edu.au

20

New Fedora Storage Layer

• What do you do if you need to ‘fix’ 50,000 objects?

• XQUERY and XQUERYUPDATE• Fedora needs ACID transactions

across a set of API transactions• One answer: use Oracle Berkeley

DBXML or equivalent• Who?

www.apsr.edu.au

21

Further Resources

• Fez Wiki http://dev-repo.library.uq.edu.au/

wiki

• Fez Sourceforge site: http://sourceforge.net/projects/fe

z Forums, User and Developer

mailing lists, Download page

• Fez Eventum (like bugzilla) – opening to the public soon