9 25-12 DuraSpace Hot Topics, Slides, Introduction to Hydra

September 25, 2012 Hot Topics: DuraSpace Community Webinar Series

Hot Topics: The DuraSpace Community Webinar Series

Series Three: “Get a Head on Your Repository with

Hydra End-to-End Solutions”

Curated by Tom Cramer

Using the Webinar Platform

• 2-way audio for all participants is muted

• We’ll utilize the Chat Window for the Q&A portion or you may use it if you are having technical difficulties

• You may type your question here & hit ‘enter’

September 25, 2012 Hot Topics: DuraSpace Community Webinar Series

September 25, 2012 Hot Topics: DuraSpace Community Webinar Series

Webinar 1: Introduction to Hydra

Presented by: Tom Cramer,

Chief Technology Strategist at Stanford University Libraries

GET A HEAD ON YOUR REPOSITORY

Tom CramerChief Technology StrategistStanford University Libraries

What Is Hydra?

• A robust repository fronted by feature-rich, tailored applications and workflows (“heads”)

➭ One body, many heads• Collaboratively built “solution bundles” that

can be adapted and modified to suit local needs.

• A community of developers and adopters extending and enhancing the core

➭ If you want to go fast, go alone. If you want to go far, go together.

Fundamental Assumption #1

No single system can provide the full range of repository-based solutions for a given institution’s needs,

…yet sustainable solutions require a common repository infrastructure.

For Instance…

- Generally a single PDF

- Simple, prescribed workflow

- Streamlined UI for depositors, reviewers & readers

Digitization Workflow System

General Purpose Institutional Repository

Simple Complex

- Potentially hundreds of files type per object

- Complex, branching workflow

- Sophisticated operator (back office) interfaces

- Heterogeneous file types

- Simple to complex objects

- One- or two-step workflow

- General purpose user interfaces

ETD Deposit System

Hydra Heads: Emerging Solution Bundles

Institutional RepositoriesUniversity of Hull University of VirginiaPenn State University

ImagesNorthwestern University (Digital Image Library)

Hydra Heads: Emerging Solution Bundles

Archives & Special CollectionsStanford UniversityUniversity of VirginiaRock & Roll Hall of Fame

MediaIndiana UniversityNorthwestern University Rock & Roll Hall of FameWGBH

Hydra Heads: Emerging Solution Bundles

Workflow Management (Digitization, Preservation)Stanford UniversityUniversity of Illinois – Urbana-ChampagneNorthwestern University

ExhibitsNotre Dame

Hydra Heads: Emerging Solution Bundles

ETDsStanford UniversityUniversity of VirginiaEtc.

(Small) Dataeveryone…

Fundamental Assumption #2

No single institution can resource the development of a full range of solutions on its own,

…yet each needs the flexibility to tailor solutions to local demands and workflows.

Hydra Philosophy -- Community• An open architecture, with many

contributors to a common core

• Collaboratively built “solution bundles” that can be adapted and modified to suit local needs

• A community of developers and adopters extending and enhancing the core

• “If you want to go fast, go alone. If you want to go far, go together.”

One body, many heads

Community

• Conceived & executed as a collaborative, open source effort from the start

• Initially a joint development project between Stanford, Univ of Virginia, and Univ of Hull

• Close collaboration with DuraSpace / Partnership with MediaShelf, LLC

• Complementary strengths and expertise

Hydra Partners

0

2

4

6

8

10

12

OR09 OR10 OR11 OR12

OR = Open Repositories Conference

LibDevConX^3, Stanford, 2012: (Hydranauts and Fellow Travelers)

University of Virginia, 2008

Currently - DuraSpace

- Hull- MediaShelf

- Stanford- Virginia

Hydra Steering Group- small coordinating body

- collaborative roadmapping (tech & community)

- resource coordination- governance of the "tech core"

and Hydra Framework- community mtce. & growth

Hydra Partners- shape and direct work- commission "Heads"

- functional requirements & specs

- UI design & spec- Documentation

- Training- Data & content models

- "User groups"Founders- Duraspace

- Hull- Stanford

- UVa

Hydra Developers - define tech architecture

- code devleopment- integration & release

Committers

Contributors

Tech. Users

CommunityModel

Managing the Community

• Founding partners have an MoU governing how the community is managed• Subsequent partners have signed up to this MoU

through a partner agreement addendum

• Requirements of Partners• Use the software

• Contribute to the project

• Collaborate with other partners

• Commit to collectively advancing the project and the community

• Funding / payment is NOT required

Hydra Partners…

…are individuals, institutions, corporations or other groups that have committed to contributing to the Hydra community; they not only use the Hydra technical framework, but also add to it in at least one of many ways: code, analysis, design, support, funding, or other resources.

Hydra Partners collectively advance the project and the community for the benefit of all participants.

https://wiki.duraspace.org/display/hydra/Hydra+Community+Framework

Code Licensing

• All Hydra code is available under Apache License, Version 2.0

• All code commitments are being managed through Contributor License Agreements• Individual – so each developer is clear about

what they are contributing

• Corporate – so each institution is clear about what it is contributing

• Code contributors maintain ownership of their IP• And grant a non-exclusive license to the project

and its users

Sustainability

No animals were harmedin the making of this film.

Job Postings

If You Want To Go Fast… …go alone.…use Hydra?

• Notre Dame deployed a video cataloging head in 6 weeks, from scratch

• Ohloh.net stats (as of July 2012) • 16 regular contributors in last 12 months

(26 in total)• Top 10% of open source teams• ~8 person years of development

Hypatia Development – 8 week sprint

80/20 – 8 Weeks of Developmenthttps://github.com/projecthydra/hypatia/graphs/impact

Hydra-based Applications at Stanford

ETD’s – Electronic Theses & Dissertations

SALT – Self-Archiving Legacy Toolkit

EEMs – Everyday Electronic Materials

Argo – Repository Reporting and Management

Hypatia – Archives & Special Collections

Hydra Philosophy -- Technical

• Tailored applications and workflows for different content types, contexts and user interactions

• A common repository infrastructure• Flexible, atomistic data models• Modular, “Lego brick” services• Library of user interaction widgets• Easily skinned UI

One body, many heads

Shared, Primitive Functions

• Deposit

• Manage– Edit Objects– Set Access

• Search

• Browse

• Deliver

PlusAuthenticationAuthorizationWorkflow

Content Framework

• Key to enabling re-use of Hydra repository solutions is a common baseline to how objects are structured• Objects must include rights metadata

• Objects must include a statement of what content models the objects adhere to

• That’s it!

• The Hydra community has developed some basic building block content models (the Lego brick approach)• Combine and/or extend these to meet your

needs

Technical Framework - Components

• Fedora provides a durable repository layer to support object management and persistence

• Solr, provides fast access to indexed information

• Blacklight, a Ruby on Rails plugin that sits atop solr and provides faceted search & tailored views on objects

• Hydra Head, a Ruby on Rails plugin that provides create, update and delete actions against Fedora objects

Blacklight for Repositories

• Repository-agnostic, feature-rich, content-aware, turnkey access interface for repositories

• Aggregate content from multiple repositories, with links back to source systems

• Vibrant, multi-institutional, open source community on its own

• Can be used independently, or as the first component of, Hydra

CRUD in Repositories

Repository/Persistent Storage

Create/Submit/Edit(CUD)

Search/View(R)

CRUD in Repositories

Repository/Persistent Storage

Create/Submit/Edit(CUD)

Search/View(R)

Major Hydra Components

Fedora Solr

Solrizer

Blacklight(R)

hydra-headRails Plugin

(CUD)

Blacklight(Read Only)

A Note on Ruby on Rails• Rapid application development for web

applications: “Convention over configuration”

– 10x productivity

• Supportable: MVC (Model-View-Controller) and Rails framework make code well-structured, predictable

• Testable: Rspec and Cucumber give powerful, automatable, testing tools

• Learnable: Stanford went from 1 to 8 Ruby savvy developers in one year (no new hires)

– 1 week learning curve to basic proficiency

Philosophies

• Building a framework, not an application (variation is part of the plan)

• Opinionated software

• Invest time & resources into collaborative community (face time!)

• Trainings & workshops

• Openness, transparency (code, designs, discussions)

• Commit to contributing back to core

• Design for re-use

Best Practices in Development

• Agile, user-centric development process

• Test driven development & continuous integration

• Take a light touch when dealing with big topics: “working software wins”

• Distributed version control, github & public software repositories

• Rotating release managers for components

• Weekly “stand up” meeting w/ JIRA

• Daily chats in IRC

• Documentation

So What is Hydra?

• Framework for generating Fedora front-end applications w/ full CRUD functionality

• That follows design pattern with common componentry and platforms

– Fedora, Ruby on Rails, Solr, Blacklight

• That supports distinct UI’s, content types, workflows, and policies

So What is Hydra?

• And a growing community of institutions and developers committed to framework and collaboration

– Not grant-based

– Distributed

– Robust

– Open

Connect

• http://www.projecthydra.org

• Weekly developer calls: • Mondays 8:30 AM California time

• Email list: hydra-tech@googlegroups.com

• IRC: chat.freenode.net #projecthydra

• Quarterly Hydra Partner meetings

• Annual HydraCamp• Next Up: 8-12 October ’12, Penn State

Hydra Webinar #2

Case Studies in Repository Applications

Tuesday, October 16 @ 11 AM EDT• Richard Green, University of Hull & • Rick Johnson, University of Notre Dame

A Deep Dive into the Hydra Technology Stack

Tuesday, October 30 @ 11 AM EDTMatt Zumwalt, MediaShelf

Hydra Webinar #3

http://projecthydra.org

September 25, 2012 Hot Topics: DuraSpace Community Webinar Series

Questions?

Tom CramerStanford University

tcramer@stanford.edu