My Media at University of Toronto Libraries

http://mymedia.library.utoronto.casian.meikle@utoronto.ca

gordon.belray@utoronto.cabilal.khalid@utoronto.ca

graham.stewart@utoronto.ca

Sian MeikleDigital Services LibrarianInformation Technology ServicesUniversity of Toronto Librariessian.meikle@utoronto.ca

Overview• How the media server came to be

• How to use the media server

• How the media server was built

• Rapid application deployment: media server as a case study

• What’s next

Why a media server?• Give university a home for media

storage

• Provide tools for publishing media to web pages easily

• Integrate media resources into discovery tools such as library’s discovery layer, university search engine

Why a media server? File format Creators can make video in many formats

13 very common formats .3g2 .3gp .asf .asx .avi .flv .mov .mp4 .mpg .rm .swf .vob .wmv23 common formats .diva .dream .dvr-ms .f4v .fbr .hdmov .m4v .mkv .moi .mpeg .mts .mxf .

ogm .rcproject .rmvb .scm .smil .srt .stx .ts .vro .wtv .xvid 172 average or less common file formats .3gp2 .3gpp .3mm .aep .ajp .amv .amx .arf .avb .avs .avs .axm .bik .bin .bix .box .bsf .byu .camrec .cvc .d2v .d3v

.dat .dce .dir .dmb .dpg .dv .dvx .dxr .evo .eye .fbz .fcp .flc .flh .fli .flx .gl .grasp .hkm .ifo .imovieproj .imovieproject .ivf .ivr .ivs .izz .izzy .lsf .m1pg .m21 .m21 .m2t .m2ts .m2v .m4u .mgv .mj2 .mjp .mnv .moov .mp21 .mp21 .mpv .mqv .msdvd .msh .mswmm .mtv .mvb .mvc .mvd .ncor .nsv .nvc .ogv .par .pds .piv .playlist .pmf .prel .pro .prproj .pxv .qtch .qtl .qtz .rdb .rec .roq .rp .rts .rts .rum .rv .sbk .scn .sfvidcap .smi .smk .spl .ssm .svi .swi .swt .tda3mt .tivo .tod .tp .tpr .trp .vc1 .vcv .vdo .veg .vf .vgz .vid .viewlet .viv .vivo .vlab .vp6 .vp7 .w32 .wcp .wm .wmd .wmmp .wmx .wp3 .wvx .yuv .zm1 .zm2 .zm3 .zmv .dsy .gvi .m4e .mmv .mod .mpf .mpg2 .mpv2 .scm .60d .dav .ddat .dif .gvp .iva .lsx .m1v .m2a .meta .mjpg .modd .movie .mp2v .mp4v .mpe .pva .qt .sbt .smv .str .vem .vfw

MyMedia converts media to just 3 formats for users

flashmp4mp3

Why a media server? File size.

Text: 5 MB / 1000 pgs (5 kb / pg)

Sound: 460 MB / hour (128 kb / s)

Video: 5 GB / hour ( 1.4 Mb / s)

Why a media server NOW?

• Exploratory work since early 2008• Commercial options megalithic, expensive

• August 2009: pandemic preparedness• Media server built and deployed in 4 weeks

Media creator

1. Upload mediaa) Got media? Upload it to MyMedia b) Need to make media? Use a tool like Camtasia Relay to capture video. Camtasia Relay saves your media to the media server.

2. Describe media Fill in a web form: i) rights: UofT / world ii) play type: stream / downloadiii) metadata:keywords title description

3. Publish mediai) Embed it in your web pageii) Link to it on media server

2 Describemedia

3Publish media

UofT Portal (Blackboard)University web sitesAny web page

Upload media1

LibraryMedia Server

(eg Camtasia Relay)

Capture media

How people use the media server• Instruction

– Lecture capture– 3rd party content– How-to:

Library, Blackboard, …• Scholarship

– Conferences, invited lectures • University life

– Video contests

• NOT live web casting Stream

only64%

Download / stream

36%

UofT only62%

Public access

38%

Gordon BelraySystems and Interface DesignerInformation Technology ServicesUniversity of Toronto Librariesgordon.belray@utoronto.ca

• Pandemic planning required rapid integration with Camtasia Relay server

• https://eclasscam1.erin.utoronto.ca/Relay

• What is streaming?• Checks the connection speed of the user and buffers a small

portion of the video using a Flash player• Uses RTMP (real time messaging protocol) on a Flash Media

Server • File formats: Flash (.flv) MPEG4 (.mp4) encoded with H.264

codec, and MP3s• Progressive Downloads• Media must be downloaded to the end user’s browser• Cannot scrub beyond point of download

• Demo

• Login using UTORid

• Home page showing processed and unprocessed media

• Media management page

• Example of published media using the myMedia links

• Permanent link to play media

Bilal KhalidSenior Application Developer,Information Technology ServicesUniversity of Toronto bilal.khalid@utoronto.ca

Software Components

• Database Schema (MySQL 5)• Monitoring script for Camtasia (perl)• MyMedia administrative interface

(Java/WebObjects)• Video encoding using ffmpeg• Media viewing interface (PHP)

filedump

t_user

t_media

Database Schema• Temporary staging area for files uploaded through Camtasia• Populated by perl script

• Stores user information• Populated using LDAP feed, after WebLogin

• Maintains media information• Populated by user input/video file analysis

Note: A transition to a more comprehensive schema is planned for the next phase of development.

Camtasia Monitor• Camtasia has been setup to

transfer uploaded user videos to our server

• Perl script detects any newly transferred files, creates entry in filedump table

• Media file is renamed using randomized string token

• Accompanying xml file contains file information (e.g. duration, resolution etc.)

• Email sent to user, pointing them to MyMedia application

Comic Source: http://www.agileali.com/myblog/2009/10/02/script-to-see-if-perl-is-working-e/

MyMedia Application

• http://mymedia.library.utoronto.ca• Media administration• Developed in Java/WebObjects• Source can be Camtasia OR direct user

uploads• WebLogin-based authentication, followed by

user profile creation• Media metadata management (e.g. title,

description, keywords etc.)

Media Access Options

• Four profiles offered, analogous to server-side directories with permissions– uoft + stream-only– uoft + download/stream– public + stream-only– public + download/stream

User Uploads/Media Encoding• Alternative to Camtasia• Incoming files encoded using ffmpeg (http://www.

ffmpeg.org/)

• Many supported file formats (e.g. .avi, .mov, .m4v, .m4a .wmv)

"/opt/local/bin/ffmpeg -i /upload-dir/incoming_file -acodec libfaac -ab 128k -ar 44100 -r 20 -vcodec libx264 -b 256000 -s 800x600 -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -me_method umh -subq 5 -trellis 1 -refs 2 -bf 1 -coder 1 -me_range 16 -g 300 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -bt 256000 -maxrate 4M -bufsize 4M -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -level 21 /encoded-output-dir/output_file.mp4";

Media Viewing• http://media.library.utoronto.ca• Written in PHP• WebLogin layer activated

depending on access permissions of media

• Basic metadata displayed on page below media

• LongTail Video - open source player used for media display (http://www.longtailvideo.com/)

Scenario 1: Camtasia Recording

1. User generates video via local Camtasia Relay Client

2. Media uploaded to Camtasia Server, encoded into mp4

3. Media transferred to stream.library Server via SMB mount

4. Perl script detects file, creates entry in ‘filedumpᾼ

5. Email with link to MyMedia sent to user

7. User edits Media info (stored in ‘t_mediaᾼ)

6. User logs onto MyMedia (via WebLogin)

6b. Profile creation for first-time login (stored in ‘t_userᾼ)

8. Media made viewable at media.library

Scenario 2: Direct File Upload

1. User obtains Media by own means

4. User edits Media info (stored in ‘t_mediaᾼ)

2. User logs onto MyMedia (via WebLogin)

2b. Profile creation for first-time login (stored in ‘t_userᾼ)

3. User uploads Media to Server

5. Media encoded using ffmpeg

6. Media made viewable at media.library

Graham StewartNetwork and Storage Services Manager, Information Technology ServicesUniversity of Toronto graham.stewart@utoronto.ca

MyMedia hardware

• Servers: Apple XServe / Dell PowerEdge

• OS: Mac OS X / Redhat Linux• Storage: Pillar Data Systems SAN• Network: gigabit Ethernet from MyMedia

servers to campus backbone and gateway.

Modular service design• 6 services: front end, upload, conversion, stream, download,

database.• Currently running on 3 servers: can expand to 6 or more as

demand increases.• Target for Fall 2010: file conversion on separate server.• Target for January 2011: full virtualization of most services

allowing for quicker resource allocation. We do virtualization on Ubuntu Linux using KVM/QEMU.

MyMedia components

stream.library

MySQL

Storage Media Conversion

UTORauth

MyMedia

media.library

User: HTTP, HTTPS

Inter-server: iSCSI, NFS, MySQL, SMB

Open Source vs. Proprietary

Rapid application deployment• Sudden requirement• Staff pulled away from other projects• Intensive 4 week schedule• Several versions / iterations• Modular application development and testing• Staff flexibility• Management support

What’s next? • Deepen metadata:

– Preservation metadata (PREMIS)– Rights metadata: more granular– Descriptive metadata (CanCore, IEEE LOM)

• Integrate content:– Build feeds for public content to university search

engines– Build embed tools for selected targets

(FADIS, Blackboard)

• More tools for content creators:– Batch upload– File conversion tailored to purpose

Questions?Web address:

http://mymedia.library.utoronto.ca

Media server support:mymedia@library.utoronto.ca

Camtasia video production support:camtasia.support@utoronto.ca

We welcome your input, suggestions and videos!