Franziska Frey 2 / DHV13

10/7/13

1

Preservation Strategies for Digital Image Collections

Franziska Frey"Malloy Rabinowitz Preservation Librarian"Head of Preservation and Digital Imaging Services"Harvard Library

Requirements To Make Digital Work

•  Deep and longstanding institutional commitment to traditional preservation

•  Full integration of technology into information management procedures and processes

•  Significant leadership in developing appropriate definitions and standards

Responsibility •  “Digital preservation will only happen if

organisations and individuals accept responsibility for it.

•  Acceptance of responsibility should be explicitly and responsibly declared…..”""UNESCO, “Guidelines for the Preservation of Digital Heritage,” 2003

Control •  Move objects to a safe place •  Uniquely identify and describe images with

appropriate metadata for resource discovery, management, and preservation

•  Use standardised metadata schemas for interoperability

•  Ensure that links between digital objects and their metadata are securely maintained, and that the metadata are also preserved.""UNESCO, “Guidelines for the Preservation of Digital Heritage,” 2003

Stewardship •  Long-term management of heritage materials

(digital objects) through collaboration, throughout all phases of object life cycle. – Rights holders – Collection managers – Repository/preservation staff – Centers of expertise (researchers, scientists) – Auditors – Content users and their communities

10/7/13

2

Stewardship ─ Collection"Manager’s Responsibilities

•  Intellectual property rights: manage legal rights, including rights to make copies

•  Metadata: provide appropriate administrative, technical, and structural metadata for objects

•  Discovery: ensure that description of objects are publicly available in online discovery systems

•  Access: ensure that a version of the object is available to the Harvard community

•  Financial considerations: pay for repository and preservation services"" Harvard University Library, DRS Policy Guide

Techniques to Preserve Images

•  Phase 1―Production •  Phase 2―Appraisal •  Phase 3―Deposit •  Phase 4―Archiving and Preservation •  Phase 5―Discovery and Delivery

Phase 1 ― Production" •  Imaging does matter •  Formats do matter •  Documentation does matter

Art-si.org

Image Quality Matters •  High quality images can be repurposed and are

worth maintaining •  Steve Puglia: ”We feel that the managed

environment needs to be extended beyond the digital repository and forwarded in time to include the digitization process….”"(IS&T Archiving Conference, 2008)

Building Teams •  Preserving visual cultural heritage

materials involves one additional field: Imaging Science –  It is imperative that the person

involved in creating these materials, whether born digital or digitized, has a good knowledge of imaging

10/7/13

3

Consequences of These Decisions"Vis-à-vis Preservation

•  Resolution – As size increases (e.g., decisions to

capture and keep 48 bit, high resolution files), management overhead increases • This holds true especially if the storage

unit bills per MB or GB per year

How are Digital Libraries Evaluated?

•  Almost no research on implications of image quality

•  User interfaces and usability in terms of finding the right image have been evaluated

•  Why this gap? –  Do users know what they can demand in terms of

image quality? •  Visual literacy

–  Image quality studies are complex and expensive

Reproductions of Cultural Heritage Materials Needed for…

•  On-line databases •  Posters, calendars, and

postcards •  Exhibition catalogues •  Education •  Conservation •  And more

Survey—Imaging Purposes

To protect vulnerable originals from use 67% To produce printed reproductions 77% To make collection accessible over the Internet 86% To include in a collection management system 86% To document conservation treatment 58% Other 28%

10/7/13

4

However… •  Reproducing cultural heritage materials can be

difficult –  Color and texture –  Printing may be taking place half a world away

•  It is of interest to limit the number of times an artwork is imaged –  Potential for damage to the artwork –  Expensive

•  Resources are limited –  Budget cuts –  Many institutions do not have dedicated reproduction

departments

Viewing Conditions •  Reproductions are viewed under various

lighting conditions… – Museum shop, living room, class room – Displays

•  …even for image evaluation – Light booth, gallery, office

•  Significant issues – Metamerism – Color appearance – Consistency

Project Objectives •  Determine the optimal reproduction processes

presently available –  Understand the workflow processes in use in

cultural heritage institutions today –  Determine the image quality inherent in these

processes in print and on line –  Understand the image quality expectations of the

users involved •  Develop a framework to serve as a guideline

for cultural heritage institutions to follow when reproducing fine art

ImageMuse •  Establish a user group devoted to imaging,

archiving, and reproducing cultural heritage •  17+1 institutions took part in our experiments

Image Quality Metrics •  Document current workflows •  Develop a practical characterization test

method: industry solutions •  Document available targets to measure

objective image quality

Workflow Charts Capture

Illumination Camera

Post-Processing

Proofing

Further Processes

10/7/13

5

Documented Reproduction Workflows Workflow Process General Function

Specific Workflow Process Steps and Considerations

Additional Steps and Considerations

1. Image capture Objective targets used Lighting set up used to illuminate the artwork including polarization

Camera calibration Flat-fielding

2. Proofing and image file preparation

Monitor Calibration Working color space Screen background used for file viewing Viewing environment Physical image size on the screen

Sharpening Image orientation Resolution and file size

3. Image delivery File format Image layers for documentation of image processing conducted ICC color management

Delivery media Guide prints and proofs

4. Image archiving Archiving protocol Proper handling and storage of guide prints Metadata

Image naming

Hidden-Target Paintings

Gamblin Artist’s Oil Colors

Comparison of Corrected Paintings CS1 CS2 CS3 CS4

Universal Test Target

Results from UTT Cameras—Color Performance

10/7/13

6

Objective Targets •  Input targets—output targets

Experimentation •  Define quality criteria based on objective

and subjective metrics •  Develop a method to connect objective,

measurable image quality to subjective image quality as perceived by the observers

•  Benchmark current quality

Subjective Targets

Press Sheets

Experimental Methodology •  Emphasis on the perceptual image quality

of printed reproduction and on display – Objective targets measured as well

•  Evaluation performed using a variety of pictorial “targets” – Sent to a variety of cultural heritage institutions

for them to put through their imaging processes

10/7/13

7

Images Printed at RIT’s Print Applications Laboratory

•  Heidelberg Speedmaster sheet-fed press –  ISO 12647 – Visual match to guide prints – NewPage Sterling 80# Gloss Text

•  HP Indigo Digital Press

Perceptual Testing •  Observers experienced with fine art

reproduction –  Fine art photographers –  Curators –  Art historians –  Conservators –  Librarians –  RIT students & staff

Experiments Conducted •  The Impact Of Lighting On Perceived Quality Of

Fine Art Reproductions •  Evaluating CATs as Predictors of Observer

Adjustments in Softcopy Fine Art Reproduction •  Comparing Hardcopy and Softcopy Results In the

Study of the Impact of Workflow on Perceived Reproduction Quality of Fine Art Image

•  Evaluating Digital Printing for Fine Art Reproduction

•  Fine Art Reproduction Workflows for the Web Environment

Objective Targets

Experimental Methodology •  17 institutions participated •  30 hard-copy renditions of each of image

were included – 19 prints made ‘to the numbers’ – 11 visual matches made to guide prints

•  All prints made on NewPage Sterling Ultra 80# Matte Text paper

•  16 soft-copy renditions used •  Variety of cameras and color spaces

10/7/13

8

Psychophysical Testing •  Hard copy experiments followed rank order

protocol –  Observers ordered the prints from best to worst

reproduction or representation of the original –  Most to least preferred rendition

•  Soft copy experiments followed paired comparison protocol –  Best reproduction or representation of the original –  Most preferred rendition

Soft-copy set up

Hard-copy set up

Experimental Setups

Key Findings •  Results with and without the original present are more

consistent for hard-copy prints than soft-copy images •  Hard-copy results are more consistent with soft-copy

results when the original is present –  Original is typically not present when users are viewing fine art

reproductions •  Observers did not like lower contrast images when they

were electronically displayed •  Of interest to identify workflows that provide both

acceptable representations of the originals as well as pleasing images on screen and in print

Color Difference (ΔEab) at Capture

Lightness Difference (ΔL) versus"Perceptual Quality Rating (Z-score)

R2 = 0.8144

0

2

4

6

8

10

12

-1 -0.5 0 0.5 1

Mean Z-scores

Mea

n D

elta

L

Mean Delta LForm 5Form 11Form 13Form 19Linear (Mean Delta L)

Experiments Conducted •  The Impact Of Lighting On Perceived Quality Of Fine Art

Reproductions •  Evaluating CATs as Predictors of Observer Adjustments in

Softcopy Fine Art Reproduction •  Comparing Hardcopy and Softcopy Results In the Study of

the Impact of Workflow on Perceived Reproduction Quality of Fine Art Image

•  Evaluating Digital Printing for Fine Art Reproduction •  Fine Art Reproduction Workflows for the Web

Environment

10/7/13

9

Color Management Check Web Experiment User Interface

Key Areas of Interest Key Areas for Photographers versus Other Occupations

Key Findings •  Testing conditions had a limited impact

on the preference judgments for these images

•  Ranking results for the experiments conducted in the lab without the original and via the web were highly correlated, indicating that, when the original is not included, a web-based test may be a reasonable approach

Key Experimental Findings •  Camera make, lights, file format did not influence our

results –  Everybody is using equipment uniformly capable of doing this job

•  Lighting conditions may have a strong impact on image appearance –  Proofing protocols will have to be revisited

•  The use of a target to ensure proper capture setup is recommended

•  Main goal: get the tone scale right at capture •  Following standardized workflows, ISO printing standards

and viewing standards reduces need for manual post processing

10/7/13

10

Key Findings−Interviews •  Define imaging goals and talk to your users

– This will help set expectations •  Acceptability varies for the different stakeholders –

this needs to be clearly communicated •  Document workflows in detail

–  No undocumented processing should be performed along the image interchange cycle

–  The more often a file is touched the worse the results •  Close the communication loop in the image

interchange cycle

Art Image Interchange Cycle Photographers

Paper Manufacturers

Conservators

Publication Staff Curators

Printers

Visitors

Imaging Scientists

Standards Experts

Graphic Designers

Managers of Imaging Studios

Art Historians

DAM staff

Equipment Manufacturers

Digital Imaging Specialists

Licensing Staff Exhibitions

Editorial

Pre-press Merchandising

Librarians

Metatorial

Future Work •  Standardization even more important with

globalized workflows –  ISO JWG 26: combining existing guidelines and

standards for quality evaluation of imaging systems –  Training for implementation of standards needed –  Define stepping stones to get to a standardized

workflow

•  Bring all threads of imaging in an institution under “one roof”

Techniques to Preserve Images

•  Phase 1―Production •  Phase 2―Appraisal •  Phase 3―Deposit •  Phase 4―Archiving and Preservation •  Phase 5―Discovery and Delivery

Phase 2 ― Appraisal •  Deciding what is essential

– Characteristics that give object meaning, integrity, authenticity

•  Encode what is essential – Metadata production

•  Validating objects – Are they what they seem to be?

Checksums

Metadata •  Descriptive

–  You cannot preserve what you do not know you have –  You cannot sustain use for items that cannot be

identified •  Structural

–  Encoding of relationships facilitates management, use •  Administrative

–  Ownership, rights of access, provenance •  Technical/Preservation

–  Format attributes –  Documentation of significant properties and

preservation intentions to inform preservation strategies

10/7/13

11

www.loc.gov/standards/premis/"v2/premis-2-0.pdf Metadata Containers

•  Directory and file names •  File headers •  XML

– XMP (e.g., within JP2), EXIF, NISO MIX, METS

•  Database tables •  Printed reports

NISO Phase 3 ― Deposit •  Choosing a repository: build or buy? •  Packaging data for deposit •  Validating data and objects

Phase 4 ― Archiving and Preservation�

•  Repositories •  Standards and guidelines

Storage Options •  Interim storage

– Digital asset management system – Store data off-line on magnetic or

optical media •  Repository storage

– Build a repository – Pay annual fee to use an external

repository

10/7/13

12

Interim Storage─The Bare Essentials •  Assign checksums to images early in the production

process •  Document rationale for creating images

–  At very least, include read me file on storage media; database is best

•  Avoid use of “meaningful” filenames •  Use new media

–  Follow advice/recommendations of IT9.21 and IT9.23 standards

•  Create duplicates and store duplicates in separate locations

•  Create explicit “links” between catalog records and images

•  Assign preservation responsibility to appropriate entity

Preservation Repository

•  Long term storage strategy for masters – Preservation responsibilities delegated to

service provider – OAIS – Accountable, auditable and fiscally

sustainable

Managing Risk •  Security and access control

– Preventing unauthorized use, tampering or theft

– Protecting rights holders •  Data obsolescence

– Media incompatible with players – Formats

•  Functional obsolescence – Formats incompatible with user needs

•  Fiscal obsolescence

Phase 5 ― Discovery and Delivery •  Digital library infrastructure

– Catalog or other database for descriptive information

– Persistent naming – Access management

•  Rendering – Hardware, web browsers – Emulator

Pricing Components"

•  Various pricing models – Subscription (JSTOR) – Storage (Harvard Digital Repository

Service) – Accession, subscription and storage

(OCLC Digital Archive)

Managing Costs •  Minimize number of conservation and

reformatting interventions over entire life-cycle

•  Manage the storage environment – “Geography is preservation destiny”

•  Negotiate costs of outsourced services, e.g., through consortia

10/7/13

13

Summary •  Digitization is not preservation •  Storage is not synonymous with digital

preservation, and storage is neither free nor cheap •  Stewardship and digital preservation require active

oversight of content, technologies, and user expectations

•  Preservation planning depends and relies upon extensive, well-managed metadata

•  Distributed, but shared expertise centers and tools will be essential to managing costs

Acknowledgements

The Andrew W. Mellon Foundation Participating Institutions Susan Farnand, RIT Observers Steven Chapman, Harvard University