Managing Digital Storage

Managing Digital Storage

Sexto Seminario Internacional de Archivos Sonoros y AudiovisualesJUEVES 26 DE JUNIO DE 2014

MESA 6: El almacenamiento digitalRichard Wright

2006: EBU Tech Review

• "holographic media could provide a long-awaited solution for broadcast archives"

...

• "Archive customers require not only stable and long-lived media with a minimum 50-year archive life, but also require extremely high levels of equipment reliability and the same multi-generation backward read compatibility offered by tape. Random-access performance that allows rapid access to their data is another critical need."

2006: EBU Tech Review

�persistence – we want to get back what we put in;currency � – we want to be able to use what we get back.

What’s an archive?• The Archivists definition of Archive

• Where data lives: a repository, ready and waiting to be accessed.

• Where data lives: the archive is the application. The user of the archive expects to get to data within seconds, just like for any other application. However, the user of an audiovisual archive may be willing to wait longer for delivery of full-quality video and audio.

• An archive is the place where people go to get archive content.

• An archive is the application.

Archive Requirements

• Archive service requirements– what archives do for others;

• Archive functional storage requirements� – what archives want from storage;

• �Storage service requirements– technical requirements (size, speed, uptime ...);

• �Storage media requirements– how storage operates.

Archive service requirements

• Acquisition• Documentation�• Access• Re-Use�• Asset management, life-cycle management

Digital Archives solve:

• contention for the one tape that everybody wants at the same time;

• circulation control;�• chasing returns;�• making extra copies;�• getting material to distant places, quickly.�

Archive functionalstorage requirements

• Persistence– the ability to get content out of storage;

• Currency� – the ability to use that content.

Storage Service requirements

• Size of the storage• Bandwidth of the access to the storage�• Reliability:

– Of the service (uptime)– Of the storage (failures)

Storage Media requirements

• Not interested !• (which is why I wrote the paper, because I

didn’t think holographic storage was ‘a long-awaited solution for broadcast archives’

[holographic storage never became a commercial product anyway]

2006: ancient history?

Digitising Contemporary ArtD6.2 Best practices for a digital storage

infrastructure for the long-term preservation of digital files Henriksen, Seuskens, Wijers (LIMA) May, 2013

http://www.dca-project.eu/images/uploads/banners/DCA_D62_Best_practices_for_a_digital_storage_infrastructure_20130506_Version1.pdf

What are the choices?

• 3. Data tapes ................................................................................................... 9• 3.1 LTO (Linear Tape Open) .............................................................................. 9• 3.2 Autoloaders / tape libraries ...................................................................... 11• 3.3 Preservation issues .................................................................................... 12• 4. Hard disk drives ........................................................................................... 14• 4.1 Connecting an external HDD ...................................................................... 14• 4.2 SCSI and SATA technology recommended .................................................. 15• 4.3 Server systems ............................................................................................ 15• 4.4 Preservation issues ..................................................................................... 17• 5. Solid State Drives ........................................................................................... 19• 5.1 Single-Level Cell Flash recommended ......................................................... 19• 5.2 Preservation issues ...................................................................................... 20• 6. Optical disks ................................................................................................... 21• 6.1 Blu-ray .......................................................................................................... 21• 6.2 Preservation issues ...................................................................................... 21• 7. Cloud storage ..

What are the problems?

Maintenance

• Multiple copies• Fixity• Checking the files are correct• Checking the files are usable• Migration

PrestoCentre models

• For file checking• For file migration

Migration of File Formats

Is the fo rm at a p rob lem?

S TA R T H E R E

Arch ive fo r a few years

W hat cost/qua lity/risk op tion can you a ffo rdC om press

lossy

Y E SN O

U ncom pressC om press lossless

E N D H E R E

(1 )

(2 )

(3 ) (4)

(5a)(5b)

(5c)

Migration with Emulation

Is th e fo rm a t a t risk?

START HERE

A rch ive fo r a fe w ye a rs

W h a t co s t/q u a lity/risk ca n

yo u a ffo rd?C o m p re ss

lo ssy

YESNO

U n co m p re ssC o m p re ss

lo ss le ss

END HEREMultivalent