From EDRMS to TDR: A Wish-list for Ways to Preserve Digital Records

@ulcc www.ulcc.ac.uk

IRMS Conference May 2015

From EDRMS to TDR: a wish-list

for ways to preserve digital

records


About me

• Ed Pinsent, Digital Archivist at ULCC since 2004

• Teaches digital preservation on the DPTP

• Background as archivist / records manager

• Experience in web-archiving, repository management, metadata projects, migration, digitisation, project management, etc.

• See more at digital archives blog http://dablog.ulcc.ac.uk/

http://dablog.ulcc.ac.uk/


Imagineering: Themes & Dreams for Tomorrow

• Do we need to preserve born-digital records?

• If so, how can we do it?

• Will refer to EDRMS (Electronic Document

Records Management Systems)…

• And to a TDR (Trusted Digital Repository)…

• That conforms with Open Archival Information

System (OAIS) Reference Model


Some issues

1. Need for keeping records may exceed life of the EDRMS

2. Need for keeping records may exceed life of file formats they are stored in

3. Can we get content / metadata out of the EDRMS?

4. If we need preservation, where do we do it?


#1: The EDRMS problem

• You may have a long-term business need

for retention

• May only affect a small percentage of

records (e.g. Pension records, contracts)

• Will your EDRMS last that long?


#2: The file format problem

• Digital file formats may become obsolete / unsupported / unreadable

• Common response in digital preservation is to migrate

• But it’s extra work: picking a suitable target format, finding a conversion tool

• Doing this creates even more metadata

• We’d want to automate it, ideally


#3: Can we export from EDRMS?

• Can we get the digital objects / files out?

• Can we get the metadata out?

• Do we want to?

– Presumably MOREQ conformant…

– Reports on the system itself (audit trail)

– Reports on use of the records

– Both add authenticity / legal admissibility to

records


#4. Where can we do preservation?

• In the EDRMS itself?

• Or in a dedicated archival repository

system?


EDRMS might not work for preservation

• According to DIRKS (2012)…

• Q: Will the EDRMS automatically manage the long term needs of records?

• A: No. There are a number of factors that can still threaten the long term accessibility and use of records.

• Vendor support

• Software upgrades

• Move to new EDRMS must be handled carefully

• Data loss, corruption, alteration risks


EDRMS might not work for preservation

• Archives New Zealand:

• Archives New Zealand is now in a position to accept digital records exported into its custody where the records have been appraised as having ongoing archival value. Requirements for digital transfer remain flexible to incorporate existing digital records not accompanied by recordkeeping metadata to the standard.


Possible Gaps (Technical)

1. No “bridge” from EDRMS to TDR

2. Not always clear if we can export out of

EDRMS, or in what format

3. Do we want to capture the EDRMS audit

trail?

4. Can’t easily change file formats in the

EDRMS

5. Additional metadata needed for preservation


Possible Gaps (Human)

Records Manager Archivist IT Manager

All I care about is current

records and the next five years

Our selection policy means I have to

preserve a history of this organisation

forever

I don’t care about either

of you


Life Cycle of Record

CreationMaintenance

and Use

Preservation

Destruction

Disposition

EDRMS Functions

OAIS Functions


What is OAIS / TDR?

• ISO standard(s)

• Proposes a dedicated space for storing archival copies

• Allows us to protect important content in archival packages

• Can carry out migration in a controlled and professional way

• Uses tools which EDRMS doesn’t have

• Can create preservation metadata


Some proposed steps towards

preservation

1. Survey

2. Define use cases

3. Do preservation planning

4. Define ingest requirements

5. Define data management requirements

6. Define access requirements

7. Think about migration


Terms…

• Migration = transforming digital objects

from one file format to another

• Moving = taking electronic records out of

the EDRMS

• Export = will use this term to apply only to

metadata


1. Survey

• Survey record holdings

• Create an inventory

• Identify record types with long retention

needs (e.g. 30-50 years)

• Note file format types in use

• May be a small percentage


2. Use cases

• Identify types of users

• Find out what these users expect to be

able to do with the records

• Make decisions about how much you can

support technically


Example A

• Internal users might want…

– To find documents

– To read documents

– To amend or edit them (unlikely, but possible)

– To delete them

– To operate a disposal rule (if they are the

records manager)


Example B

• External users might want access, but…

– Documents might be closed to the public

– When can they be opened?

– Can they be released in redacted form?

– What about an auditor or other legal official?

– What are your legal obligations?


And what about…

• Individual digital objects

– Will you need to retain formatting? Colours? Fonts?

– Editability and processability?

– Formulae in spreadsheets?

– Email attachments?

• Provenance and original order

– Will you retain original file names? Folder names? References? History of use?


The value of use cases

• Have implications for what kind of

preservation system you are going to build

– Functionality

– Search facilities

– Access mechanisms (security marking,

redaction)


The value of use cases

• Have implications for managing digital

objects

– When and how will you migrate objects?

– What can you afford to lose in migration?

– How much do you need to support?


3. Preservation planning

• When will you move?

• What will you be moving?

• What about keeping metadata from the EDRMS,

e.g.:

– File plan scheme?

– Retention scheduling rules?

– Use history?

– Access / security markings?

• Will you migrate file formats? When? How?


Can we move / export?

• TNA have noted:

• Most systems “have not been designed for

easy import and export of information.”

• “It is rare that an EDRMS export captures

everything held in the system”.

• (Migrating Information between EDRMS,

Digital Continuity Project, 2010)


4. Define ingest rules

• Move objects from EDRMS

• Export EDRMS metadata

• Extract technical metadata from objects

• Identify access / closure requirements

• Build preservable packages


5. Define data management

• How will you store and manage metadata?

– Keep it as a chunk of XML

– Incorporate metadata with your database

– Map it to METS

• What about preservation metadata?


6. Define Access

• Devise access rules for records, e.g.:

– Security markings

– Authentication procedures

– Redaction processes

• Apply them to your preserved objects


7. Migration

• Convert old file formats into supported file formats

• Devise acceptance criteria (part of preservation

planning); what essential properties must not be lost

through migration?

• Learn from your use cases; you cannot support

“everything”

• Choose conversion tools

• Identify formats that may cause problems

• Decide when you will migrate

• Keep records when you do it (preservation metadata)


Possible result

OAIS compliant digital archive Access function

Internal user

External user

Auditor

SearchReadEditAuditDispose

- Preserves- Holds metadata- Keeps some object functionality- Replicates some functions of EDRMS- Builds accessible DIPs- Redacts- Migrates


Wishlist: Technical

• Automation

• More bridge-building between systems

• Keeping as much original metadata as

possible

• Can we map MoREQ to METS?

• Move towards agreement on the ideal

sharing format (e.g. XML)


Wishlist: Planning

• An integrated digital records-to-digital

archives lifecycle

• Do migration early, where appropriate

• Authenticity and integrity of records

retained

• Someone in organisation taking ownership

of preservation


Wishlist: Human

Records Manager Archivist IT Manager

We need long-term retention

of these records, but I can’t do it in my EDRMS

I’ll learn more about practical

digital preservation

You both have credible business

plans, so we’ll invest in the IT

you need

From EDRMS to TDR: A Wish-list for Ways to Preserve Digital Records

Technology

life of records

digital records

migration of records

life of file formats

short life

pension records

digital preservation

small percentage of