Who Decides?Reinterpreting archival processes for the management of digital research
Gareth KnightCentre for e-Research, King’s College London
2
Presentation Themes
1. Need to re-appraise definition & criteria for a Record
2. Challenges posed when attempting to archive digital records
3. Technical architecture and processes required to manage digital records
3
What is a record?
“information created, received, and maintained as evidence and information by an organization or person, in pursuance of legal obligations or in the transaction of business”
ISO 15489-1:2001. Records management
“a specific piece of recorded information generated, collected or received in the initiation, conduct or completion of an activity and that comprises sufficient content, context and structure to provide proof or evidence of that activity “
The International Committee on Archives (ICA) Committee
4
What is a record?
“information created, received, and maintained as evidence and information by an organization or person, in pursuance of legal obligations or in the transaction of business”
ISO 15489-1:2001. Records management
“a specific piece of recorded information generated, collected or received in the initiation, conduct or completion of an activity and that comprises sufficient content, context and structure to provide proof or evidence of that activity “
The International Committee on Archives (ICA) Committee
5
What is a Record today?What is a Record tomorrow?Criteria for defining records that have archival value is subject to range of factors and may change over time:
Legal obligation:• Varies between institution of different type• Geographic location• E.g. Public Records Act 1958 (government), Data Protection Act, Freedom of
Information (FoI) Act 2000, Environmental Information Regulations (2004), etc.
Proof of business:• What type of business? Commercial, non-commercial• Research as a business activity
Proof of activity:• May be broad. Influenced by function, evidential value, uniqueness
•Memory of institution• Same as above – requires consideration of function, value, uniqueness and
other criteria
6
Re-evaluating Records management at King’s College LondonCollege Archives acquire, preserve and makes available all material of long-term, evidential and research interest that forms part of the College’s heritage•Acquisition policy:
• Paper and electronic records of day-to-day business operation, private papers of academics and researchers and material related to College heritage
•Preservation policy:• Preserve archives in their original physical format
(storage, packaging). • Maintain in appropriate environmental conditions – in
compliance with BS5454: 2000
http://www.kcl.ac.uk/iss/archivespec/spcm.html
7
New archiving challenges
New types of information:• Published research papers, research data – funder
requirements for data management
Creation of increasingly diverse content types:• Hybrid (paper+digital), digital content• CAD designs, interactive resources, datasets. Publication
of dynamic content
Update frequency:• Web site, blogs, twitter, object revisions. Versioning
Access lifecycle:• Technology dependencies – software & hardware
Uncertain value:• What is the business value? What period of time?
8
Preservation Exemplars at King’s (PEKin)Project objectives:
1. Develop a management system capable of handling digital business records ANDAND research material
2. Adopt a management strategy that brings together archival AND data curation approaches
3. Embed preservation practices within the institution through a multi-layered strategy:• work with central services to develop a preservation strategy and service for digital
records• work with academic units and professional services to ensure local data producers
and systems managers are provided with targeted advice, guidance and tools to support decision-making
Project Partners:
Centre for e-Research (CeRch) & Archives & Information Management (AIM) at King’s College London
Funder:JISC Information Environment 09-11 preservation exemplars strand
9
Audit framework
PEKin audit framework combines sections of DAF, DRAMBORA, DIRKS & other audit work
10
Audit management practices
Purpose of audit was to identify:• Functions within the organisation that create records• Who used records and for what purpose• Location and responsibility for storage• Time period they are currently / should be retained for• Future stakeholders that need/may wish to use records
Survey academic and business units• Core business units:
• College Estates, Student Records, College Committees
• Research groups:• Twins Early Development Study, Regional Information Collection
Centre, Environment Research Group, Randall Division of Cell and Molecular Biophysics
11
Types of digital information
•Business records:• Estates records – Property records, contract for building
maintenance, Computer Aided Design (CAD)• Student Records – Students, courses, grades• Committee Records – Structure, operation
•Research records:• Commercial – research data created for commercial
purposes, e.g. pollution monitoring, patents• Funded research – Contracts awarded by funding bodies• Unfunded research – Academic researcher who has interest
in topic
Each record has different value and retention period.
12
Many types of lifecycleRecord lifecycle
(variants: Information, data lifecycle)
Access lifecycle
(e.g. digital lifecycle)
13
Analysis of lifecycle risks
•Identify & evaluate risks that occur in the lifecycle
•Applied a ‘light touch’ DRAMBORA (http://www.repositoryaudit.eu/) methodology to case studies. Influenced by DIRKS
•Risk categories• Organisation Management, Staff, Tech Infrastructure, Acquisition &
Ingest, Preservation & Storage, Access & Dissemination
•Risk Description• Definition, manifestations, consequences, severity (risk impact x
probability), mitigation strategies
14
Recognised risks
•Storage:• Insufficient capacity: local drives, network drives, 3rd party server
•Authenticity & integrity:• Unidentified/unknown change. Some staff rely upon print-outs of
digital original
•Archival value and retention period:• Different criteria & quality thresholds• Business records – Recognised legal value & retention period• Research data – Archival value of research papers understood.
Retention period of data has, until recently, not been recognised
•Access and usage:• Business records have well-defined period of primary use, but
unrecognised secondary use• Research papers understood, but do not always consider datasets &
other outputs
15
Risk management Strategy
•Storage and management infrastructure:• Technical infrastructure to store and manage their data
•Education:• Develop staff understanding of data management and archival
principles• Topics include: Authenticity and integrity, assessment of
archival value• Methods: Practical documentation on data
creation/management, training events
•Policies:• type of record collected, time period for collection, appraisal
criteria for long-term retention
•Procedures:• Data capture, curation, preservation
Developed with consideration of cost implications
16
KCL Archives Preservation Repository
An preservation repository for college data of short/long-term value:
Standards compliance• OAIS Reference Model, TRAC, ISO 15489
•Bitstream preservation:• fixity creation/verification, online + offline storage
•Information Content Preservation:• Format conversion, event logging – audit trail
•Access:• Limited to archive reading room, catalogue descriptive MD to
common standard
•Interoperable:• Interact with other college & public systems eg student records
17
OAIS Reference Model
18
Technical Infrastructure
19
Alfresco Actions
•Actions - a parameterized unit of work that can be applied to a node
•Parameters – rules for action execution and type to be performed
•jBPM synchronous or asynchronous workflows
•Actions be performed at different stages of workflow
20
Ingest Actions
•Content model compliance• Conforms to defined structure & object types
•Fixity generation• All: MD5, SHA-1, CRC
•Format identification• All: File(1), DROID
•Technical metadata extraction• Format specific: JHOVE, MP3Info, others
•Specification conformance• branch workflow according to threshold
•Conversion to preservation & dissemination derivative• parameters for each format & licence• OpenOffice, ImageMagick, SoX
•Data Packaging• Generate METS package, record action results as PREMIS Event
21
Archiving actions
•Transfer into Fedora archive• when collection closed (e.g. all papers submitted for meeting
collection• After specified time period, e.g. 3 months
•Fixity verification:• Conform that fixity unchanged
•Manual activity for future date:• Anonymisation in 2 years,• Re-appraisal in xx years – Retain or remove
•Obsolescence monitoring?• Possible future implementation
22
Content Models
•Content models define rules that govern collection structure, data type & behaviour
•Preservation Archive Content Models designed with consideration of resource type and Alfresco & Fedora capabilities
•Content model for each resource type composed of 3 layers
•Each layer is a Fedora Object that contains different metadata
Filing Cabinet
Drawer
Folder
23
Content Model examples
Each item represents a Fedora Object
Each FO holds user provided metadata
Different MD required at each layer
Committee Records
Meeting
AgendaPrevious minutes
PapersCurrent minutes
Research Project
Year
Data Papers Meeting notes Documents
Filing Cabinet
Drawer
Folder
24
Findings (so far)
• Data audit & risk analysis provide useful frameworks for analysing data management practices & justifying data archiving
• No single definition of archival value – different criteria & quality thresholds
• Application of archival principles to data assets provide demonstrable benefits
• Duration requirements of business & research data are broadly similar
• Digital repository architecture provide sufficient flexibility to manage data assets created for different purposes
25
Contact
Gareth Knight
Centre for e-Research, King’s College London
020 7848 1979
http://www.kcl.ac.uk/iss/cerch/projects/portfolio/pekin.html
Centre for e-Research :
www.kcl.ac.uk/iss/cerch
Archives and Information Management (AIM) :
http://www.kcl.ac.uk/iss/explore/team/aim