Pulling it all together… with thanks to Sheila Anderson.

Pulling it all together…

with thanks to Sheila Anderson

Pulling it all together

Think of it as a continuum…………………..

2

Su b m is s io n

3 4

R e vis io n (s )

Qu a lity As s e s s m e n ta n d Pu b lica tio n

6

Te ch n ica lOb s o le s ce n ce

Res ourc e Dis c ov ery MetadataTec hnic al MetadataRights MetadataFile Format Conv ers ionUnique, Pers is tent Identif ierV ers ion Contro l

Migration, EmulationOther Pres erv ationA c tion

C re a tio n

1

File Format & ContentTy pes Determined

5

R e vie wR e te n tio n

4

7

W ith d ra w

The OAIS Model

Vocabulary for Digital Repository practitioners

Sub-divides a complex task into related functions

Provides logic and structure to allow digital holdings to be managed and processed

Six Primary Functions

IngestArchival StoreAccessData ManagementAdministrationPreservation Planning

OAIS Functional Model

Some thoughts. . .

OAIS is a conceptual reference model and not a system design modelFunctions in the OAIS Reference Model may not necessarily correspond with the functional modules of a system that would implement the model. (Organisational needs vary!)Functionality of an actual digital repository may be much more limited than in the OAIS Reference Model, and may include additionsTake from it only what you need to manage your digital collections

Process and Activities

Business and strategic planning, including costs and fund raising

Resource creation (not included in OAIS model)Resource preparation and enhancement (not included in OAIS)Resource presentation: access and delivery (Access) Resource curation / Data Management (Ingest)Preservation planning (Preservation Planning)Archival storage (Archival Store)

Management and administration (Admin + bits of data management)

Process and Activities

Strategic and business planning; fund raisingWhat is the purpose of your data creation efforts?Are you undertaking a one-off project or establishing something longer termIf the latter is it short, medium or long-term?Will you be providing a service for others to use?What skills will you needWhat infrastructure? What will it cost?How will you raise the money?Do you want to do everything in-house or outsource some or all of the tasks?

Process and Activities

Resource creation: the methods used to convert analogue information into raw digital data

ScanningDigital photographyTranscriptionOCREtc. etc.

Process and Activities

Resource preparation and enhancement: methods used to transform raw data into information

Modeling and design of the resource’s information structureNormalisation and enhancement of data StructuringEncodingManipulationAdding and creating metadata

Process and Activities

Resource presentation: providing access, delivery and use of the resource

Static or interactive web-based services; Off-line servicesResource discoveryOn-line browsing and selectionRequests from users Coordinates requests and execution of requests Applies controls to limit access as neededGenerates information for disseminationDelivers information (data) to users

Process and Activities

Resource curation: management of the resource over the long-term

Maintaining schema, view definitions, referential integrityReceiving/managing updates & additions to data & metadataVersion ControlPerforming quality assuranceGenerating archive versions to archive’s data formatting and documentation standardsExtracting descriptive information for data management and to support search and retrieval functionsCoordinating updates and transfer of the data and metadata to appropriate places

Process and Activities

Preservation planningEvaluating contents of archive; periodically recommending updates to migrate current holdingsDeveloping and maintaining recommendations for archive standards and policiesMonitoring changes in technology environment, users’ service requests, and knowledge baseDeveloping and maintaining detailed migration plans, software prototypes and test plans

Process and Activities

Archival storageReceiving data from curation process and adding to permanent storageManaging storage hierarchyRefreshing/replacing mediaPerforming error checksProviding disaster recovery capabilitiesDuplicating contents for off-site storage

Process and Activities

Management and administrationNegotiating agreements and licencesManaging IPR, copyright issuesManaging quality control and audit processMonitoring systems operationsProviding inventories, reports, updatesEstablishing and maintaining archive standards and policiesProviding user support

Requirements

OAIS-Type Repository

CreationPreparation

Access Curation

Preservation Planning

Archival store

Management and Administration

Money

Skills

Strategic Business Planning

Infrastructure

Infrastructure

SpaceEquipmentFacilitiesProject Management

Systems and toolsSoftware and hardwareStandards – technical, metadata etc.

Skills

Scholars / Curators / digital librarians (creation, acquisition and curation)Scanner operatives, photographers etc. etc.Information professionals (metadata management & creation)Computer scientists (systems design and development, management tools)Preservation professionals (preservation planning and management, migration and emulation services)Systems administrators (systems management, disc partitioning, refreshment, back-up)Managers and administrators: project, process, strategic and financial etc.

Costs

Staff salaries and benefits

Operating expenses

Software

System equipment

Other?

Solutions: In-house Repository

“….is a set of services that a organisation or institution offers to the members of its community for the creation, management and dissemination of digital materials created by the organisation or institution and its community members. It is most essentially an organisational commitment to the stewardship of these digital materials, including long-term preservation where appropriate, as well as organisation and access or distribution.”

Lynch, C., ARL Bimonthly Report 226, http://www.arl.org/newsltr/226/ir.htm

A very long quote……

As with computing, the cost of data repositories (done correctly) will be dominated by the recurring costs of personnel performing curation, maintenance and upgrade,and providing user advice, assistance, and support. The most sophisticated of these personnel need professional skills in the relevant aspects of information management and information technology (e.g. databases, archival file systems, building portals),and will be developing and maintaining custom software. By using a combination of high-speed networks and local high-speed caches, there is no hard requirement to co-locate professional staff with physical storage particularly staff performing data acquisition and curation functions as opposed to disk partitioning, regeneration, and backup functions. As with computing, there is need for support personnel at local institutions, in discipline-specific groups (often located in centers),and centralized in centers. Although further analysis is needed, we expect that the most efficient approach will be to have relatively centralized storage hardware (with supporting staff) but distributed data acquisition and curation personnel.

  http://www.communitytechnology.org/nsf_ci_report/report.pdf (page 77)

Solutions: Sharing; Outsourcing

Sharing infrastructure and costsInstitutional consortiaRegional consortiaLocal/national alliances

Federated servicesLocal/national/regional

Outsourcing some or all of the tasksCreationPreservation

Example: Shared Archives Model

Institutional Repository

Accessions institutional collections and provides access and delivery services

Metadata integrated into preservation metadata store and on-line catalogue facilities

Preservation System

Harvest preservation (and discovery) metadata

Capture preservation copies

Users

Preservation ServicesRisk assessment

process

Preservation Service

Benefits and Challenges

BenefitsShared infrastructureShared expertiseEffective use of resources – human, technical and financialShared costsExperts doing what they are best at

ChallengesCreating partnerships (partnerships or outsourcing?)Agreement on policy and proceduresEstablishing relationships of trustConsistency of practice and outputs

Now it’s your turn…….

Recap: Questions to Ask What is the purpose of your data creation efforts?Are you undertaking a one-off project or establishing something longer termIf the latter is it short, medium or long-term?Will you be providing a service for others to use?What skills will you needWhat infrastructure? What will it cost?Do you want to do everything in-house or outsource some or all of the tasks?

Resource creation

Skills?

Infrastructure?

Likely costs (high, medium, low)?

Possible solutions (in-house, outsource, other)?

Other issues?

Resource enhancement

Skills?

Infrastructure?

Likely costs (high, medium, low)?

Possible solutions (in-house, outsource, other)?

Other issues?

Resource presentation

Skills?

Infrastructure?

Likely costs (high, medium, low)?

Possible solutions (in-house, outsource, other)?

Other issues?

Resource curation

Skills?

Infrastructure?

Likely costs (high, medium, low)?

Possible solutions (in-house, outsource, other)?

Other issues?

Preservation planning

Skills?

Infrastructure?

Likely costs (high, medium, low)?

Possible solutions (in-house, outsource, other)?

Other issues?

Archival storage

Skills?

Infrastructure?

Likely costs (high, medium, low)?

Possible solutions (in-house, outsource, other)?

Other issues?

Management and administration

Skills?

Infrastructure?

Likely costs (high, medium, low)?

Possible solutions (in-house, outsource, other)?

Other issues?