Mid-Sweden University/SNIA Conference 13 October 2008

1

Building the Archives of the Future

The National Archives and Records Administration

Electronic Records Preservation:Electronic Records Preservation:ERA Research and DevelopmentERA Research and Development

Lagring – Arkivering Två världar – en vardag?

13 October 200813 October 2008

Mark ConradERA Research

National Archives and Records Administration

13 October 2008 National Archives and Records AdministrationElectronic Records Archives (ERA) Program

2

NARA’s Mission

To ensure access to records of three branches of the U.S. Government. Records that:• Protect citizen’s rights• Hold Government officials accountable• Facilitate historical understanding of our

national experience


3

Electronic RecordsNARA’s ChallengesNARA’s Challenges

• Scope The entire U.S. Federal Government

• Obsolescence Constantly Changing Technology

• Access Ability to view records over time

• Volume Large amounts of records arriving to NARA

• Variety Different/Complex Types of Records

• Complexity and Records Formats


4

ERA Program History

• 1970First electronic records transferred

to NARA, begin preservation


5

0

2,000

4,000

6,000

8,000

10,000

12,000

14,000

16,000

1970-1988 1989-1995

Transfers of Digital Files to NARA (1970 – 1995)


6

0

50,000

100,000

150,000

200,000

250,000

1970-1988 1989-1995 Reagan/ Bush

Transfers of Digital Files to NARA: Reagan/Bush Presidential Records


7

Transfers of Digital Files to NARA:Clinton Presidential Records

0

5,000,000

10,000,000

15,000,000

20,000,000

25,000,000

30,000,000

35,000,000

1970-1988 1989-1995 Reagan/ Bush Clinton


8

ERA Program History (cont.)

• 1970 First electronic records transferred to NARA, begin preservation

• 1998 NARA is heading for mission failure


9

NARA’s Strategic Response: The ERA Program

1. Research and exploratory development on technologies that offer promise for addressing electronic records challenges.

2. Acquiring and building a system that meets our requirements and our mission for NARA, the Presidential Libraries, and Federal Records Centers

3. Organizational and cultural Change Management


10

ERA Program History

• 1998 ERA Research Begins– Understand the Issues

– Feasibility

– Relevant Technologies


11

ERA Program History

• 2000 ERA becomes an Official Program

• 2003 ERA Requirements Issued– http://www.archives.gov/era/pdf/requirements-

amend0001.pdf

– Based on ERA Research

– Some Standards Referenced• OAIS

• DoD 5015.2


12

ERA Program History

• 2008 ERA IOC

• 2008 - ERA Research Continues– No one knows how to preserve and provide

sustained access to authentic electronic records for most types of electronic records

– No one knows what information technology will be in the future


13

ERA Research Program

“Next-generation methods, technologies, and tools are needed to...manage massive stores of distributed, heterogeneous information (e.g., science and engineering research data, Federal records, health information).

“Next-generation methods, technologies, and tools are needed to...manage massive stores of distributed, heterogeneous information (e.g., science and engineering research data, Federal records, health information).


14

Human Computer Interaction and Information Management (HCI&IM)

NITRD Agencies: NSF, DARPA, OSD and DoD Service research organizations, NIH, NASA, NIST, AHRQ, NOAA, EPA, NARA

President’s 2009 Request

Strategic Priorities Underlying This Request

Today’s increasingly data-centric world requires the effective and strategic use of information assets. To advance the role of HCI&IM in providing strategic support for national priorities, R&D in this area focuses on:

Information integration: To support complex human ideas, analysis, and timely decision-making, large amounts of disparate forms of raw information must be managed, assimilated, and accessible in formats responsive to the user needs. Next-generation methods, technologies, and tools are needed to fully integrate and efficiently manage massive stores of distributed, heterogeneous information (e.g., science and engineering research data, Federal records, health information)

Key research issues include:

– Information standards: Data interoperability and integration of distributed data; usability; provenance and integrity (metadata); generalizable ontologies; accessibility

– Decision support: Timeliness of and access to data; user-oriented techniques and tools for summarization, synthesis, analysis, and visualization of information for decision-making; measurement and management of human responses to data

– Information management (IM): Intelligent rule-based data management, efficient integration, maintenance, and access to complex, large-scale collections of heterogeneous data; innovative systems architecture; scalable technologies; integration of policies (differential sensitivity, security, user authentication) with data; integrated distributed data repositories; testbeds; sustainability and validation of complex models


15

Army ResearchLaboratory

NationalScienceFoundation

Data-Intensive Cyber

Environments (DICE) Group

NISTNIST

Some examples of ERA Research Partnerships


16

Archive Storage VLAN

System/Business Application VLANExternal AccessGateway (DMZ Access)

ERA Management VLAN

Ingest VLAN

WebServer VLAN

Ingest WorkingStorage

ArchiveDisk

Storage(Managed Storage)

To GreenbeltERA Mgt

1Gb Ethernet 2/4Gb Fibre Channel

Firewall+ IDS

HW Load Balancer

Ingest Server (App Server)

Unix Input Workstation

Physical IngestEthernet Switch

Windows Input

Workstation

Printer

DVD

TapeDrive

CD ROM

Printer

DVD

TapeDrive

CD ROM

HW Load Balancer

Web Server

Ethernet Switch

Archive Label Printing WS

Printer

Archive Tape

Storage

Mgt Router(w/IPSec)

Accountability Server

Accountability WS

Backup Restore Server

Enterprise System Mgt

Server

Help Desk Server

Network Mgt Server

Security Server(Code Modification)

Security Server(Vulnerability Scan)

Software Deployment

Server

Extranet Router(w/FW + IDS + IPSec)

Firewall + IDS

Firewall + IDS

Firewall + IDS

Firewall + IDS

Central Data Manager Server

Data Services Server (Mgmt

DB Server)

ERA Extranet

GOVTAgencies

NARA

Core Switch(w/FW + IDS + LB)

Firewall + IDS

Firewall + IDS

Firewall + IDS

Firewall + IDS

Ethernet Switch Database Server (T&I)

Derived from:ERA Hardware Block Diagram -2007 0823(Tab: I1R2 U/USBU Detailed Block)Updated 24 Aug 2007

SOCSOC WS

Ethernet Switch

Firewall+ IDS

Ethernet Switch

Firewall+ IDS

Documentum Server

Strong Auth

Server

DNS/Directory

Server

Database Server (SBA)

Asset Mgnt WS

App Server (SBA)

Instance Datastore

FC Switch

Mgmt LAN,Storage Network

Backup Network

Ops LAN,Management LAN,Storage Network



Ops LAN,Management LAN

Fundamental Requirements for The ERA System

1. Evolvability

2. Scalability

3. Extensibility


17

ERA Research

• Data Intensive Cyber Environments Group– Transcontinental Persistent Archives

Prototype (TPAP)– intelligent Rule Oriented Data System

(iRODS)– Make Policies/Rules Explicit– http://www.irods.org


18

ERA Research

• A Sample of Continuing Collaborations– ARL – High Confidence Computing

– NCSA – Storage and Retrieval of 3D+Time Data Representations

– NSF – Cyberinfrastructure

– Pittsburgh Supercomputing Center - SLASH


19

Building for the Future

1. Anticipate changes – characteristics of electronic records, – Preservation and Access technologies – Researcher expectations and behaviors

2. Recognize those things that will not or should not change

– Archival science provides stable principles, concepts, requirements and understanding.

– NARA’s mission and the functions

3. Make reasonable assumptions about the future – Use of computers will continue to become more common – Information use and Technology will continue to grow– Decline in Information Technology costs – The Internet will continue to grow


20

ERA ResearchERA Research

[email protected]@nara.gov

The ERA Web site is:http://www.archives.gov/erahttp://www.archives.gov/era

Your Contact in the ERA Program Office

Mid-Sweden University/SNIA Conference 13 October 2008

Technology

era research program

era requirements

era program history

records formats

records administration

era ioc

types of electronic

electronic records challenges