University of Oslo's TSD service - storing sensitive & restricted data by Dag-Erling Smørgrav

TSD:Securing sensitiveand restricted data

Dag-Erling Smørgrav

EuroBSDCon 2014

What are sensitiveand restricted data?

● Gene sequences● Patient records● Survey responses● A/V recordings of patients and respondents

Quoth the law

TL;DR:

Personally identifiable data may only be collected and retained with the person's informed consent,

for a specific purpose, for a specific length of time.

The dilemma

The data must remain be kept under lock and key.

The data must be accessible to those who collected it.

The solution

Provide a fully functional working environment within which the data is accessible, but from which the data may not be (easily) extracted.

A fully functionalworking environment

● Storage● Databases● Virtual Windows and Linux desktops with

remote access– Office software

– Scientific software (Biopython, R, Matlab, Stata, SPSS)

● High-performance computing cluster

Data transfer

● The only direct access is through RDP or SPICE (remote desktops) over an SSH tunnel.

● Clipboard, shared folders and other easily-used side channels are disabled.

● Data is transferred through a data lock which logs all transfers with user, file name, file size and SHA256 checksum.

Storage

HPC

Database

Desktop

File lock

Jumphost

The Big, Bad

Internet

Bird's-eye view

Network topology

Jumphost

Data lock(ext)

Jumphost

Big Cisco box

SmallerCisco box

Storage VLAN

Data lock(int)

Storage

Prism

Management VLAN

DC

DNS

Nexus

RHEVM

Jumphosts

● Dual role: router / firewall and login– In hindsight, these should have been separate;

there are (surmountable) technical obstacles.

● Router / firewall: pf + pfsync + carp + authpf● Login: OpenSSH with two-factor authentication

– RFC 6238 TOTP for users with smartphones

– RFC 4226 HOTP with YubiKeys for others

Multiplicity

There are, at present, around 45 different research projects using TSD.

They must, of course, be kept separate.

Network topology (again)

Jumphost

Jumphost

Storage VLAN

Management VLAN

DRAC VLAN

Hypervisor VLAN

Project VLANProject VLANProject VLANProject VLANProject VLANProject VLAN

IAA

● Multiple provisioning systems, for historical reasons.

● Cerebrum is the main database, pushes data to other systems.

● Active Directory used for identity (LDAP) and authentication (Kerberos) internally

● RADIUS used for OTP verification (OATH implementation from OpenPAM)

SSH RADIUS Nexus AD CerebrumFirewallcreate user

create user

set password

set OTP key

get user identity

verify OTP

verify password

verify OTP

insert user rules

remove user rules

IAA and the login process

Provisioning

● Automatic provisioning (creation and installation) of VMs based on data in Cerebrum

● Automatic configuration of new VLANs and subnets (only partially implemented)

● pf address tables updated based on machine roles assigned in Cerebrum, allows fine-grained control of network traffic

● Users are affiliated with projects, allows restricting their authpf ruleset to only VMs belonging to those projects (not yet implemented)

% ssh [email protected] are connecting to the University of Oslo's Secure Research Facility.Access to this facility is restricted to duly accredited members ofparticipating research projects. Please ensure that you only connectto this facility using equipment and networks which have been approvedby your project manager and / or parent institution.

Additional information may be found at the following URL:

http://www.uio.no/tjenester/it/forskning/sensitiv/

One-time code: 608911Password: Last login: Thu Sep 18 12:56:38 2014 from nargothrond.uio.no

Hello p01-des. You are authenticated from host "193.157.137.122"

To connect to the TSD server "tsd-altinn" as user "tsd-kenneth", firstset up an SSH tunnel with the following command:

% ssh -L9999:tsd-altinn:3389 [email protected]

You can now connect to the server by running the following command in adifferent terminal window:

% rdesktop localhost:9999

or start an RDP client and specify "localhost" as the server and 9999 asthe port.

To connect to multiple servers using the same SSH tunnel, you will have touse a different port for each server:

% ssh -L1000:tsd-altinn:3389 -L1001:tsd-altut:3389 \ [email protected] % rdesktop localhost:1000 # connect to tsd-altinn % rdesktop localhost:1001 # connect to tsd-altut

To disconnect, press Ctrl-C in this window. You may have to press it twice.

Where the rubbermeets the road

IPv6

TSD was intended to be an IPv6-only environment, but a lot of software still does not support IPv6, or does not support it correctly.

– RHEV does not support IPv6 at all.

– Cannot use SLAAC: Linux source address selection is broken. Forced to use carp on the inside.

– Also found and fixed bugs in FreeBSD's rtadvd before we gave up SLAAC.

pf and carp

● Found and fixed bug in FreeBSD source address selection (only when using carp)

● Found, but haven't fixed, bug with routing of IPv6 UDP packets (possibly checksum corruption in pf)

● State table filled up with long-lived state entries for DNS, NTP, Kerberos etc. requests– Greatly reduced timeout for UDP state entries– Greatly increased table size

IAA issues

● FreeRADIUS is difficult to configure correctly and slightly unreliable

● nss_ldap is slightly broken (my bad!)

Questions