Why We Tried (and Ignored) Famous IaaS To Deliver SecurePass

31st January 2015Giuseppe Paternò@gpaterno

Why We Tried (and Ignored) Famous IaaS To Deliver

SecurePass

https://www.flickr.com/photos/kewl/8475764430

Knowing “GIPPA”...

CTO of GARLSwiss company behind the SecurePass cloud identity management service. GARL is mostly focused on identity and security

Trusted advisor for customers on cloud and complex OSS architectures: OpenStack, CloudStack, OpenNebula & Ganeti Previously Senior Solution Architect in Canonical, Red Hat, Sun Microsystems and also in IBM.

I work with Linux since 1996. In my (little) spare time, I publish books and whitepapers

MAKING THE CLOUD A SAFER SPACE

IT security products and cloud services focused on identity protection on the Cloud. Born from Symantec, conducting pentest and vulnerability assessment on their behalf in EMEA.

Most of the customers in finance and telco operators.HQ based in Switzerland (Lugano and Zurich) and office in London.

User privacy is protected by strict Swiss privacy regulations, no UE or US exceptions allowed.

GARL?? What is that?

Cloud with SecurePass: centralized identities

SecurePass is your Swiss knife to protect and manage identity in the cloud: a suite of integrated tools that allow web apps, OS and devices to quickly manage users and secure access.

Cloud Identity Management (LDAP)Strong authentication (RADIUS)Web Single Sign-On (CAS)Federation (SAML)Central logging platform (next gen)

Supported in the distribution by:One Time Password

345227

345227345227 Identity

Management

Single Sign-On

SecurePass, the cloud identity platform

Datacenter 3 (Active)Switzerland(Former Military Premise, secret location)

Datacenter 2 (Active)

Italy

Datacenter 1 (Active)

Switzerland

High-speed secure replication among all sitesMulti-datacenter Global Secure Architecture

Off-sitesGlobal load balancers

Off-sitesGlobal load balancers

Site Overview

Keepalived+ LVS

RADIUSResponder

LDAPResponder

CASResponder

Data Node(Cassandra +

Python wrappers)

MySQL(Billing)

Web AdminPanel

RESTful APIsendpoint

Master Keys

MySQL(Billing)

Keepalived+ LVS

LDAPResponder

Data Node(Cassandra +

Python wrappers)

Master Keys

LDAP is OpenLDAP plus custom backend plugin

Keepalived as a balancer, cannot use haproxy for missing UDP

MySQL mostly used only for billing, no actual user data

Master Keys kept in a secure location

Data nodes with Cassandra wrapped in a Python for internal APIs and crypto

Site Overview

Mixed use of CentOS 6 and

Debian Wheezy/Squeeze

(in update)

Mostly Python

Healthchecks custom-made to

simulate OTP requests

https://www.flickr.com/photos/neutronboy/5497209926

Feature rich (vSphere HA, vMotion, DRS, I/O control)

Very large ecosystems

All os vendors make it supported and certified under ESXi

ESXi can be downloaded and used freely

Per core license - expensive

Proprietary platform

ESXi can not be APIs accessed - need to buy licenses

Latest versions can be improved, getting complicated even for easier tasks

Pros/Cons

https://www.flickr.com/photos/silverstack/7642706420

Used to be a contributor in patches

Very similar to the VMWare approach

It’s free and opensource, based on Debian with Perl

Has initial concept of software-defined storage (Ceph), but doesn’t work very well

Some weakness on the networking side, it is complex to manage multiple VLANs, need to do bridges by hand

Doesn’t play nice with more than two nodes in the cluster (in our experience back in 2012)

Doesn’t work well on some hardware, kernel is modified from stock Debian (and sometimes broken)

Community sits around 2 brothers in Vienna and with one of them there are “relational problems”

Pros/Cons

https://www.flickr.com/photos/navalhistory/5983640432

Probably the most successful example of opensource project after Linux

Support from many OEMs and OS vendors

Interoperability with many components, just pick your favorite one and plug it in

Standard and well accepted APIs

Very complex to setup and troubleshoot

Although common codebase, might differ from implementations

Need high numbers of management nodes

Difficult to maintain for a small team, not worth for same application

Pros/Cons

https://www.flickr.com/photos/mitko/6877788589

GANETI

Lightweight architecture

Can start with a single node and scale out easily

Designed for use local storage and cheap storage (like Ceph)

Great for "standard" Linux and Windows workloads

Easy to pick up for a standard Linux sysadmin

HA of master need to be triggered from Monitoring platform

Lack of some features (ex: storage vmotion)

Start to become complex from a code perspective

Release cycles too short (can’t upgrade every 3 months!)

GANETI

Pros/Cons

https://www.flickr.com/photos/postaletrice/3799225740

We like the templating idea

Great to deploy frontends and upgrade easily

..... but:

(For us) don’t make sense as we have RPM packages for our software and kickstarts

Security concerns running a single kernel image

Pros/Cons

So what??

Stable virtualization platform (KVM preferred)

Works on a broad range of hardware, also refurbished one

HA not needed in the platform (performed at application level)

Trigger creation of virtual machines through scripts or APIs

Flexible VLANs, without need of reconfigure the platform

Backup of the VMs not needed

Cheap storage solution

Our requirements

Base CentOS 6

Openvswitch from RDO

GlusterFS from gluster.org (EPEL release, only for the control nodes)

Corosync/Pacemaker (only for control nodes)

What we didBuild our own flavor with:

https://www.flickr.com/photos/stickkim/7377611424

Control Nodes(libvirt/KVM + Open vSwitch

+ Corosync + Gluster)

Service Node(Libvirt/KVM + Open vSwitch)

Service Node(Libvirt/KVM + Open vSwitch)

Service Node(Libvirt/KVM + Open vSwitch)

Switching fabric(single stack)

Control node hosts:Firewalls CentOS/Debian Mirrors Kickstart generator + PXE Custom repositories Puppet (migrating to ansible) Backups (Data + Control VM)

Current deployed architecture

GlusterVM datastore

GlusterBackup Area

OVS Network

VM VM VM VM

VM VM VM VM

LIBVIRT

CoroSync/Pacemaker

Bootstrap& mgmt VM

Central management/Orchestration(PXE + Mirror + Puppet)

Control Nodes

<network connections='3'> <name>ovs-network</name> <uuid>c162d855-eae3-26e8-9cf4-27fffc10faa0</uuid> <forward mode='bridge'/> <bridge name='ovsbridge0' /> <mac address='52:54:00:CA:E5:4B'/> <virtualport type='openvswitch'/> <portgroup name='mgmt' default='yes'> <vlan> <tag id='10'/> </vlan> </portgroup> <portgroup name='publicnet'> <vlan> <tag id='1000'/> </vlan> </portgroup></network>

<interface type='network'> <mac address='52:54:00:49:95:c0'/> <source network='ovs-network' portgroup='publicnet'/> <model type='e1000'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </interface>

portgroup in libvirt

VLAN tag in VM

Open vSwitch in libvirt

98755c2a-d40d-4e5d-a8f7-e0bedeaca2fa Bridge "ovsbridge0" Port "ovsbridge0" Interface "ovsbridge0" type: internal Port "vnet0" tag: 10 Interface "vnet0" Port "vnet2" Interface "vnet2" Port ovsbond Interface "eth0" Interface "eth1" Port "vnet1" tag: 1000 Interface "vnet1" Port "vnet3" tag: 1000 Interface "vnet3" ovs_version: "2.1.3"

VLAN tags

Open vSwitch VLAN tag propagation

www.secure-pass.net

(Security Bonus Slide :-)

Q&A

31st January 2015Giuseppe Paternò@gpaterno