Anynines - Building a European PaaS

Building a European Cloud

Mittwoch, 16. Oktober 13

European Cloud?

Mittwoch, 16. Oktober 13

Hungarian Cloud?

Mittwoch, 16. Oktober 13

Budapest Cloud?

Mittwoch, 16. Oktober 13

Your personal Cloud??

Mittwoch, 16. Oktober 13

The cloud is about sharing.

Mittwoch, 16. Oktober 13

Spare capacity,Virtualization,

Share spare capacityPay as you go

Mittwoch, 16. Oktober 13

So why not share globally?

Mittwoch, 16. Oktober 13

Privacy

Mittwoch, 16. Oktober 13

Any transfer of personal data of EU citizen to a non-EU

state with a lower data privacy level compared to EU

standards is prohibited.- Directive 95/46/EC

Mittwoch, 16. Oktober 13

EU Safe Harbor

Mittwoch, 16. Oktober 13

• is a EU directive

• regulates the processing of personal data within the European Union

Mittwoch, 16. Oktober 13

U.S. - EU Safe Harbor

Mittwoch, 16. Oktober 13

• Self(!)-certification process

• = swear to the United States Department of Commerce to comply to EU privacy laws

Mittwoch, 16. Oktober 13

A memo from the EU commision:

Mittwoch, 16. Oktober 13

"The Safe Harbour agreement may not be so safe after all."

European CommissionMEMO/13/710 19/07/2013

http://rh.gd/1hBKIrf

Mittwoch, 16. Oktober 13

Patriot Act

Mittwoch, 16. Oktober 13

"Uniting (and) Strengthening America (by) Providing Appropriate Tools

Required (to) Intercept (and) Obstruct Terrorism Act of 2001."

Mittwoch, 16. Oktober 13

• United States federal law

• Significantly enhanced and broadened federal government powers in the realm of 

• Electronic Surveillance 

• Anti-money laundering 

• Border Security, ...

Mittwoch, 16. Oktober 13

10 Titles of the Patriot Act

Mittwoch, 16. Oktober 13

• Title I: Enhancing domestic security against terrorism

• Title II: Surveillance procedures

• Title III: Anti-money-laundering to prevent terrorism

• Title IV: Border security

• Title V: Removing obstacles to investigating terrorism

• Title VI: Victims and families of victims of terrorism

• Title VII: Increased information sharing for critical infrastructure protection

• Title VIII: Terrorism criminal law

• Title IX: Improved Intelligence

• Title X: Miscellaneous          

Mittwoch, 16. Oktober 13

Patriot Actbeats

Safe Harbor

Mittwoch, 16. Oktober 13

Where security meets freedom

Mittwoch, 16. Oktober 13

The story oflavabit.com

Mittwoch, 16. Oktober 13

• Encrypted email service (*2004) by Ladar Levison

• Used by Edward Snowden

• Ordered to turn over its SSL private key

Mittwoch, 16. Oktober 13

Levison's was put to the decision: shutdown or “become complicit in

crimes against the American people”.

Mittwoch, 16. Oktober 13

Lavabit.com was shut down on August 8, 2013

Mittwoch, 16. Oktober 13

"This experience has taught me one very important lesson: without

congressional action or a strong judicial precedent, I would strongly

recommend against anyone trusting their private data to a company with

physical ties to the United States".- Ladar Levison, Lavabit.com

Mittwoch, 16. Oktober 13

• It's not about having data on European servers

• It's not about having a European company

Mittwoch, 16. Oktober 13

It‘s aboutstaying completely off any US provider and don‘t tie

to the US in person or with your company.

Mittwoch, 16. Oktober 13

Relying on open source software is a good choice, too.

Mittwoch, 16. Oktober 13

How to build a European cloud?

Mittwoch, 16. Oktober 13

Cloud Building

Mittwoch, 16. Oktober 13

Cloud,a term

that has beenoverdone

Mittwoch, 16. Oktober 13

IaaSPaaSSaaS

Mittwoch, 16. Oktober 13

IaaSPaaSSaaS

Mittwoch, 16. Oktober 13

A 2013 proposal for an open source based

Cloud

Mittwoch, 16. Oktober 13

Mittwoch, 16. Oktober 13

Hardware

Mittwoch, 16. Oktober 13

Hardware

Infrastructure as a Service (IaaS)

Servers, Network,Storage

Mittwoch, 16. Oktober 13

Hardware

Infrastructure as a Service (IaaS)

Servers, Network,Storage

PaaS (PaaS)

VMs, Network,Storage

Mittwoch, 16. Oktober 13

Hardware

Infrastructure as a Service (IaaS)

Servers, Network,Storage

PaaS (PaaS)

VMs, Network,Storage

Applications

CF API (deploy, scale, services, ...)

Mittwoch, 16. Oktober 13

Mittwoch, 16. Oktober 13

Hardware

Mittwoch, 16. Oktober 13

Hardware

OpenStack (IaaS)

Servers, Network,Storage

Mittwoch, 16. Oktober 13

Hardware

OpenStack (IaaS)

Servers, Network,Storage

Cloud Foundry (PaaS)

VMs, Network,Storage

Mittwoch, 16. Oktober 13

Hardware

OpenStack (IaaS)

Servers, Network,Storage

Cloud Foundry (PaaS)

VMs, Network,Storage

Applications

CF API (deploy, scale, services, ...)

Mittwoch, 16. Oktober 13

OpenStack

Mittwoch, 16. Oktober 13

OpenStack architecture

Mittwoch, 16. Oktober 13

Key-Stone

Mittwoch, 16. Oktober 13

Nova

Mittwoch, 16. Oktober 13

Glance

Mittwoch, 16. Oktober 13

Cinder

Mittwoch, 16. Oktober 13

Swift

Mittwoch, 16. Oktober 13

Neutron

Mittwoch, 16. Oktober 13

OpenStack provides usan IaaS ready to deploy

Cloud Foundry.

Mittwoch, 16. Oktober 13

Cloud Foundry

Mittwoch, 16. Oktober 13

• CF = large distributed system

• Inner shell vs. outer shell

• Bosh = Bosh outer shell > deploy CF

Mittwoch, 16. Oktober 13

SimplifiedCloud Foundry

Architecture

Mittwoch, 16. Oktober 13

Service(e.g. MySQL)

Services(e.g. MySQL)

Services(e.g. MySQL)

Services(e.g. MySQL)

RouterRouter

DEA

RouterHealth Manager

RouterCloud Controller

Cloud ControllerDatabase

Get desired states

Request droplet start/stop

DEADEADEADEA

Droplet / Service metadata

API request Droplet request

Droplet changenotifications

Droplet heartbeat & exit messages

Consume a service

Mittwoch, 16. Oktober 13

Cloud Controller

• Offers the CF API endpoint

• System authority for issuing commands

• Start apps

• Create service

• Binding services

Service(e.g. MySQL)

Services(e.g. MySQL)

Services(e.g. MySQL)

Services(e.g. MySQL)

RouterRouter

DEA

RouterHealth Manager

RouterCloud Controller

Cloud ControllerDatabase

Get desired states

Request droplet start/stop

DEADEADEADEA

Droplet / Service metadata

API request Droplet request

Droplet changenotifications

Droplet heartbeat & exit messages

Consume a service

Mittwoch, 16. Oktober 13

DEA

• droplet = dea.staging(app_code)

• Staging = executing buildpacks

• Warden

• Starts and runs dropletsService

(e.g. MySQL)Services

(e.g. MySQL)Services

(e.g. MySQL)Services

(e.g. MySQL)

RouterRouter

DEA

RouterHealth Manager

RouterCloud Controller

Cloud ControllerDatabase

Get desired states

Request droplet start/stop

DEADEADEADEA

Droplet / Service metadata

API request Droplet request

Droplet changenotifications

Droplet heartbeat & exit messages

Consume a service

Mittwoch, 16. Oktober 13

Health Manager

• compares desired system state with actual system state

• sends advice to CC

• CC actsService(e.g. MySQL)

Services(e.g. MySQL)

Services(e.g. MySQL)

Services(e.g. MySQL)

RouterRouter

DEA

RouterHealth Manager

RouterCloud Controller

Cloud ControllerDatabase

Get desired states

Request droplet start/stop

DEADEADEADEA

Droplet / Service metadata

API request Droplet request

Droplet changenotifications

Droplet heartbeat & exit messages

Consume a service

Mittwoch, 16. Oktober 13

Router

• knows on which DEAs your app instances are

• routes incoming requests to the right DEAs

Service(e.g. MySQL)

Services(e.g. MySQL)

Services(e.g. MySQL)

Services(e.g. MySQL)

RouterRouter

DEA

RouterHealth Manager

RouterCloud Controller

Cloud ControllerDatabase

Get desired states

Request droplet start/stop

DEADEADEADEA

Droplet / Service metadata

API request Droplet request

Droplet changenotifications

Droplet heartbeat & exit messages

Consume a service

Mittwoch, 16. Oktober 13

Services

• Create service = provision

• Bind = create credentials

• Apps bind to services

• Credentials as ENV variables

Service(e.g. MySQL)

Services(e.g. MySQL)

Services(e.g. MySQL)

Services(e.g. MySQL)

RouterRouter

DEA

RouterHealth Manager

RouterCloud Controller

Cloud ControllerDatabase

Get desired states

Request droplet start/stop

DEADEADEADEA

Droplet / Service metadata

API request Droplet request

Droplet changenotifications

Droplet heartbeat & exit messages

Consume a service

Mittwoch, 16. Oktober 13

What you get?

Mittwoch, 16. Oktober 13

Mittwoch, 16. Oktober 13

Questions?

Mittwoch, 16. Oktober 13

Thank you!

Mittwoch, 16. Oktober 13

Coderequire "fileutils"

require "find"

require "fog"

class Blobstore

  def initialize(connection_config, directory_key, cdn=nil, root_dir=nil)

    @root_dir = root_dir

    @connection_config = connection_config

    @directory_key = directory_key

    @cdn = cdn

  end

  def local?

    @connection_config[:provider].downcase == "local"

  end

  def exists?(key)

    !file(key).nil?

  end

  def download_from_blobstore(source_key, destination_path)

    FileUtils.mkdir_p(File.dirname(destination_path))

    File.open(destination_path, "w") do |file|

      (@cdn || files).get(partitioned_key(source_key)) do |*chunk|

        file.write(chunk[0])

      end

    end

  end

  def cp_r_to_blobstore(source_dir)

    Find.find(source_dir).each do |path|

      next unless File.file?(path)

      sha1 = Digest::SHA1.file(path).hexdigest

      next if exists?(sha1)

      cp_to_blobstore(path, sha1)

    end

  end

  def cp_to_blobstore(source_path, destination_key)

    File.open(source_path) do |file|

Mittwoch, 16. Oktober 13