Docker: Testing to Production

Docker: TestingDocker: Testingto Produtionto Prodution

Edwin FuquenEdwin [email protected]

@efuquen

About MeAbout MeQueens, NY => Florida => Queens, NYUniversity of Florida 2004 to 2009Grooveshark, Livestream, Getty Images, Bloomberg,and GoogleBackend Development and Infrastructure

Server Administration to Distributed SystemsSome Frontend (mostly personal)Python, JS/Node, Scala, Java

Managing aManaging aDatacenterDatacenter

Mid 90's to early 2000's.Expensive equipmentSpecialized knowledgeTime consumingSlow to increase capacityNot very fun

VirtualVirtualMachineMachine

Fully mimics an OSSecurely isolated from eachotherMany virtual servers on thesame bare metalLarge pool of servers can many more VMsEasily balance workloads

The Problem(s)The Problem(s)VMs need to be configured, complicatedAs software still heavy weight and slowNot good for application deploymentWe solve the Ops problem, not the Dev one

Containers: A solutionContainers: A solutionA process runs in isolation but with the same OSDoes not mimic an entire machineDone via two mechanisms

Namespaces - per process resource isolationCgroups - per process resource management

This provides a completely separate environment foran application without the weight of a virtual machine

DockerDockerUser friendly commandline interface tocontainersDockerfile - Rulesdescribe what goes in acontainerLayered File System -applies rules to FS,saving final imageDaemon - Tracksrunning containers andimages

DockerDockerUser friendly commandline interface tocontainersDockerfile - Rulesdescribe what goes in acontainerLayered File System -applies rules to FS,saving final imageDaemon - Tracksrunning containers andimages

https://github.com/efuquen/https://github.com/efuquen/

fullstack-dockerfullstack-docker

Docker ArchitectureDocker Architecture

CoreOSCoreOSLinux OS based on Gentoo distribution.No package manager and few preinstalled tools.The most essential being docker, etcd, & fleetd.An OS fully built around managing containers in adistributed, fault tolerant cluster of machines.

etcdetcdA distributed key/valuestore.Meant for config data, nothigh latency/throughput.Strongly consistent, veryreliable.

fleetfleetUses etcd as distributedconfig store.Runs distributed serviceson many nodes.Uses standard linuxservice files, but withsome customized options

Load BalancerLoad BalancerApplication ip & portare registered in etcd.confd gets notifiedwhen certain keys inetcd are modifiedThen haproxy.cfgtemplates get updatedwith added or removedapplication ip & port.

https://github.com/efuquen/docoreoshttps://github.com/efuquen/docoreos

-express-demo-express-demo

Production ProblemsProduction ProblemsMany competing cluster/cloud solutionsDocker, Layered Filesystems, and kernel features allvery new. Still maturing and changing rapidly.Logging is a mess.Security

No isolation like VMsMistakenly store sensitive information in images.Daemon requires privileged control.Community has historically not focused on it.

The FutureThe FutureStandards: appc, runc, and the open containerinitiative.

Will allow container alternatives.Docker Compose for production deployments withSwarmAll the issues mentioned are actively being worked onby Docker & the community.

Questions?Questions?