Orchestrating Last Privilege - Diogo Monica - DevOpsDays Tel Aviv 2016

Orchestrating Least Privilege

~2000 Today

What is an Orchestrator?

What is an Orchestra?

SWARM

Job of a Conductor

- Casting - Assign sheet music - Unify performers - Set the tempo

Job of an Orchestrator

- Node management - Task assignment - Cluster state reconciliation - Resource Management

What is a Least Privilege Orchestrator?

What is Least Privilege?

A process must be able to access only the information and resources that are necessary for its legitimate purpose.

Principle of Least Privilege

An Orchestrator that follows the principle of least privilege in the

strictest manner possible.Least Privilege Orchestrator

Why Least Privilege?

Cluster

Internet

Cluster

Internet

A

M

M M

AA A

M

M M

WW W

M

M M

WW W

How far away are we right now?

How do we achieve Least Privilege Orchestration?

Mitigating External Attacker

- Externally accessible service ports are explicitly defined

- Administration endpoints are authenticated and authorized

Mitigating External Attacker

web: image: web-app expose: 443 links: - redis redis: image: redis

Mitigating Internal Network Attacker

- Authentication of both network and cluster control-plane communication

- Service to service communication is authorized, with orchestrator managed ACLs

Mitigating Internal Network Attacker

[ { "permission": { "method": "GET", "resource": "/user" }, "allow": ["web", "fulfillment", "payments"] },

{ "permission": { "method": "POST", "resource": "/user" }, "allow": ["signup", "web"] },

{ "permission": { "method": "DELETE", "resource": "/user/.*" }, "allow": ["web"] }]

Mitigating MiTM Attacker

- All control and data-plane traffic is encrypted.

Mitigating MiTM Attacker

rails-app: image: rails-app links: - mysql mysql: image: mysql

MTLS

Mitigating Malicious Worker

‣Should only have access to resources currently in use ‣No ability to modify or access any

cluster state except their own. ‣Identity is assigned, never requested

Mitigating Malicious Worker

Push

Worker

Manager

WorkerWorker

Mitigating Malicious Manager

‣Disallow running arbitrary code ‣No access to secret material ‣No ability to spin up unauthorized

nodes/impersonate existing nodes. ‣No ability to read service-to-service

communication

Mitigating Malicious Manager

Worker

Manager

WorkerWorker

web: image: web-app expose: 443 links: - redis tls-auth: - OU: api-client redis: image: redis

web: image: web-app expose: 443 links: web:

image: web-app expose: 443 links:

web: image: web-app expose: 443 links:

SWARM

Mutual TLS by default

• First node generates a new self-signed CA.

Mutual TLS by default

• New nodes can get a certificate issued w/ a token.

Mutual TLS by default

• Workers and managers identified by their certificate.

Mutual TLS by default

• Communications secured with Mutual TLS.

The Token

SWMTKN-1-mx8susrv1etsmc8omaom825bet6-cm6zts22rl4hly2

Prefix to allow VCSsearches for leaked

Tokens

Token Version

Cryptographic Hashof the CA Root Certificate

for bootstrap

Randomly generatedSecret

Bootstrap

1. Retrieve and validate Root CA Public key material.

2. Submit new CSR along with secret token.

3. Retrieve the signed certificate.

Automatic Certificate Rotation

1. Submit new CSR using old key-pair.

2. Retrieve the new signed certificate.

Support for External CAs

• Managers support BYO CA.

• Forwards CSRs to external CA.

Secrets

• Secret is sent to a manager and stored encrypted at rest.

Secrets

• Secret gets securely replicated to other managers.

Secrets

• Workers get sent a secret only when they need it.

Secrets

• No access to secret from other workers.

Open Problems Opportunities.

Byzantine Consensus.

Signed Specs.

SWARMweb: image: web-app expose: 443 links: - redis tls-auth: - OU: api-client redis: image: redis

web: image: web-app expose: 443 links:

web: image: web-app expose: 443 links:

web: image: web-app expose: 443 links:

End-to-end Secrets.

SWARM

Thank you