AWS re:Invent 2016: Enterprise Fundamentals: Design Your Account and VPC Architecture for Enterprise Operating Models (ENT203)

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Shahbaz Alam – Manager, AWS Professional Services

Pawan Agnihotri – Principal, AWS Solutions Architect

Greg Dumont – Director of Technology, Nielsen

November 29, 2016

ENT203

Enterprise FundamentalsDesign Your Account and VPC Architecture

for Enterprise Operating Models

How do I

make

everybody

happy?

How do I

separate

production and

non-production?

Hmm…How

many accounts

/ VPCs /

subnets do I

need?

Do you even

know what others

are doing?

What We Hear From Customers

AWS Account and VPC Review

AWS Global Infrastructure

14Regions

38Availability Zones

AWS Region Overview

• Mesh of Availability Zones (AZ) and

Transit Centers

• Redundant paths to transit centers

• Transit centers connect to:

• Private links to other AWS Regions

• Private links to customers

• Internet through peering and paid

transit

• AZs within a region are connected to

be < 2ms apart (usually < 1ms)

AZ

AZ

AZ AZ AZ

Transit

Transit

AWS Availability Zone Overview

• Regional cluster of discrete data centers

(DCs)

• Separate redundant power, networking,

connectivity and facility

• Each region has 2 or more AZs

• Each AZ is comprised of 1 or more DCs

• No data center spans two AZs

• Some AZs have as many as 6 DCs

• DCs within an AZ are connected to be

less than ¼ ms apart

AZ

AZ

AZ AZ AZ

Transit

Transit

AWS Data Center Overview

• Single DC typically has over 50,000

servers (often over 80,000 servers)

AWS Virtual Private Cloud (VPC) Overview

• Your own logically isolated section of the

Amazon Web Services (AWS) Cloud

• You have complete control over your virtual

networking environment

• Proven and well-understood networking

concepts:

− User-defined IP address range

− Subnets

− Route Tables

− Access Control Lists

− Network Gateways

Select a Region Within Your AWS Account

AWS Region

Create Your VPC

VPC CIDR: 10.1.0.0 /16

AWS Region

Select Your Availability Zones

Availability Zone A Availability Zone B

VPC CIDR: 10.1.0.0 /16

AWS Region

Create Your Subnets

Availability Zone A Availability Zone B

VPC CIDR: 10.1.0.0 /16

AWS Region

Subnet (10.1.1.0 / 24) Subnet (10.1.2.0 / 24)

AWS Account Properties Overview

Security Boundary

• Any and all access granted is limited only to users, groups, and/or resources created and managed within the specified account

• All data stored within an account is controlled and managed only by the security policies of that account

Resource Containment

• Resources created within an account are limited to that specific account (i.e., cannot span multiple accounts)

• Resources cannot dynamically migrate from one account to another

• AWS resources are constrained by hard and soft limits per account

Financial Responsibility

• Billing and financial details (including tagging) are defined and controlled per account

• Reserved Instances and volume discounts are calculated at the account level

• Trusted Advisor analysis is conducted at the account level

Multiple AWS accounts may be used for the following governance reasons:

IT Operating Models

Coordination• Unique business units

servicing a common customer base

• Key IT Capability: access to shared data, through standard technology interfaces

Bu

sin

ess P

rocess In

teg

ration

Business Process StandardizationLOW HIGH

LO

WH

IGH

Unification• Operate as a single business

with global processes, standards, and global data access

• Key IT Capability: enterprise systems reinforcing standard processes and providing global data access

Diversification• Independent business units

with different customers and expertise

• Key IT Capability: provide economies of scale without limiting independence

Replication• Independent business units

but similar business units sharing best practice

• Key IT Capability: provide standard infrastructure and application components for global efficiencies

© MIT Sloan Center for Information Systems Research.

Source: Enterprise Architecture as Strategy, Creating a Foundation for Business Execution, J. Ross, P. Well, D. Robertson, HBS Press 2006

IT Operating Models – Unification




Bu

sin

ess P

rocess In

teg

ration


LO

WH

IGH


with global processes, standards, and global data access





Replication• Independent but similar

business units sharing best practice




On-Premises IT Infrastructure

Pattern 1: Unification Operating Model –

Business Setup

IT Organization Setup

CIO

CISOInfra /

NetworkOperations Development Help Desk

Key Distinguishing Features

• Single technology leader

• Shared infrastructure and operations

• Data shared across organization

• Shared financial modelSingle Data Center

DEV LAN QA LAN UAT LAN PROD LAN

Pattern 1: Unification Operating Model – Key

Business Requirements

?

Centralized management

and centralized IT decisions

Standardized IT processes across the company

Shared infrastructure

and application data


Baseline AWS Architecture Design

Dev Private Subnet

Non Production VPC

AWS Account

QA Private Subnet UAT Private Subnet Prod Private Subnet

Corporate data center

Key AWS Design Elements

• Single account

• Security federation via LDAP/AD or native

AWS Identity & Access Management (IAM)

• Centralized IT teams responsible for IAM

Consolidated Billing Account

Dev Public Subnet QA Public Subnet UAT Public Subnet Prod Public Subnet

Production VPC


AWS Design ImplicationsSecurity

• Can leverage existing security

processes and controls to

manage AWS Cloud

infrastructure

• Ability to control your blast radius

solely based on AWS IAM,

Security Groups, and Network

Access Control Lists (NACLs)

• Complex IAM controls required

to support segregation of duties

Operational

• Aligned to existing data center

concept, which may ease

transition into cloud

• Simplified infrastructure management and connectivity options

• Higher chance of reaching

account limits quickly

Financial

• Cost allocation tagging must

occur at the workload or

application level

• Easier to use AWS Cost Explorer

to associate costs back to

business

• Budgeting and forecasting may

requires coordination between

multiple teams

IT Operating Models – Coordination




Bu

sin

ess P

rocess In

teg

ration


LO

WH

IGH


with global processes, standards and global data access

• Key IT Capability: enterprise systems reinforcing standard processes, and providing global data access










Pattern 2: Coordination Operating Model –

Business Setup


Key Features

• Single technology leader for overall company

• Shared infrastructure and network across multiple lines of

business (LOB)

• Data shared across LOB to cross-sell products to the same

customer base

• Development and operations teams sit within each

respective LOBSingle Data Center


LOB 1

LOB 2

CIO

CISOInfrastructure

/ NetworkLOB 1 IT Director

Development

Operations

LOB 2 IT Director

Development

Operations

Help Desk


Key Business Requirements

?Share customer and/or product

data

Unique lines of business (LOB) have separate

application requirements

Standardized IT processes by

LOB

Application decisions made

by LOB

Shared infrastructure



LOB 1 NON-PROD

Non-Production Account

Core Services (Optional)

LOB 2 NON-PROD PROD

Corporate Data Center


• Single consolidated billing account

• Separate accounts for Production and

Non-Production

• Security federation via LDAP/AD or

native IAM

• Application development teams

working with role-based permissions in

Non-Production

• Potential to share services by using

VPC peering


Core Services (Optional)

Production Account

Subnet Subnet Subnet Subnet

Subnet Subnet

Public Subnet Private Subnet


AWS Design Implications

Security

• Easy separation of environment:

by Production and Non-

Production

• Ability to control connectivity to

on-premises using existing

security tools (i.e., firewalls)

• Network and user access

separation between Production

and Non-Production by account

Operational

• Increased complexity of network

routing, peered VPCs, and

corporate connectivity

• Need to federate into multiple

AWS accounts

• Standardized production

environment

Financial

• Marginal increase in cost as a

result of VPC peering

• Need to tag resources for cost

allocation

• Budgeting and forecasting may

requires coordination between

multiple teams

IT Operating Models – Diversification




Bu

sin

ess P

rocess In

teg

ration


LO

WH

IGH



• Key IT Capability: enterprise systems reinforcing standard processes, and providing global data access









On Premises IT Infrastructure

Pattern 3: Diversification Operating Model –

Business Setup


Key Features

• Multiple / distinct lines of business (LOB) across the

company, each with their own leadership

• Each LOB has its own technology leader, technology

teams, and technology assets

• Every LOB employs their own standards and practice

• No data is shared across the company

LOB 1 Data Center


LOB 2 Data Center


LOB 3 Data Center


CEO

LOB 1 CEO

LOB1 CIO

CISOInfra /

NetworkDevelopment Operations

LOB 2 CEO

LOB 2 CIO

CISOInfra /

NetworkDevelopme

ntOperations



?Little to nosharing of data

Each lines of business has

separate application

requirements

Each line of business makes all

application decisions

Each line of business has

different financial

structures

No standard IT processes by

line of business

No shared infrastructure




• Multiple accounts with multiple VPCs


IAM and separated by line of business

• Application IT teams working with role-

based permissions for seamless

infrastructure managementLOB 1

NON PROD

LOB 1

PROD

LOB 2

NON PROD

LOB 2

PROD

LOB 1 Data Center LOB 2 Data Center

LOB 1 Consolidated Billing Account LOB 2 Consolidated Billing Account

LOB 1

NON PROD VPC

LOB 1

PROD VPC

LOB 2

NON PROD VPC

LOB 2

PROD VPC

Subnet

Subnet

Subnet

Subnet

Subnet Subnet

Subnet Subnet



Security

• Able to delegate access control

by LOB

• Easy separation of environments

and applications, thus limiting

the blast radius

• Network isolation is based on

VPC boundaries

Operational

• Easily able to scale by adding

accounts and/or VPCs

• Increased difficulty in network

routing configuration between

on-premises and AWS

• Risk of not standardizing across

LOBs

Financial

• Ability to use Detailed Billing

Reports to gain a granular view

for each LOB

• Each LOB is responsible to

manage their own budget and

forecast

• No consolidated view of overall

financial footprint

IT Operating Models – Replication



• Key IT Capability: Access to shared data, through standard technology interfaces

Bu

sin

ess P

rocess In

teg

ration


LO

WH

IGH













Pattern 4: Replication Operating Model – Business

Setup


Key Features

• Shared service model with shared infrastructure and

network across multiple lines of business (LOB)

• Little to no data shared across LOBs

• Development and Operations teams sit within each

respective LOB

• LOBs share best practices but are not centrally

managedMultiple Data Centers

LOB 1 LAN LOB 2 LAN LOB 3 LAN

SHD SVC

LAN

NON PROD

CEO

LOB 1 CEO

LOB1 CIO

Development

Operations

LOB 2 CEO

LOB 2 CIO

Development

Operations

CIO Shared Services

CISOInfra /

NetworkOperations

NON PROD NON PROD NON PROD

PROD PROD PROD PROD

Pattern 4: Replication Operating Model –


?Little to no sharing

of data

Unique lines of business have

separate application requirements

IT processes and infrastructure

standardized across the company via shared services

model

Standardized data definitions and

structures but data maintained by LOB




• Multiple accounts with multiple VPCs


IAM and separated by line of business

• Application IT teams working with role-

based permissions for seamless

infrastructure management

• Potential to share core services using VPC

peering

LOB 1

NON PROD

Shared Services VPC

LOB 1 PROD LOB 2 NON PROD LOB 2 PROD

Corporate Data Center


Subnet

Subnet

Subnet

Subnet

Subnet

Subnet

Subnet

Subnet



Security

• Separate network routing for each

LOB and environment

• Easy separation of environments

and applications, thus highly limiting

the blast radius

• Able to delegate access control and

VPC configuration to different

application teams within and across

LOBs

Operational

• Able to scale by adding Accounts

and VPCs

• Increased complexity with network

configuration

• Standardized templates and

configuration management can be

leveraged and reused across LOBs

Financial

• Ability to separate non-production and production spend by cost center

• Provide financial accountability by LOB via discreet AWS accounts

• Centralized financial view, centralized volume discounts for cost optimization through consolidated billing

Greg Dumont – Director of Technology

(a.k.a. The Cloud CEO)

The premier market research company that provides a comprehensive

understanding of what consumers watch and buy.

Who Are Nielsen?

$100+

countries

44,000

employees

$6.2B

revenue

5.9B

consumers

25M

stores

Nielsen – More than just TV

Why We Selected AWS

Pay as you go

Elasticity

Agility

Experimentation

GlobalStandards

What We Use

Amazon

EC2

Amazon ECS

AWS

Lambda

Elastic Load

Balancing

Amazon

CloudFront

Amazon

Glacier

Amazon

S3

Amazon

DynamoDB

Amazon

RDS

Amazon

Redshift

AWS Direct

Connect

Amazon

Route 53

AWS

CloudFormation

AWS IAMAWS KMS

Amazon

Elasticsearch

Service

Amazon

EMR

Amazon

Kinesis

Amazon

QuickSightAmazon

SQS

Amazon

SWF

Org Structure/Network Structure


CEO

CTO

Watch CTO

Development

Tech Strategy and Delivery

Engineering CTO

Development

Buy CTO

Development

Service Delivery

eXelate CTO

Development

CIO

Infrastructure

Corporate Platforms

CISO

Corporate Security

Key Features

• Single technology leader accountable to CEO

• Technology leaders by business vertical

• Shared infrastructure and corporate platforms

• Data shared across organization

• CTO funding allocated by LOB

• 22,000 servers, 100 storage arrays

• 10,000 network devices

• 213 offices

Multiple Data Centers

LOB 1 LAN LOB 2 LAN LOB 3 LAN

SHD SVC

LAN

NON PROD NON PROD NON PROD NON PROD

PROD PROD PROD PROD

On Premises IT Infrastructure

Nielsen AWS Account Structure

Advantages

• Limited blast radius between Production and

Development environments

• LOBs “control their own destiny” by having individual

accounts

• Consolidated master ensure all of Nielsen benefits

from discounts and Reserved Instance purchases

• Internal network connectivity can be shared across

accounts

• Financial accountability by LOB

Disadvantages

• Duplication of effort across accounts (VPCs, roles &

security policies, logging, etc.)

• More upfront work to allocate IP ranges between

cloud and on-premises

• Divergence at account level could lead to lack of

standardizationNon-Production

Accounts

Watch and

Engineering

Buy

Excelate

Shared Services

Nielsen Consolidated

Account

Production

Accounts

Watch and

Engineering

Buy

Excelate

Shared Services

Our Network and VPC Design

Availability Zone 1 (US-East-1a)

Application

DataStore

Web – External

HADOOP | RDS | PostGres | EnterpriseDB

Tomcat | Java | Docker | Sencha |

HazelCast

Apache | IIS

One VPC per account

(Watch Prod, Watch Non-Prod, etc.)

VPC Subnet (Private)


VPC Subnet (Public)

Security Group

E

L

B

Availability Zone 2 (US-East-1b)

Application

DataStore

Web – External

HADOOP | RDS | PostGres | EnterpriseDB

Tomcat | Java | Docker | Sencha |

HazelCast



VPC Subnet (Public)

Security Group

Internet

Gateway

East Region

Directory ServiceIAM

Data

Encryption

Keys

Nielsen Lebanon Data Center

Directory ServiceIAMData

Encryption

Keys

Nielsen Tampa Data CenterNielsen “CSP”

MPLS Network

Internet

AWS Direct Connect10Gb/sec 10Gb/sec

Nielsen “Global”

MPLS Network

Putting it All Together

1. Understand your current IT environment

2. Determine which IT operating model maps closest to your

current set-up

3. Understand your propensity to update, change, or

maintain your IT operating model

4. Use one of the patterns as the baseline architecture

design and customize as needed based on requirements

5. When in doubt – default to Pattern 3 (Diversification)

Thank you!

Remember to complete

your evaluations!