AWS re:Invent 2016: Extending Datacenters to the Cloud: Connectivity Options and Considerations for Hybrid Environments (NET305)

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Benjamin Feldon, Solutions Architect, AWS

Sidhartha Chauhan, Solutions Architect, AWS

November 30, 2016

Extending Data Centers to the Cloud

Connectivity Options and Considerations for Hybrid

Environments

NET305

Hybrid environments

Intro to VPN & AWS Direct Connect

Connectivity architectures

What are we connecting to?

What to expect from this session

What are the options for connecting into AWS?

What is appropriate for my workloads?

How can I start small and grow with time?

What is the meaning of life? (optional)

Key takeaways

Connecting to AWS

Hybrid environments

Hybrid connectivity

CORP

Hybrid connectivity – split architecture

CORP

Web App DB

Hybrid connectivity - data streams / replication

CORP

DB

DB

DB

DB

Hybrid connectivity – storage / backup / archive

CORP

S3

DB

App

Archive

Hybrid connectivity – virtual desktops

CORP

Amazon

WorkSpaces

DB

App

Hybrid connectivity – disaster recovery

CORP

DB

App

App

Hybrid connectivity

CORP

Intro to VPN

• VPC != VPN

• IPsec authentication & encryption

• AWS options

• AWS Managed VPN

• Software VPN (EC2)

Intro to AWS Direct Connect (DX)

• Offered since 2011

• Private connection, separate from Internet

• Consistent network experience

• Connect through one of 40 locations

• Each connection connects into 1 AWS Region

• Multiple options for each AWS Region

Oregon

N. California

AWS Direct Connect (DX) in the United States

SuperNAP

Equinix SE

CoreSite LA

N. Virginia

CoreSite NY

Equinix DC

CoreSite SV

OhioEquinix CH

QTS Chicago

Equinix DA

CoreSite VA

Equinix LA

Equinix SV

TierPoint

EdgeConneX

Pittock Block

Frankfurt

AWS Direct Connect (DX) in Europe and Asia Pacific

Digital RealtyEircom Interxion Frankfurt

Sydney

Ireland

Tokyo

Singapore

Equinix OS

Beijing

Equinix TY

Equinix FR

Equinix SY

Global Switch

Equinix SG

CIDS

Sinnet

Eqinix LDInterxion

Interxion Madrid

Interxion Stockholm

Equinix AM

Global Switch

Mumbai

GPXSify Rabale

Seoul

KINX

Telehouse

Connectivity architectures

Connectivity architecturesCORP

Connectivity architecturesCORP

Connectivity architecturesCORP

VPC

VPC

VPC

VPC

VPC

VPC

Connectivity architectures

CORP

Internet

Connectivity architectures

VPN

VPC

VPC

VPC

AWS managed VPN

CORP

Internet

VPC

VPC

VPC

AWS managed VPN

CORP

Internet

Customer

Gateway

Virtual Private

Gateway

VPC

VPC

VPC

AWS managed VPN

CORP

Internet

Customer

Gateway

VGW

VPC

VPC

VPC

AWS managed VPN

CORP

Internet

VGW

CGW

VPC

VPC

VPC

AWS managed VPN

CORP

Internet

VGW

CGW

• AES-256

• SHA-2

• Phase 1 DH groups - 2, 14-18, 22, 23, and 24.

• Phase 2 DH groups - 1, 2, 5, 14-18, 22, 23, and

24.

• NAT-T

VPC

VPC

VPC

AWS managed VPN

CORP

Internet

VGW

CGW

VPC

VPC

VPC

AWS managed VPN

CORP

Internet

VGW

CGW

23.22.66.xx

50.16.172.yy

VPC

VPC

VPC

AWS managed VPN

CORP

Internet

VGW

CGW

23.22.66.xx

50.16.172.yy

VPC

VPC

VPC

AWS managed VPN

CORP

Internet

VGW

CGW

23.22.66.xx

50.16.172.yy

VPC

VPC

VPC

AWS managed VPN

CORP

Internet

VGW

CGW

1 VPN Connection = 2 VPN tunnels

23.22.66.xx

50.16.172.yy

VPC

VPC

VPC

AWS managed VPN, 2 X CGW

CORP

Internet

VGW

CGW

VPC

VPC

VPC

AWS managed VPN, 2 X CGW

CORP

Internet

VGW

CGW

CGW

VPC

VPC

VPC

AWS managed VPN, 2 X CGW

CORP

Internet

VGW

CGW

CGW

VPC

VPC

VPC

AWS managed VPN, 2 X CGW

CORP

Internet

VGW

CGW

CGW

VPC

VPC

VPC

AWS managed VPN, 2 X CGW

CORP

Internet

VGW

CGW

CGW

2 VPN Connections = 4 VPN tunnels

VPC

VPC

VPC

AWS managed VPN, 2 X CGW

CORP

Internet

VGW

CGW

CGW

VPC

VPC

VPC

AWS managed VPN, 2 X CGW

CORP

Internet

VGW

CGW

CGW

VPC

VPC

VPC

AWS managed VPN, 2 X CGW

CORP

Internet

VGW

CGW

CGW

VPC

VPC

VPC

AWS managed VPN, 2 X CGW

CORP

Internet

VGW

CGW

CGW

VPC

VPC

VPC

AWS managed VPN, 2 X CGW

CORP

Internet

VGW

CGW

CGW

VPC

VPC

VPC

AWS managed VPN, multiple VPCs

CORP

Internet

VGW

CGW

CGW

VGW

VPC

VPC

VPC

AWS managed VPN, multiple VPCs

CORP

Internet

VGW

CGW

CGW

VGW

VPC

VPC

VPC

AWS managed VPN, multiple VPCs

CORP

Internet

VGW

CGW

CGW

VGW

2 VPCs X 2 CGWs = 8 VPN tunnels

VPC

VPC

VPC

AWS managed VPN, multiple VPCs

CORP

Internet

VGW

CGW

CGW

VGW

2 VPCs X 2 CGWs = 8 VPN tunnels

AWS managed VPN

Cost

Performance

Flexibility

Resiliency

• Easy install, minutes to setup

• NAT-T, AES-256, SHA-2 and latest

DH groups

• Static (1 prefix) or BGP (<100

prefixes)

• Repeat for every VPC

• $0.05 per VPN connection hour

• Data transfer

• Leverage both VGW endpoints (2

tunnels per VPC)

• Think about CGW redundancy (4

tunnels per VPC)

• Multi Gbps can be achieved per

VPC (limited at VGW)

VPC

VPC

VPC

Software VPN (EC2)

CORP

Internet

VPC

VPC

Software VPN (EC2)

CORP

Internet

VPC

VPC

Software VPN (EC2)

CORP

Internet

VPC

VPC

Software VPN (EC2)

CORP

Internet

VPC

VPC

Software VPN (EC2)

CORP

Internet

VPC

VPC

Software VPN (EC2)

CORP

Internet

London DX

Seattle DX

Branch

Remote

workforce

Global HQ

Regional HQ

us-west-2 region

Transit VPCVPC

VPC

eu-west-1 region

Transit VPC VPC

VPC

ap-northeast-1

region

Transit VPC VPC

VPC

VPN

VPN

Transit VPC Global VPN

backbone

https://aws.amazon.com/answers/networking/transit-vpc/

Software VPN (EC2)

Cost

Performance

Flexibility

Resiliency

• Any open-source or commercial vendor

• Opens up proprietary feature sets

• Customer responsible for HA and scaling

• Advanced solutions can be built using

automation

• Vendor licensing

• EC2 hourly cost

• High availability cost

• Data transfer

• VPC endpoint HA achieved by

additional EC2 instance in 2nd AZ

• Customer-side HA also recommended

• Defined by EC2 instance size & type

• Multi Gbps can be achieved per

VPN instance (for all tunnels)

• Multiple instances for the same VPC

are possible

Connectivity architectures

AWS Direct Connect (DX)

Direct Connect

CORP

Internet

VPC

VPC

VPC

Direct Connect

CORP

Internet

DX Location

VPC

VPC

VPC

Direct Connect

CORP

Internet

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Direct Connect – physical connectivity

Direct Connect – physical connectivity

1) Customer presence in the same DX location

Direct Connect – physical connectivity

1) Customer presence in the same DX location

2) Circuit between customer data center and DX location

Direct Connect – physical connectivity

1) Customer presence in the same DX location

2) Circuit between customer data center and DX location

3) Service provider network extending to DX location

1) Customer router in colo

CORP

Internet

AWS Direct

Connect

Routers

Customer

Router

Colocation

DX Location

`

VPC

VPC

VPC

AWS Direct Connect

Letter of Authorization and Connecting Facility Assignment

Please consider this letter as notification for connecting facility assignment for the purpose of

establishing or augmenting connectivity between the parties identified above. This document authorizes

a connection to the ports indicated above. All charges for the physical connection are the sole

responsibility of company.

For location specific information on requesting a cross-connect, visit the "Requesting Cross-Connects"

section of the user guide:

http://docs.aws.amazon.com/DirectConnect/latest/UserGuide/Colocation.html

The requester(s) use of AWS services will be governed by the terms of the AWS Customer Agreement

(available at http://aws.amazon.com/agreement), or a separate agreement between the requester(s)

and AWS.

EXPIRATION NOTICE The authorized connectivity must be completed within 90 days of this LOA-CFA's

issue date or this LOA-CFA will expire.

* Amazon Corporate LLC is a subsidiary of Amazon.com, Inc.

Issue Date .

Oct 13, 2016

Issued By* .

Amazon Web Services Spain S.L.

Facility - Meet Me Room .

Interxion MAD2 – MAD2.211

Customer Demarcation/ZSide .

Rack: R77B1.R99B09 Patch Panel: PP2:SOUTH Strands: 40818

Requested By .

Company requesting name

Issued To .

Interxion, Madrid, ESP

Connection ID ..

MAD50_Test

Optic and Connector Types ..

1000BASE-LX Single Mode Fiber (SMF) Lucent Connector (LC)

Letter of Authorization

and Connecting

Facility Assignment

1) Customer router in colo

CORP

Internet

AWS Direct

Connect

Routers

Customer

Router

Colocation

DX Location

`

VPC

VPC

VPC

2) Partner-built circuit

CORP

Internet

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

APN Partners supporting AWS Direct Connect

https://aws.amazon.com/directconnect/partners/

2) Partner-built circuit

CORP

Internet

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

3) Service provider network

CORP

Internet

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

3) Service provider network

CORP

Internet

AWS Direct

Connect

Routers

DX Location

Service Provider

Network

VPC

VPC

VPC

3) Service provider network

CORP

Internet

AWS Direct

Connect

Routers

DX Location

Service Provider

Network

VPC

VPC

VPC

DX physical connectivity considerations

AWS account that owns the DX port?

Adding/removing virtual interfaces?

Routing ownership?

End-to-end costs?

Direct Connect – physical connectivity

1) Customer presence in the same DX location

2) Circuit between customer data center and DX location

3) Service provider network extending to DX location

Direct Connect – physical connectivity

1) Customer presence in the same DX location

2) Circuit between customer data center and DX location

3) Service provider network extending to DX location

Customer’s AWS account, interface control, routing. Cost: port + data transfer

Direct Connect – physical connectivity

1) Customer presence in the same DX location

2) Circuit between customer data center and DX location

3) Service provider network extending to DX location

Customer’s AWS account, interface control, routing. Cost: port + data transfer

Same as #1; add circuit cost. Sub 1-Gig can create only 1 virtual interface

Direct Connect – physical connectivity

1) Customer presence in the same DX location

2) Circuit between customer data center and DX location

3) Service provider network extending to DX location

Customer’s AWS account, interface control, routing. Cost: port + data transfer

Same as #1; add circuit cost. Sub 1-Gig can create only 1 virtual interface

Depends on provider’s offering

Direct Connect – physical connectivity

1) Customer presence in the same DX location

2) Circuit between customer data center and DX location

3) Service provider network extending to DX location

Customer’s AWS account, interface control, routing. Cost: port + data transfer

Same as #1; add circuit cost. Sub 1-Gig can create only 1 virtual interface

Depends on provider’s offering

Direct Connect cost considerations

Port hour + data transfer

Data in $0; data out differs by region

Factor in circuit costs

Calculate data center Internet costs (VPN)

Connectivity architectures

Direct Connect – resiliency

Direct Connect

CORP

Internet

AWS Direct

Connect

Routers

Customer

Router

Colocation

DX Location

`

VPC

VPC

VPC

Direct Connect

CORP

Internet

AWS Direct

Connect

Routers

Customer

Router

Colocation

DX Location

`

VPC

VPC

VPC

Direct Connect + VPN

CORP

Internet

AWS Direct

Connect

Routers

Customer

Router

Colocation

DX Location

`

VPC

VPC

VPC

2 X DX ports

CORP

Internet

Customer

Router

Colocation

DX Location

`

AWS Direct

Connect

Routers

VPC

VPC

VPC

Internet

2 X DX ports, 2 X customer routers

CORP

Colocation

DX Location

`

AWS Direct

Connect

Routers

Customer

Routers

`

VPC

VPC

VPC

2 X DX ports, 2 X circuits

CORP

Internet

DX Location

AWS Direct

Connect

Routers

VPC

VPC

VPC

CORP

Internet

DX Location

AWS Direct

Connect

Routers

2 X DX ports, 2 X circuits into 2 data centers

VPC

VPC

VPC

2 X DX, active/active

CORP

Internet

DX Location

AWS Direct

Connect

Routers

10 Gbps active

10 Gbps active

20 Gbps

VPC

VPC

VPC

2 X DX, active/standby

CORP

Internet

DX Location

AWS Direct

Connect

Routers

10 Gbps standby

10 Gbps

10 Gbps active

VPC

VPC

VPC

2 X DX, active/active

CORP

Internet

DX Location

AWS Direct

Connect

Routers

10Gbps Active

10 Gbps active

10 Gbps

VPC

VPC

VPC

Facility failure

CORP

Internet

DX Location

AWS Direct

Connect

Routers

VPC

VPC

VPC

2 X DX, 2 X DX locations

CORP

Internet

Customer

Routers

Colocation

DX Location 1

`

Customer

Routers

Colocation

DX Location 2

`

AWS Direct

Connect Routers

AWS Direct

Connect Routers

VPC

VPC

VPC

2 X DX, 2 X DX locations

CORP

Internet

Customer

Routers

Colocation

DX Location 1

`

Customer

Routers

Colocation

DX Location 2

`

AWS Direct

Connect Routers

AWS Direct

Connect Routers

VPC

VPC

VPC

VPN backup

CORP

Internet

Customer

Routers

Colocation

DX Location 1

`

Customer

Routers

Colocation

DX Location 2

`

AWS Direct

Connect Routers

AWS Direct

Connect Routers

VPC

VPC

VPC

Direct Connect (DX)

Cost

Performance

Flexibility

Resiliency

• 14 AWS regions, 40 POPs worldwide

• LOA provided within up to 72 hours

• Lead time of circuit build-out could take

weeks

• Port hours

• Data out transfer

• Service provider circuit / MPLS

• Colo cage (if applicable)

2 x DX in 2 locations + VPN

2 x DX in 2 separate locations

2 x DX in 1 DX location

DX + VPN

DX

• 1 Gbps or 10 Gbps ports

• 100, 200, 300, 400 or 500 Mbps

ports available through partners

• Equal-cost multipath via BGP means

2x10 G = 20 Gbps

Adapting the architecture

• Start with 1 AWS managed VPN

• Use VPN while DX is being built out

• Port hour charges begin when DX is up or 90 days

• DX is favored over VPN when both exist

• Add additional DX ports for resiliency / bandwidth

• Plan for failure, including facility failure

• Control traffic flow both ways using BGP and routing

• Raise support cases with AWS with any questions

Connectivity architectures

What are we connecting to?

Internet

CORP

AWS Direct

Connect

Routers

Customer

Router

Colocation

DX Location

`

VPC

VPC

VPC

Multiple VPCs

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

Multiple VPCs

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

Prod

Test

Dev

Multiple VPCs

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

Prod

Test

Dev

VPC

VPC

VPC

Non

-Pro

dP

rod

Multiple VPCs – VPC Peering

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

Prod

Test

Dev

VPC

VPC

VPC

Non

-Pro

dP

rod

VPC Peering

Multiple VPCs – VPC Peering

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

Prod

Test

Dev

VPC

VPC

VPC

Non

-Pro

dP

rod

VPC Peering

Multiple VPCs – VPC Peering

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

Prod

Test

Dev

VPC

VPC

VPC

Non

-Pro

dP

rod

VPC Peering

Connecting to VPC over DX

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

Prod

Test

Dev

VPC

VPC

VPC

Non

-Pro

dP

rod

Private virtual interface

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

Prod

Test

Dev

VPC

VPC

VPC

Non

-Pro

dP

rod

Private virtual interface

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

Prod

Test

Dev

VPC

VPC

VPC

Non

-Pro

dP

rod

VLAN

400

Private virtual interface

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

Prod

Test

Dev

VPC

VPC

VPC

Non

-Pro

dP

rod

VLAN

400

BGP

BGP

Private virtual interface

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

Prod

Test

Dev

VPC

VPC

VPC

Non

-Pro

dP

rod

VLAN

500

VLAN

400

Private virtual interface

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

Prod

Test

Dev

VPC

VPC

VPC

Non

-Pro

dP

rod

VLAN

500

VLAN

400

VLAN

600

Private virtual interface

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

Prod

Test

Dev

VPC

VPC

VPC

Non

-Pro

dP

rod

BGP

BGP

BGP

BGP

BGP

Access to VPC resources

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

Prod

Test

Dev

VPC

VPC

VPC

Non

-Pro

dP

rod

Access to VPC resources

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

Prod

Test

Dev

VPC

VPC

VPC

Non

-Pro

dP

rod

Access to VPC resources

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

Prod

Test

Dev

VPC

VPC

VPC

Non

-Pro

dP

rod

VPC Peering

Access to VPC resources

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

Prod

Test

Dev

VPC

VPC

VPC

Non

-Pro

dP

rod

VPC Peering

Access to VPC resources

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

Prod

Test

Dev

VPC

VPC

VPC

Non

-Pro

dP

rod

VPC Peering

Hairpinning

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

Prod

Test

Dev

VPC

VPC

VPC

Non

-Pro

dP

rod

Hairpinning

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

Prod

Test

Dev

VPC

VPC

VPC

Non

-Pro

dP

rod

Access to S3 using VPC Endpoints

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

VPC

VPC

VPC

Non

-Pro

dP

rod

VPC Endpoints

Access to S3 using VPC Endpoints

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

VPC

VPC

VPC

Non

-Pro

dP

rod

VPC Endpoints

Access to S3 using VPC Endpoints

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

VPC

VPC

VPC

Non

-Pro

dP

rod

VPC Endpoints

Public Virtual Interface

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

VPC

VPC

VPC

Public Virtual Interface

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

VPC

VPC

VPC

VLAN

800

BGP

BGP

Public Virtual Interface – Filtering prefixes

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

VPC

VPC

VPC

VLAN

800

BGP

BGP

PROMPT> ec2-describe-prefix-lists

PREFIXLIST pl-12345678 com.amazonaws.us-east-1.s3

CIDR 54.123.456.7/19

Public Virtual Interface

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

VPC

VPC

VPC

VLAN

800

BGP

BGP

Public Virtual Interface + VPN

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

VPC

VPC

VPC

Public Virtual Interface – US Regions

AWS Direct

Connect

Routers

DX Location

VPC

VPC

VPC

Customer

Router

VPC

VPC

VPC

AWS Direct

Connect

Routers

DX Location

VPC

VPC

Customer

Router

VPC

VPC

VPC VPC

us-east-1

us-west-1 us-west-2

AWS Direct

Connect

Routers

DX Location

VPC

VPC

Customer

Router

VPC

VPC

VPC VPC

us-east-1

us-west-1 us-west-2

AWS Direct

Connect

Routers

DX Location

VPC

VPC

Customer

Router

VPC

VPC

VPC VPC

us-east-1

us-west-1 us-west-2

AWS Direct Connect in the United States

Equinix SV

us-west-1

us-west-2

us-east-1

AWS Private Network

VPN to VGW

What are the options for connecting into AWS?

What is appropriate for my workloads?

How can I start small and grow with time?

What is the meaning of life? (optional)

Review

What are the options for connecting into AWS?

Review

• AWS-managed VPN

• Software VPN (EC2)

• Private virtual interface

• Public virtual interface

VPN Direct Connect

Review

Flexibility

What is appropriate for my workloads?

Cost

Resiliency

Performance

How can I start small and grow with time?

Review

• Connect using VPN in parallel to DX build out

• More DX locations = more resiliency

• Plan and test for resiliency, and repeat

periodically

• Talk to your AWS team

What is the meaning of life? (optional)

Review

Related Sessions

• NET402 Deep Dive - AWS Direct Connect and VPNs

• NET301 - Cloud Agility and Faster Connectivity with

AT&T NetBond and AWS

• ARC401 - From One to Many: Evolving VPC Design

Remember to complete

your evaluations!

Thank you!