© 2010 Cisco and/or its affiliates. All rights reserved. 1 Carrier-Grade NAT IPv4 Exhaust and IPv6 Transition in Internet Josef Ungerman Cisco, CCIE#6167
© 2010 Cisco and/or its affiliates. All rights reserved. 1
Carrier-Grade NAT IPv4 Exhaust and IPv6 Transition in Internet
Josef Ungerman
Cisco, CCIE#6167
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 2
Motivation
World IPv6 Launch 6/6/2012
Carrier-Grade NAT
Definition and design
Dual-stack
v4v6, v6-only, NAT64, 464
IPv6 in Mobile
Role in 3G and EPS
IPv6 in Wireline
PPPoE and IPoE sessions
Cisco CGN Products
ASR1000, ASR5000, ASR9000, CRS
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 3
RIR Pool
IANA Pool
Feb 3, 2011
*
Feb 6, 2012
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 5
• Mar 23, 2011: $11.25 per IPv4
• http://blog.internetgovernance.org/blog/_archives/2011/3/23/4778509.html
• Need for SIDR (Secure Inter-Domain Routing)
• Distributed database and RPKI infrastructure for verifying PREFIX origin AS with RIR
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 6
Internet v6 Content
YouTube goes IPv6
- DE-CIX: 30x increase
Google is 1/10th of
Internet
Netflix Video surpasses
p2p in US (29.7%)
NIX.CZ – World IPv6 Day (June 8, 2011)
NIC.CZ – cca 70.000 domains with AAAA
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 7
• What was it?
A single day (24 hrs) where major content providers advertised a AAAA DNS record for their production service (e.g. www.cisco.com, www.facebook.com); coordinated by the Internet Society
• Who participated?
Google, Facebook, Yahoo!, Akamai , Cisco , Limelight Networks were among 434 participants that offered content from their main websites over IPv6 for a 24-hour "test drive“. Cross-industry community effort: http://www.worldipv6day.org/participants/index.html
• Why do this?
Demonstrates commercial viability of IPv6
Helps identify areas of improvement in IPv6 functionality
• What happened? Nothing!
Only isolated issues reported
>3% of v6 traffic is v6-enabled countries like France
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 8
Example: Y! – 2.2M users served over IPv6, 10 support calls
Example: Akamai – 8M requests during W6D
Example: AAAA to everyone (incl. 2.5M FB-Connect websites)
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 9
• What is it?
www.worldipv6launch.org ; coordinated by the Internet Society
• W6L: Turn it on, leave it on.
Since 6/6/12, IPv6 becomes part of a regular business!
• Who will turn on IPv6 AAAA forever?
Google, Facebook, Yahoo!, Akamai , Microsoft…
CPE vendors – Cisco, D-Link
• Practical support: http://www.internetsociety.org/deploy360/
• V6 World Congress, Feb 2012
Motto links to W6L: Open The Floodgates
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 10
strategy alignment example
Cisco Confidential © 2011 Cisco and/or its affiliates. All rights reserved. 11
National IPv6 Strategies
Compliance: U.S. Federal Mandate, IPv6 task force
Next Generation Internet (CNGI) project in China and Japan
European Commission Recommendation
IPv6
IPv4 Address space completion
Public or Private Space
Limiting network expansion and putting at risk business continuity
Introducing Operational challenges
Infrastructure Evolution
Next generation Network architecture require IPv6
DOCSIS 3.0,Quad Play
Mobile SP
Networks in Motion
Networked Sensors, i.e.: AIRS
IPv6 in Client Software
IPv6 “on” in Microsoft Vista
Sensor Networks
Apple's “Back to My Mac”
v6 over v4 OTT tunnel providers
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 12
AreCharacteristic Reason Example
Infrequent Use Maintaining NAT bindings for rare occurrence events is inefficient
Earthquake Warning service NTT IPv6
Smoke detectors: 6LoWPAN
Universal Connectivity
Reachability of devices in the home
Dozens of IPv6 Tunnel brokers = unconstrained Peer-to-peer
Green Network A PC with many networked applications sends many keep-alives. Each needs Δ power across network.
Skype for iPhone drains batteries from application via data plane keep-alive
Scalable/Green Data Center
Persistent client/server transport connection is needed to keep NAT open
Facebook IM long polling
High bit Rate+NAT
Smaller SP margin per bit for AFT vs competitors without that cost
Netflix On-Demand supports IPv6.
Google 1/10th Internet traffic
FCB Internet: Faster, Cleaner, Better.
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 13
All IPv6 IPv4 Private IP 6 over 4 4 + 6 4 over 6
= IPv4 = Private IP = IPv6
CGN (NAT44) Dual Stack DS-Lite
6PE, 6rd, MIP, PPP
NAT64, 4rd, dIVI/MAP-T
Preserve
Prepare
Prosper
Dual-stack variations – CGNv4 needed anyway.
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 14
Motivation
World IPv6 Launch 6/6/2012
Carrier-Grade NAT
Definition and design
Dual-stack
v4v6, v6-only, NAT64, 464
IPv6 in Mobile
Role in 3G and EPS
IPv6 in Wireline
PPPoE and IPoE sessions
Cisco CGN Products
ASR1000, ASR5000, ASR9000, CRS
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 15
Courtesy of Jason Fesler, Yahoo (V6 World Congress 2012)
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 16
Public IPv4 Deployment
• Public IPv4 addresses used in Transport Network
• Public IPv4 addresses used on Handset for Service access
• Declining Adoption
<30% of all carriers offer public IPv4 addresses to their subscribers
PDNGW Serving Gateway eNB
IPv4 Public
IPv4 Public
public IPv4 public IPv4
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 17
NAT44
NAT44 Central Large Scale NAT44
Limited IPv4 life extension
SP operates non overlapping private address space
UE obtains a IPv4 address from the private SP address space
CGN/CGv6 performs NAT(P)44 with high scalability
Many UEs are serviced by fewer Public IP-Address on LSN Dynamically reuses available pool of Public IP-address/port bindings
PGW eNB
IPv4 IPv4
private IPv4 private IPv4
IPv4 Public
public IPv4
CGN/ CGv6
SGW
Large Scale NAT44 • O(10G) throughput • O(20M) bindings • Some subscriber awareness
NAT
Private IPv4 Address assigned to UE
Public IPv4 Address/ port assigned by CGN
IPv4 user plane with
3GPP defined tunneling: - GTP
- PMIP/GRE - IPsec
v4 Core Network:
- native IPv4
v4 user plane:
- Native IPv4 forwarding to/from CGN
Evolution of current NAT solutions • ~70% of all mobile operators leverage NAT44 • Many deployments implement NAT44 on Enterprise-Class Firewalls: Scale & throughput challenges
Cisco Confidential © 2011 Cisco and/or its affiliates. All rights reserved. 18
• Multiple customers multiplexed behind an SP managed NAT device (a Large Scale NAT)
LSN44 multiplexes several customers onto the same public IPv4 address
Each customer has unique private IPv4 address
• NAT44 can be deployed as centralized or distributed function.
• CPE based NAT44 + LSN44 = NAT444 solution
NAT44
AAA
BRAS Access
Node
Home
Gateway
IPv4 Internet
NAT44
IPv4-Private
NAT
CGN
IPv4-Private
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 19
Most of Broadband users are behind NAT today!
• NAT
First described in 1991 (draft-tsuchiya-addrtrans), RFC1631
1:1 translation: Does not conserve IPv4 addresses
Per-flow stateless
Today’s primary use is inside of enterprise networks
Connect overlapping RFC1918 address space
Note: NAT66 is stateful or stateless, but it is not NAPT
• NAPT
Described in 2001 (RFC3022)
1:N translation
Conserves IPv4 addresses
Allows multiple hosts to share one IPv4 address
Only TCP, UDP, and ICMP
Connection has to be initiated from ‘inside’
Per-flow stateful
Commonly used in home gateways and enterprise NAT
When say “NAT”, they typically mean “NAPT”
“NAT44” is used to differentiate IPv4-IPv4 NAPT from Address Family Translation, typically referred to as NAT64 and NAT46”
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 20
Courtesy of Jason Fesler, Yahoo (V6 World Congress 2012)
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 21
CGN = IP Address Sharing
• Inherent issues
draft-ford-shared-addressing-issues
• Servers must log also source port numbers
Shared IP address = shared suffering
Blacklisting, spam,…
Tracking and Law Enforcement
draft-ietf-intarea-server-logging-recommendations
• Requesting specific ports – “Not everyone can get port 80”
• Geo-Location issues (“get me the nearest ATM”)
• Complicates inbound access to media
• Keepalives power consumption, mobile battery drain
• Adds transport cost [$/Gbps]
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 22
ALG (Application Layer Gateway). L3 L4 L7…
• Fixup for applications that have problems with Firewall (and Symmetric NAT)
No Inbound connections (media, p2p,…)
No problem with Full Cone NAT (ALG not needed)
• Fixups for NAT-unaware applications
Applications that embed the IP-address in the payload or use it as user identity (did the developers respect the OSI model?)
Old applications, Enterprise-oriented applications
• No ALG’s for many applications
Encrypted or Integrity-protected protocols
eg. SIP over TLS, HTTPS://1.2.3.4 (with IPv4 address literal),…
• Modern Internet Apps work fine through NAT/FW
Why the world uses Skype and not SIP?
m/c=10.1.1.1/1234
m/c=161.44.1.1/5678
Internet
FW/NAT with
SIP ALG
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 23
• Operational headache
Undefined performance impact, numerous DoS attack vectors
Different application versions need different ALG’s
Extensions, deviations – eg. Microsoft NetMeeting different from Polycom H.323
ALG’s from different vendors behave differently, tough upgrades
In case of a bug – which vendor is guilty? How long will it take to get a fix?
• Regulatory issues
ISPs can’t sniff/modify Over The Top applications data using ALGs
eg. break location awareness in Vonage emergency calls
eg. break RTSP media streaming from NetFlix or Amazon
ALG interference with NAT traversal techniques – SIP ICE, RTSP mmusic,…
ALGs work fine in the closed Enterprise IT environment, but are ALGs desirable in Internet?
Are there any NAT-unaware Internet apps yet?
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 24
iTunes
Windows Live
Messenger
Maps
Playstation
Network
Talk Temporary exceptions (old protocols) – RTSPv1 (m.youtube.com) or MS PPTP
iPhone
App
Store
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 25
• Firewalling behavior
• Often implemented on Firewalls, CPE routers…
User-A
User-B
User-C
NAT/PAT
Inside local
Inside global
Outside local
Outside global
192.168.1.1 :5000
140.0.0.1 :6000
150.0.0.1 :6000
150.0.0.1 :6000
② Translates src-ip and src-port 192.168.1.1:5000 → 140.0.0.1:6000
① User-A sends packets to User-B
③ PAT device generates PAT entry such as below.
○
×
150.0.0.1/24
160.0.0.1/24
192.168.1.1/24 NAT POOL 140.0.0.1/24
• User-B is only translated to go into inside network.
• User-C can not reach User-A.
Symmetric NAT is …
To: 140.0.0.1:6000
To: 140.0.0.1:6000
Symmetric NAT
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 26
Full cone NAT
• Free NAT traversal requires “Full cone NAT”.
• Full cone NAT is mentioned in RFC3489 Section-5.
• What is “Full cone NAT”?.
User-A
User-B
NAT/PAT
Inside local
Inside global
Outside local
Outside global
192.168.1.1 :5000
140.0.0.1 :6000
any any
② Translates src-ip and src-port 192.168.1.1:5000 → 140.0.0.1:6000
① User-A sends packets to User-B
③ PAT device generates PAT entry such as below.
○
150.0.0.1/24
160.0.0.1/24
192.168.1.1/24 NAT POOL 140.0.0.1/24
• Not only User-B but also User-C can reach to User-A
Full cone NAT is …
○
User-C
To: 140.0.0.1:6000
Match all !!
To: 140.0.0.1:6000
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 27
X:100
Y:200
A:1000 B:2000
B:2001
Endpoint Independent Address Dependent Address and port Dependent
A:1000 B:2000
B:2001
A:1000 B:2000
B:2001
IP Addres: Port Number
Inside Outside Dst
X:100 Y:200 -
Inside Outside Dst
X:100 Y:200 A:1000
X:100 Y:300 B:2000
X:100 Y:400 B:2001
Inside Outside Dst
X:100 Y:200 A:any
X:100 Y:300 B:any
Y:200 Y:300 Y:200 Y:300 Y:400
X:100 X:100
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 28
Endpoint Independent Address Dependent Address and Port Dependent
IP Addres: Port Number
Inside Outside from
X:100 Y:200 -
Inside Outside from
X:100 Y:200 A
Inside Outside from
X:100 Y:200 A:1000
X:100
Y:200
A:1000 B:2000 A:1001
X:100
Y:200
A:1000 B:2000 A:1001
X:100
Y:200
A:1000 B:2000 A:1001
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 29
Filtering
behavior Independent Address
Dependent
Address:Port
Dependent
Ma
pp
ing
Independent
Address
Dependent
Address:Port
Dependent
Restricted CGN
IOS Router
Full Cone NAT Address Restricted NAT
Port Restricted NAT
Symmetric NAT
Linksys
WRT610N
IOS Router(enable-sym-port)
Classic STUN : simple traversal of UDP through NAT(RFC3489)
now : Session Traversal Utilities for NAT(RFC5389)
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 30
• FTP PASV, data connection always to server
• ICE, STUN, TURN
NAT EIM/EIF – Intelligence in endpoint
Useful for offer/answer protocols (SIP, XMPP, probably more)
Standardized in MMUSIC and BEHAVE
• RTSPv1, effectively replaced with Flash over HTTP
• RTSPv2, ICE-like solution
• Skype, encrypted and does its own NAT traversal
• Port 80/443 apps
STUN: “Session Traversal Utilities for NAT” – RFC 5389 ICE: “Interactive Connectivity Establishment” – RFC 5245 TURN: “Traversal Using Relays around NAT” – RFC 5766
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 31
with EIM/EIF (Full Cone NAT)
• Requirement: Endpoint Independence on ALG/fixups, Maximum application transparency
• Use Case Example: This is for Session Traversal Utilities for NAT (STUN, ICE) and is used by P2P apps to advertise themselves such that others can contact from outside-in
* source: RFC4787, RFC5382, RFC5508
NAT NAT
STUN Server
1) User-A connects to STUN Server
1) User-B connects to STUN Server
2) STUN Serv returns User-A’s translated (src-ip, src-port) to User-B
2) STUN Serv returns User-B’s translated (src-ip, src-port) to User-A
3) User-A and User-B can communicate with each other directly.
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 32
“Session Traversal Utilities for NAT” – RFC 5389
• Request/response protocol, used by:
STUN itself (to learn IP address)
ICE (for connectivity checks)
TURN (to configure TURN server)
• The response contains IP address and port of request
Runs over UDP (typical) or TCP, port 3478
• Think http://whatismyip.com
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 33
“Interactive Connectivity Establishment” – RFC 5245
• Procedure for Optimizing Media Flows
• Defines SDP syntax to indicate ‘candidate addresses’
• Uses STUN messages for connectivity checks
Sent to RTP peer, using same ports as RTP
• First best path wins
• Basic steps:
1. Gather all my IP addresses
2. Send them to my peer
3. Do connectivity checks
EXAMPLES
Google chat (XMPP)
Microsoft MSN (SIP inside of XML)
Yahoo (SIP)
Counterpath softphone (SIP)
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 34
“Traversal Using Relays around NAT” – RFC 5766
• Media Relay Protocol and Media Relay Server
• Only used when:
Both endpoints are behind ‘Address and Port-Dependent Filtering’ NATs (rare, about 25% of NATs), or
One endpoint doesn’t implement ICE, and is behind a ‘Address and Port-Dependent Filtering’ NAT
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 35
• New IP Infrastructure Element
Separate “Infrastructural Necessity” from Services (firewalling, etc.)
No ALG’s, no firewalling behavior
• Focus on:
Transparency – keep just the necessary, endpoint independence
Scale & Performance – minimal cost
Security – logging, port limits
IPv6 preparation – NAT64, 6RD, etc.
• IETF BEHAVE working group
Behavior Engineering for Hindrance Avoidance
IETF target is to promote IPv6, not to prolong IPv4 forever
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 36
RFC4787 (July 2007)
A CGN is defined by constrained behavior:
NAT Behavior Compliance (RFC4787, RFC5382, RFC5508)
Endpoint Independent Mapping and Filtering (Full Cone NAT)
Paired IP address pooling behavior
Port Parity preservation for UDP
Hairpinning behavior
Static Port Forwarding (PCP)
Current ALGs: RTSPv1, sometimes PPTP
Management
Port Limit per subscriber
Mapping Refresh
NAT logging
Redundancy (Intra-box Active/Standby, Inter-box Active/Active)
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 37
• Paired (recommended) : use the same external IP address mapping for all sessions associated with the same internal IP address
• Some peer to peer applications don’t negotiate the IP address for multiple sessions (eg. apps that are not able to negotiate the IP address for RTP and RTCP separately)
X:102
A:202
Inside
Outside
Inside Outside
X:100 A:200
X:101 A:201
X:102 A:202
Y:100 B:200
Y:101 B:201
Y:102 B:202
X:101
X:100
A:201 A:200
Y:102
B:201
Y:100
Y:101
B:202 B:200
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 38
• Use Case: Allow communications between two endpoints behind the same NAT when they are trying each other's external IP addresses
Inside
Outside Inside Outside
X:100 A:200
Y:100 B:200
X:100
A:200
Y:100
B:200
Notation X:100 IPv4 address:Port *
* TCP/UDP port or Query ID for ICMP
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 39
• Requirement: Ability to configure, a fixed private (internal) IP address:port associated with a particular subscriber while CGN allocates a free public IP address:port
• Future: PCP (Port Control Protocol) for users
Delegate port numbers to requesting applications/hosts to avoid requirement for ALGs
draft-ietf-pcp-base
Option 1: Handset/Host with PCP Client
Option 2: PCP Client, UPnP IGD proxy; NAT-PMP proxy
PCP Server
NAT-PMP
UPnP IGD
Option 2: PCP client on CPE
PCP
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 40
No Port Overloading
• A NAT must not have a "Port assignment" behavior of "Port overloading”( i.e. use port preservation even in the case of collision). Most applications will fail if this is used.
Port Parity Preservation
• An even port will be mapped to an even port, and an odd port will be mapped to an odd port. This behavior respects the [RFC3550] rule that RTP use even ports, and RTCP use odd ports.
Port Limit Per Subscriber
• Configurable port limit per subscriber for the system (includes TCP, UDP and ICMP). NAT Security – DoS attack/virus exhaust prevention.
* source: RFC4787, RFC5382
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 41
Example: GoogleMaps with Max 30 Connections Example/Slides Courtesy of NTT, See Also: Hiroshi Esaki: www2.jp.apan.net/meetings/kaohsiung2009/presentations/ipv6/esaki.ppt
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 46
Courtesy of NTT, see also Hiroshi Esaki:
www2.jp.apan.net/meetings/kaohsiung2009/presentations/ipv6/esaki.ppt
See also “An Experimental Study of Home Gateway Characteristics”
https://fit.nokia.com/lars/papers/2010-imc-hgw-study.pdf
http://www.ietf.org/proceedings/78/slides/behave-8.pdf
Source:
Application behaviors in in terms of port/session consumptions on NAT
http://opensourceaplusp.weebly.com/experiments-results.html
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 47
IOS XR: per CGN instance, default is 100 service cgn CGN1
portlimit 300
RP/0/RP0/CPU0:R#show cgn demo stat sum
Statistics summary of cgn: 'demo'
Number of active translations: 86971
Translations create rate: 0
Translations delete rate: 0
Inside to outside forward rate: 101
Outside to inside forward rate: 4
Inside to outside drops port limit exceeded: 5
Inside to outside drops system limit reached: 0
Inside to outside drops resource depletion: 0
Outside to inside drops no translation entry: 6216513
Pool address totally free: 507
Pool address used: 69
XR: When Port limit is exceeded, the Pkt is dropped and an ICMP with Type3:
Destination Unreachable, Code13: Communication Administratively
Prohibited is returned to the Sender
Classic IOS: per box, default is none, ASR1K since 3.4S
ip nat translation max-entries all-host 300
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 48
• NAT Session Setup Rate [sps] – sessions per second
Average # of New Sessions per User, during peak hours
Huge load during a failover scenarios or after a power blackout
Failing to cope with SPS = huge TCP delays, timeouts/retransmissions
• Session limit per user
Maximum # of Concurrent Sessions per User
AJAX-based applications with tens/hundreds of TCP sessions
Eg. Relaunching Firefox with Tabs opens hundreds of sessions
• Maximum Number of Sessions per CGN
Average # of Concurrent Sessions per User, during peak hours
UDP must not expire in less than 2 minutes (RFC4787)
UDP/TCP timers for Initializing and Established sessions should be configurable
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 49
L (Low-scale) Scenario – 3G mobile users, smart-phones
M (Medium-scale) Scenario – ADSL subscribers, PC users with 3G/4G dongles, Tablets, WiFi and top smart-phone users
H (High-scale) Scenario – heavy Broadband users, Internet sharing
100K BB users = up to 100Ksps and 10Mcs during peak hour!
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 50
Type Default Value
ICMP 60 sec
UDP init 30 sec
UDP active 120 sec
TCP Init 120 sec
TCP active 30 min
*) Default Refresh Direction is Bidirectional (configurable to OutBound only)
timeout: 86,400 seconds (24 hours)
udp-timeout: 300 seconds (5 minutes)
dns-timeout: 60 seconds (1 minute)
tcp-timeout: 86,400 seconds (24 hours)
finrst-timeout: 60 seconds (1 minute)
icmp-timeout: 60 seconds (1 minute)
pptp-timeout: 86,400 seconds (24 hours)
syn-timeout: 60 seconds (1 minute)
IOS XR
IOS XE (ASR1000)
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 51
• High Availability scenarios
Intra-chassis, Inter-chassis
Active/Standby, Active/Active
• Stateful or stateless
Millions of short-lived Layer-4 session
Stateful sync makes no sense for such ephemeral state (memory & CPU) – eg. ASR1000 does not sync http
Stateless redundancy
1Msps = 100K active users (10Mcs) are up in 10s minimal loss
Load-sharing = simple ECMP routing
Best Practice: Simple Non-Revertive 1:1 Warm Standby
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 52
• Data Retention Law compliance, user trackability
Who posted a content to a server on Tue at 8:09:10pm?
Global IP:port CGN Log Private IP:port MSISDN
Directive 2006/24/EC - Data Retention
• Logging Format
Must be fast and efficient (binary format)
Syslog – very chatty, inefficient ASCII encoding
1 Msps = cca 176 Mbps, 14.7 Kpps
• Netflow v9 or IPFIX
21B add-event, 11B delete-event
Compare to ASCII syslog (113B for add-event)!
Up to 68 add-events per 1500B export packet
Dynamic, template-based format
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 53
Field ID Attribute Value
234 Incoming VRF ID 32 bit ID
235 Outgoing VRF ID 32 bit ID
8 Source IP Address IPv4 Address
225 Translated Source IP
Address
IPv4 Address
7 Source Port 16 bit port
227 Translated Source Port 16 bit port
4 Protocol 8bit value
Delete Event
Template 257
(11B)
Field ID Attribute Value
234 Incoming VRF ID 32 bit ID
8 Source IP Address IPv4 Address
7 Source Port 16 bit port
4 Protocol 8bit value
Add Event
Template 256
(21B)
Tip: IsarFlow – tested CGN NFv9 Collector
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 54
Collector Performance – 100K users, average and peak
Reality check: 100K CGN users would consume 3.5TB storage per year
(compressed, fully SQL searchable data)
E-Shop: 4TB disk, 300 Euro…
Storage Capacity – includes per-day user behavior
No need to bother with logging reduction…
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 55
and data analytics
• Destination Based Logging
Keep and log destination IP:port
Just like in a Symmetric NAT/Firewall, but still keep EIM/EIF
Usage
Servers that do not log port (Apache default)
Data Analytics (Full Netflow like info)
Per-user functions (Firewall, LI, AAA) still
must be done on private IP (before NAT).
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 56
Field ID Attribute Value
234 Incoming VRF ID 32 bit ID
235 Outgoing VRF ID 32 bit ID
8 Source IP Address IPv4 Address
225 Translated Source IP Address IPv4 Address
7 Source Port 16 bit port
227 Translated Source Port 16 bit port
12 Destination Address IPv4 Address
11 Destination Port 16 bit port
4 Protocol 8 bit value
NAT44: • Add Event, Template 271 (27B)
• Delete Event, Template 272 (17B)
NAT64: • Add Event, Template 260 (47B)
• Delete Event, Template 261 (37B)
Add Event
Template 271
(27B)
Tip: IsarFlow – tested CGN NFv9 Collector
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 57
• Syslog (ASCII) cannot really log at full speed
Example (RFC5424 compliant):
1 2011 May 31 10:30:45 192.168.2.3 - - NAT44 – [UserbasedA - 10.1.32.45 INVRFA – 100.1.1.28 – 12544 12671]
Huge load (compare 113 or 250 B for syslog and 21 B for Netflow v9)
Both Syslog and Netflow are UDP, but syslog misses the sequence #
• Solution: Bulk port range allocation
Pre-allocates a port-set per user (eg. 512 ports)
PROS: Log size reduction (is it a problem in today?)
CONS: breaks randomization (port guessing attacks), cannot log the destination
• SDNAT (Staleless Deterministic NAT), aka. Algorithmic NAT
No logging at all, but…
Unrealistic requirements (eg. control of host stack and A+P routing changes)
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 58
• Normal non-bulk port allocation is random
Random ports, prefer IP address with at least 1/3rd free ports
The first 1024 ports are reserved (never allocated)
Paired pooling behavior and port parity preservation during allocation
Problem: bulk port alloc may break TCP port randomization
Algorithms in host stacks preventing guessing for TCP hijacking
Implementation
• When subscriber creates first connection, N contiguous outside ports are pre-allocated (additional connections ≤ N will use one of the pre-allocated ports).
• Bulk-allocation message is logged for the port-range, bulk-delete logged if no more sessions in this range.
Example: bulk-port-alloc size 512
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 59
Field ID Field Size
234 Incoming VRF ID 4 bytes
235 Outgoing VRF ID 4 bytes
8 Incoming/Inside Source IPv4 Address 4 bytes
225 Translated Source IPv4 Address 4 bytes
295 Translated Source Port Start 2 bytes
296 Translated Source Port End 2 bytes
Field ID Field Size
234 Incoming VRF ID 4 bytes
8 Incoming/Inside Source IPv4 Address 4 bytes
295 Translated Source Port Start 4 bytes
Add Event, Template 265
Delete Event, Template 266
NOTE: Bulk Port Allocation is mutually exclusive with Destination Based Logging (DBL).
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 60
PGW eNB
IPv4
private IPv4
IPv4 Public
public IPv4
SGW
NAT44
PGW eNB
IPv4 IPv4
private IPv4 private IPv4
IPv4 Public
public IPv4
CGN/ CGv6
SGW
NAT
NAT44
NAT
Option 1: NAT on BNG/PGW/GGSN (per-subscriber)
Option 2: NAT on Internet Gateway (as far from subscribers as possible)
Key Benefits: • Subscriber aware NAT - per subscriber control - per subscriber accounting • Large Scale (further enhanced by distribution) • Highly available (incl. geo-redundancy) • Cisco ASR5000
Key Benefits: • Integrated NAT for multiple administrative domains (operational separation) • Large Scale • Overlapping private IPv4 domains (e.g. w/ VPNs) • Cisco Internet Gateways: CRS, GSR, ASR9K, ASR1K
BEST PRACTICE On PGW put revenue-generating services (charging, firewall,…)
On Internet Gateway put infrastructural functions (BGP, CGN,…)
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 61
• NAT ≠ Firewall
Firewall motivation is inbound filtering
ALG’s are required; NAT can be used or not
CGN motivation is IPv4 exhaust solution
Maximum simplicity, transparency, massive logging
NAT44
PGW eNB
IPv4 IPv4
private IPv4 private IPv4
IPv4 Public
public IPv4
CGN/ CGv6
SGW
NAT
DPI, LI, AAA, Firewalling…
• must be done on private address space
• after NAT, it would be too late (NAT hides user’s L3 identity)
• CGN is one of the last operation before packet goes to Internet
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 62
IGW
PDP, LI, DPI…
IPv4
private IPv4
IPv4 Public
public IPv4
CGN, logging
Gi Firewall Protects against overcharging for usage-billed (non flat-fee) APN’s
Protects against network scans waking phones from fast dormancy state (battery drain)
CGN does not do help, real firewall is needed
private IPv4
Gi FW
Firewall, ALG’s (no NAT)
PGW, GGSN
IGW
PDP, LI, DPI, ALG… Per-PDP Firewall (no NAT)
IPv4
private IPv4
IPv4 Public
public IPv4
CGN, logging
private IPv4
PGW, GGSN
Solution 1
Solution 2
Solution 3
IGW
PDP, LI, DPI, ALG… Per-PDP Firewall & NAT
IPv4
private IPv4
IPv4 Public
public IPv4
PGW, GGSN
NAT
NAT
NAT
BGP
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 63
• Current Situation
Massive growth of number of mobile data traffic and number of mobile end-points
IPv4 run out: Most Operators started to
deploy NAT44
• Offload NAT44 Infrastructure
IPv6 traffic bypasses NAT44
After W6L, IPv6 content and video comes
Regulation and New Standards
IPv6 will become cheaper (eg. Bigger volume quotas or no FUP for v6)
Ultimately: IPv4 space pollution IPv6 Faster, Cleaner and Better Internet
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 64
Motivation
World IPv6 Launch 6/6/2012
Carrier-Grade NAT
Definition and design
Dual-stack
v4v6, v6-only, NAT64, 464
IPv6 in Mobile
Role in 3G and EPS
IPv6 in Wireline
PPPoE and IPoE sessions
Cisco CGN Products
ASR1000, ASR5000, ASR9000, CRS
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 65
• Dual-Stack: The classic RFC 4213 solution
Logical deployment choice when one has little control over end-point
3GPP/3GPP2 architectures support Dual-Stack, as well as Wireline (Broadband/DSL Forum, DOCSIS…)
• IPv6 endpoint enablement
Handset upgrade often required to get IPv6 or Dual-Stack (both stacks active at a time)
DSL/FTTH/Cable CPE – no s/w upgrades new RFP needed
IMS/VoIP mass market (80% of all phones are still “voice-focused” handsets)
• Deploying IPv6 in dual stack does not solve IPv4 address exhaustion: CGN needed
IPv4
Private
IPv4
IPv4
IPv6
IPv6
IPv6
IPv4 IPv4
IPv6 CGN
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 66
I get AAAA, I have IPv6 configured locally (SLAAC). But what if IPv6 network is broken?
Behavior of a typical Web-Browser
draft-ietf-v6ops-happy-eyeballs http://www.cisco.com/web/about/ac123/ac147/archived_issues/ipj_13-3/133_he.html
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 67
Slide courtesy of Teemu Savolainen (presented at v6ops, IETF 80)
draft-ietf-v6ops-happy-eyeballs – suggest to send 2 TCP SYN’s – IPv4 and IPv6
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 68
Happy Eyeballs – improving end user experience
draft-ietf-v6ops-happy-eyeballs http://www.cisco.com/web/about/ac123/ac147/archived_issues/ipj_13-3/133_he.html
NOTE – this impacts CGN44:
high session setup rate [sps]
Implementations:
• Firefox 10
• Chrome (last stable)
• OSX 10.7 “Lion” • getaddrinfo() • Safari
• iPhone iOS 4.3.1
Cisco Confidential © 2011 Cisco and/or its affiliates. All rights reserved. 69
IPv6/MPLS Core is easy. The Access is difficult.
Access Node
• DHCPv6 snooping
• LDRA/Opt37
• ICMPv6 snooping
• IPv6 NMS
• IPv6 Security
User
• OS v6 Stack
RG
• IPv6 LAN
• IPv6 WAN
• IPv6 NMS
Aggregation
• ICMPv6 snooping
• IPv6 NMS
Core
• IPv6 Routing
• MPLS 6PE/6VPE
Aggregation
• IPv6 Stack
• IPv6 PE/VPE
• IPv6 Routing
• IPv6 NMS
AAA/DHCP
BNG Access Node
DSLAM, MSAN, OLT...
RG
IPv6 IPv4 L2
Why can’t today’s broadband user just access IPv6 Internet?
NMS/Addressing
• IPv6 Parameters
• DHCPv6
Key problem with native v6: Access Node (DSLAM, MSAN, OLT, FTTX switch), CPE (new box needed), sometimes BRAS/GGSN (no dual-stack sessions)
Tunneling IPv6 over existing PPPoE (dual-stack pppoe) or IPv4 infrastructure (6RD) provides a transition solution with minimal number of “touch points”
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 70
• Broadband PPP Access
Dual-stack IPv6 and IPv4 supported over a shared PPP session with v4 and v6 NCPs running as ships in the night.
IPCP assigns IPv4, IPv6CP + DHCP-PD assigns IPv6
ASR1000 – dual-stack pppoe (16-64k sessions), no extra BRAS sessions required, ISGv6 supported
• Broadband IPoE Access
Currently 2 sessions are needed – v4 and v6
ASR1000 – ISGv6 supports IPv6 Sessions (“unclassified ipv6 prefix” based)
-Future: dual-stack v4v6 session is being worked on in BBF (Broadband Forum, ex DSL Forum)
• Mobile Access
Four types of PDP/PDN contexts: PPP (legacy), IPv4, IPv6, new “IPv4v6” (introduced in 3GPP Rel 9)
ASR5000 – Cisco’s Packet Core solution
Dual-stack capable UEs are to request IPv4v6 PDN (MIPv6, complex roaming scenarios, etc.)
PPP Session
IPv4 IPv6
VLAN
IPv6 Session
L2 Session
IPv4 IPv6
IPv4 Session
IPv4v6 PDN
IPv4 IPv6
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 71
Core Edge Aggregation Access
IP/MPLS
Customer
Native Dual-Stack IPv4/IPv6 service on RG LAN side
NO changes in existing Access/Aggregation Infrastructure
One PPPoE session per Address Family (IPv4 or IPv6) or one PPPoE session carrying both IPv4 and IPv6 NCPs running as ships in the night
Dual stack must not consume extra BNG session state
SLAAC or DHCPv6 can be used to number the WAN link with a Global address
DHCPv6-PD is used to delegate a prefix for the Home Network
PPPoE Tag Line-id authentication, Radius IPv6 attributes as per rfc3162
BNG
Dual-stack PPPoE support in hardware – ASR1000 (32K+ sessions with features)
ASR9000 (end of 2012)
X
Use Dual-stack PPPoE
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 72
CPE – 6rd RG
(Remote Gateway)
6r
d
IGW – 6rd BR
(Border Relay)
IPv4 + IPv6
IPv4
IPv4 + IPv6 Core / Internet
IPv4 + IPv6
IPv4 + IPv6
6r
d
IPv6 Destination = Inside 6rd Domain - encapsulate in IPv4, protocol 41 (address extracted from v6 prefix that contains v4 part)
IPv6 Destination = Outside 6rd Domain - encapsulate in IPv4 for the BR
6rd (Rapid Deployment)
Automatic tunneling of 6 in 4
Simple and stateless CPE, uses /32 prefix of the ISP
Large deployments (Free France, AT&T US, DSL and Cable…)
Linksys CPE support – http://home.cisco.com/en-us/ipv6
Replaces classic 6to4 tunneling (2002::/16 being obsoleted by IETF)
6RD BR support in hardware – 7600 ES+, ASR1000, CRS CGSE
CGN
+ RG IPv4 Address + Subnet ID + Interface ID
/56 /64 /128
Residence’s IPv6 Subnet is constructed from:
ISP’s IPv6 Prefix
Use 6RD – Rapid Deployment (RFC5969)
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 73
The “One-Stack” View O
pera
tions &
Deplo
ym
ent
Cost/
Com
ple
xity
IPv4 IPv6
CGN 6rd
Dual-Stack
Dual-Stack
Lite
Stateful
NAT64 Stateless
NAT64/DIVI
Stateless
4o6/4RD
Majority IP in
Operator Network
• One Network. • Addresses Run-Out
and enables IPv6 connectivity over IPv4 infra
• Two Networks!!
• Big CGN in IPv6
network.
• IPv6 can’t talk to
IPv4
• One Network. • SP-class XLAT
is IPv6 transition vehicle for 6-4 and 4-6-4 cases
Where we are right now
Being asked to go here next
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 74
IPv6 and Large Scale Address Family Translation
• AFT64 technology is only applicable in case where there are IPv6 only end-points that need to talk to IPv4 only end-points.
• NAT64 for going from IPv6 to IPv4.
• NAT64 and DNS64 is the solution
• NAT-PT is obsoleted by IETF (due to stateful DNS)
See also draft-ietf-behave-v6v4-framework, draft-ietf-behave-v6v4-xlate, draft-ietf-behave-v6v4-xlate-stateful (now RFC6144, 6145, 6146)
PGW Serving
Gateway eNB
NAT64
IPv4
Public
NAT
IPv6
Public
IPv6
Public
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 75
NAT64
LSN64
NAT
NAT64
LSN64
NATNAT
*Note: ALGs for NAT64 and NAT44 are not necessarily the same, should be avoided in CGN
IPv4 Public
IPv6
IPv6 UE
Any IPv6 address
IPv6 addresses representing IPv4 hosts “IPv4 Mapped” IPv6 Addresses Format PREFIX :IPv4 Portion:(optional Suffix)
PREFIX::
announced in
IPv6 IGP
N:1 Multiple IPv6 addresses map to single IPv4
LSN IPv4 address
announced
DNS64
Responsible for Synthesizing IPv4-Mapped IPv6 addresses
“A” Records with IPv4 address
“AAAA” Records with synthesized Address:
PREFIX:IPv4 Portion
Stateful AFT64 • AFT keeps binding state between inner IPv6 address and outer IPv4+port • Application dependent, just like NAPTv4*
AFT64
AFT64
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 76
IPv6
IPv6 addresses assigned to IPv6 hosts “IPv4 Translatable” IPv6 addresses Format PREFIX:IPv4 Portion:(SUFFIX)
IPv6 addresses representing IPv4 hosts “IPv4 Mapped” IPv6 Addresses Format PREFIX:IPv4 Portion:(SUFFIX)
0::0
announced in
IPv6 IGP
1:1 Single IPv6 addresses map to single IPv4
ISP’s IPv4 LIR
address
announced
DNS64
Responsible for Synthesizing IPv4-Mapped IPv6 addresses
Incoming Responses: “A” Records with IPv4 address
“AAAA” Records with synthesized address:
PREFIX:IPv4 Portion:(SUFFIX)
NAT64
StatelessLSN64
NATNAT
Outgoing Responses: “A” Records with IPv4 Portion
Stateless AFT64 • AFT keeps no binding state • IPv6 <-> IPv4 mapping computed algorithmically • Application dependent still
AFT64
AFT64
IPv4 Public
IPv6 UE
*USAGE: 464 DIVI (MAP-T) or v6 DataCenter (Internet-v4 accesses v6 content)
Cisco Confidential © 2011 Cisco and/or its affiliates. All rights reserved. 77
draft-mdt-softwire-map-translation-00 (MAP-T)
Demo code ready (ASR1000 – World V6 Congress demo)
Employs port restricted NAT44 + stateless NAT46 for allowing IPv4-only host access to IPv4 internet. Also Enables IPv6-only devices to access IPv4 internet.
Algorithmic mapping (based on configured or well known schema) of IPv4 ports to/from IPv6 address
Encapsulation employs IPv4-embedded IPv6 addresses
Stateless NAT64. Can also be enabled in stateful mode for other IPv6 only clients
IPv6 hosts use native addressing and IPv6 routing to public IPv6 internet
CPE
NATe
Gateway (IPv6)
IPv6
IPv6 + IPv4 IPv4-Public
IPv6
Stateful NAT46
+ port-set Stateless
NAT64
IPv4-Only Private
IPv6
Stateless NAT64 applied (dIVI – dual46, or 464)
Cisco Confidential © 2011 Cisco and/or its affiliates. All rights reserved. 78
CPE
NATe
Gateway (IPv6)
IPv6
IPv6 + IPv4 IPv4-Public
IPv6
Stateful NAT44 port-restricted
+ v6 encaps
Stateless Relay
IPv4-Only Private
IPv6 BR
CPE (B4)
Gateway (IPv6)
IPv6
IPv6 + IPv4 IPv4-Public
IPv6
No NAT, v6 tunneling
Stateful NAT44
IPv4-Only Private
IPv6 CGN44 (AFTR)
DS-Lite (draft-ietf-softwire-dual-stack-lite) – it is available today (CRS/ASR9K, some CPE’s)
Removes NAT44 from CPE where it is today, and moves it to central CGN
Dumb tunneling, no user-to-user v4 traffic (everything must go to central AFTR)
Future, no rough consensus in IETF yet
4RD (draft-despres-softwire-4rd-u) – header mapping from 4 to 6 (with fragment hdr)
MAP-E (draft-mdt-softwire-map-encapsulation) – tunneling 4 over 6
Keep NAT44 on CPE where it is today, just adds port restriction to tackle the v4 exhaust
Avoids central stateful CGN
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 79
Concept (draft-ietf-softwire-gateway-init-ds-lite)
Public
IPv4
Internet
NA(P)T 44 Flow
Association
Access Tunnel
PGW
UE
Carrier Grade NAT (CGN)
VPN1/10.1.1.1 Tunnel1/CID-1
VPN2/10.1.1.1 Tunnel2/CID2
VPN1 10.1.1.1
TCP/4444
VPN2 10.1.1.1
TCP/5555
134.95.166.10 TCP/7777
134.95.166.10 TCP/8888
Inner portion of NAT-binding
identified by combination of
CID, Tunnel-Identifier, and
optionally other identifiers
• DS-Lite is not for Mobile– it would require PhoneOS changes (unrealistic)
• GI-DS-Lite – Gateway tunnels traffic which requires NAT44 towards CGN (“Selective Extension of Access-Tunneling”)
Gateway and CGN use Context-ID (e.g. Private IP address) for Flow-Identification
• No changes to UE (Phone OS) & Access & Roaming Architecture
• Tunnel Encapsulations: MPLS (typical today) or IPinIP, GRE in future
IP/MPLS
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 80
Motivation
World IPv6 Launch 6/6/2012
Carrier-Grade NAT
Definition and design
Dual-stack
v4v6, v6-only, NAT64, 464
IPv6 in Mobile
Role in 3G and EPS
IPv6 in Wireline
PPPoE and IPoE sessions
Cisco CGN Products
ASR1000, ASR5000, ASR9000, CRS
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 81
Recommendation (clause 10)
“3GPP specifications recognize two main strategies to provide IPv6 connectivity to UEs.
For the first strategy, the operator may provide IPv4 and IPv6 connectivity for the UE. According to the scenario considered, the operator will assign a public IPv4 address or a private IPv4 address in addition to an IPv6 prefix. The operator can select one of the technical solutions described in clause 7 of this document.
The second strategy, consisting of providing the UE with IPv6-only connectivity, can be considered as a first stage or an ultimate target scenario for operators. The operator can use NAT64/DNS64 capability to access to IPv4-only services if access to IPv4 services is needed.”
Note: Clause 7 lists 3 solutions 1) NAPT44 2) GI-DS-lite (encapsulations defined in 3GPP: GRE and MPLS VPN) 3) Stateful NAT64
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 82
• Already being done by T-Mobile USA
• Their reason make perfectly good sense
• And they are proving it can work
• Problem: v4-only apps (eg. Skype)
Source: Google IPv 6 Implementor’s Conference,
https://sites.google.com/site/ipv6implementors/2010/agenda/13_Byrne_T-
Mobile_IPv6GoogleMeeting.pdf?attredirects=0
http://www.networkworld.com/community/blog/testing-nat64-and-dns64
“..Busiest day for a NAT64 box is the
day you turn it on for the first time..”
Cameron Byrne, T-Mobile
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 83
IPv4-Public
IPv6-Public
• PDP Types: IPv4, IPv6 and IPv4v6
• IPv4v6 (duals stack)
introduced in EPC from 3GPP Release 8
in 2G/3G SGSN/GGSN from 3GPP Release 9
PCRF/AAA/DHCP
PGW SGW
0
eNodeB
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 84
Create PDP Context Reply (UE IP-address,
Protocol config options (e.g. DNS-server list,…),
cause)
AAA DHCP GGSN SGSN
Attach Request
Attach Accept
Router Solicitation
Router Advertisement
UE
DHCPv6 – Information Request
DHCPv6 PD Option 3
DHCPv6 – Reply
DHCPv6 – Relay Forward
DHCPv6 – Relay Reply
DHCPv6 – Reply DHCPv6 – Relay Reply
Prefix Retrieval Option 2
Option 1 /64 prefix allocation from local pool
SLAAC
Prefix communicated to SGSN
empty UE IP-address for dynamic allocation
/64 prefix allocation: 3 Options: Local Pool, AAA, DHCP
Create PDP Context Request (APN, QoS, PDP-type=IPv6,…)
Select GGSN for given APN
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 85
• IPv6 Config: 1 Method
SLAAC after the bearer setup (/64 prefix)
Rel-10: DHCP-PD (enables Mobile Router)
Create Session Request (APN, QoS, PDN-type=IPv6,…)
Create Session Response (UE IP-address, Protocol config options (e.g. DNS-server list,…), cause)
Create Session Response
HSS/AAA DHCP PGW SGW MME
Attach Request
Router Solicitation
Router Advertisement
UE
DHCPv6 – Information Request
DHCPv6 PD Option 3
DHCPv6 – Relay Forward
DHCPv6 – Relay Reply DHCPv6 –Reply
Prefix Retrieval from AAA Option 2
Option 1 /64 prefix allocation from local pool
SLAAC
Prefix communicated to SGW/MME
/64 prefix allocation: 3 Options: Local Pool, AAA, DHCP
eNB
Attach Request Authentication of UE
Create Session Request
Attach Accept/ Initial Context Setup request
Reconfigure Radio Bearer (per MME params)
Initial Context Response Direct Transfer
(incl. Attach Complete)
Attach Complete
Uplink Data
Downlink Data Modify Bearer Request/Response
empty UE IP-address for dynamic allocation
IPv4 Config: 2 Methods
Within EPS bearer setup signaling (typical)
DHCPv4 (DHCP optional on UE and PGW)
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 86
Charging Gateway
Data
SGSN Ga (GTP’) Ga (GTP’)
Gn Gn/Gp (GTP)
Internet
DMZ
Core Network
Billing System
Ga (GTP’) IXC
Roaming partners
GRX
RNC
NodeB Femto HNB
RAN
RADIUS
DNS
DPI
GGSN
Policy
NAT
WAP
Signaling
Content providers
IMS Core
DHCP
QS
3G MS
2G MS
Element Design consideration (If IPv6 is used for internet & internal Apps) Impact
eNodeB Radio layer. Can use IPv4 backhaul No
RNC Iu-CS/Iu-PS can use IPv4 backhaul No
SGSN Initiate mobile APN query & authentication Yes
HLR/HSS IPv6 capable Yes
GGSN IPv6 PDP, standards IPv6 features, prefix allocation Yes
Billing Mediation and processing of IPv6 CDR Yes
DPI, Quote Server Pre-paid implementation, IPv6 parsing & CDR capability Yes
WAP, Data Accelerator IPv6 packet compressions, cache capability Yes
Firewalls IPv6 rules capability, performance Yes
DNS IPv6 DNS capability Yes
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 87
Two IPv6 Deployment Domains
• Enable IPv6 customer applications
IPv6 for user plane interfaces
IPv6 related attributes for control plane interfaces
IPv6 related attributes for policy/charging/control interfaces
Note: Protocol choice analysis in TR 29.803
E-UTRAN
PCRF
S11
(GTP-C)
S1-U
(GTP-U)
S2b
(PMIPv6,
GRE)
S5 (PMIPv6, GRE)
S6a
(DIAMETER)
S1-MME
(S1-AP)
GERAN
S4 (GTP-C, GTP-U)
UTRAN
S3
(GTP-C)
S12 (GTP-U)
S10
(GTP-C)
S5 (GTP-C, GTP-U)
Gx
(Gx+)
Gxb
(Gx+)
SWx (DIAMETER)
SWn
(TBD)
S6b
(DIAMETER)
SWm
(DIAMETER)
SGi
SWa
(TBD)
Gxa
(Gx+)
Rx+
UE
S2a
(PMIPv6, GRE
MIPv4 FACoA)
Trusted Non-3GPP
IP Access Untrusted Non-3GPP
IP Access
STa (RADIUS,
DIAMETER)
SWu (IKEv2,
MOBIKE, IPSec)
Operator’s
IP ServicesPDN-GW
S-GWeNB
MME
SGSN
x-CSCF
ePDG
HSS
3GPP
AAA
Gxc
(Gx+)
Enable IPv6 transport
IPv6 Home-PLMN
IPv6 Visted-PLMN
IPv6 Interconnect-PLMN
Initial Deployment Objective / Driver
1 2
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 88
Transport Options – GTP or PMIPv6 (since R8)
E-UTRAN
PCRF
S11
(GTP-C)
S1-U
(GTP-U)
S2b
(PMIPv6,
GRE)
S5 (PMIPv6, GRE)
S6a
(DIAMETER)
S1-MME
(S1-AP)
GERAN
S4 (GTP-C, GTP-U)
UTRAN
S3
(GTP-C)
S12 (GTP-U)
S10
(GTP-C)
S5 (GTP-C, GTP-U)
Gx
(Gx+)
Gxb
(Gx+)
SWx (DIAMETER)
SWn
(TBD)
S6b
(DIAMETER)
SWm
(DIAMETER)
SGi
SWa
(TBD)
Gxa
(Gx+)
Rx+
UE
S2a
(PMIPv6, GRE
MIPv4 FACoA)
Trusted Non-3GPP
IP Access Untrusted Non-3GPP
IP Access
STa (RADIUS,
DIAMETER)
SWu (IKEv2,
MOBIKE, IPSec)
Operator’s
IP ServicesPDN-GW
S-GWeNB
MME
SGSN
x-CSCF
ePDG
HSS
3GPP
AAA
Gxc
(Gx+)
UDP
GTPv1/v0-U
IPv4 IPv6
IPv4 IPv6
IPv4 IPv6
GTP-based Architecture (3G/4G)
User-Plane GGSN/PGW SGSN/SGW
GRE IPv4 IPv6
IPv4 IPv6
IPv4 IPv6
MIP-based Architecture (SAE, 23.402)
User-Plane PGW SGW
IPsec
IPv4 IPv6
IPv4 IPv6
UDP GRE
IPv4 IPv6
IPv4 IPv6
non-3GPP access (SAE, 23.402)
User-Plane
PGW ePDG AP (e.g. Femto-AP)
SP WiFi Offload uses PMIP too
Hardware-based implementation: MAG/LMA in ASR1000, LMA in ASR5000
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 89
Motivation
World IPv6 Launch 6/6/2012
Carrier-Grade NAT
Definition and design
Dual-stack
v4v6, v6-only, NAT64, 464
IPv6 in Mobile
Role in 3G and EPS
IPv6 in Wireline
PPPoE and IPoE sessions
Cisco CGN Products
ASR1000, ASR5000, ASR9000, CRS
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 90
PPPoE
RADIUS Access-Request
RADIUS Access-Accept
PPP LCP
"user1“ Line-id
Framed-Protocol PPP User-Name “user1” Service-Type Framed (Optional) framed-ipv6-prefix PPP IPv6CP
ICMPv6 RA
RA with O-bit (Optional) Prefix
Routed RG
Radius AAA
BNG
Ethernet or DSL Access Node
DHCPv6
Link Local SLAAC + Default route to BNG installed
DHCPv6 Solicit PD + DNS
DHCPv6 Reply* PD=2001:DB8:AAAA::/56、
DNS server= 2001:DB8:BB::1
DHCPv6 Request DNS
RA with O-bit Prefix=2001:DB8:AA
AA::/64
DHCPv6 Response DNS=2001:DB8:BB::1
SLAAC 2001:DB8:AAAA
::1 + Default route installed
ICMPv6 Router Advertisement
* Assuming DHCPv6 rapid commit is in effect
DHCPv6 Relay Forward Relay-fwd
DHCPv6 Relay Reply Relay-Reply
basic Authentication/Authorization + DHCP-PD
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 91
• At L2, IPv6oE with 1:1 VLANs resembles PPPoE
Moderate changes to Access Node to support IPv6 – need to forward v6 ethertype
Point-to-point broadcast domain does not require any special L2 forwarding constraints on Access Node, and SLAAC and Router Discovery work the same
Line-identifier used for 1:1 VLAN mapping= (S-TAG, C-TAG)
• However 1:1 VLANs and IPoE do require some extra BNG functionality
Statically pre-configured VLAN subinterfaces with IPv6 parameters (eg RA + services)
ND + ND Cache limit
DHCPv6 PD Server or Relay
• DHCPv6-PD or DHCPv6 server capabilities can be used at BNG to delegate a prefix for the Home Network
Customer 1
BNG Access Node
Customer 2
1:1 VLANs
1:1 VLAN (QinQ)
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 92
Customer 1 X::/56
802.1Q
N:1 VLAN
Ethernet or DSL Access Node
Customer 2 Y::/56
Split-horizon L2 forwarding rule
User-user traffic is blocked at L2 (NBMA network behavior)
BNG is the default-gw for CPE’s (all traffic goes via BNG), no proxy-ND
Subscriber line identification
VLAN no longer provides a mapping of the subscriber line
LDRA (Lightweight DHCP Relay Agent) on the Access-Node to convey Opt.37 line-id as the circuit and remote-id (draft-ietf-dhc-dhcpv6-ldra-03)
DHCPv6 is needed, SLAAC is not enough
SLAAC has no line-id insertion, problems with failure recovery with RA, no DNS…
BNG
Shared subnet (split-horizon) - Just link local, or NMS /64
1:1 VLAN (QinQ)
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 93
ICMPv6 RA
RA with O-bit
Routed RG
Radius AAA
BNG
Ethernet or DSL Access Node
DHCPv6
ICMPv6 RA
DHCPv6 Solicit PD + DNS
DHCPv6 Reply PD=2001:DB8:AAAA::/56、
DNS server= 2001:DB8:BB::1
DHCPv6 Request DNS
RA with O-bit Prefix=2001:DB8:AA
AA::/64
DHCPv6 Response DNS=2001:DB8:BB::1
SLAAC 2001:DB8:AAAA
::1 + Default route installed
DHCPv6 Relay Forward SOLICIT + Interface-id RADIUS
Access-Request DUID,
Interface-id
RADIUS Access-Accept
DHCPv6 Relay Forward Relay-fwd
PD Route installed
DHCPv6 Relay Reply Relay-Reply
DHCPv6 Relay Reply Reply + Interface-id
Circuit-id Inserted and DHCP relayed
N:1 VLAN + DHCP-PD + AAA
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 94
Features RP2+ESP20
PPPoEoQinQ Dual-stack Sessions (PTA) 32,000
QinQ sub-interfaces 32,000
H-QoS on PTA Sessions 32,000
Per User ACL 1 ACE per ACL, input ACL only
Downstream Unicast Traffic 2Gbps (64 byte)
Upstream Unicast Traffic 2Gbps (64 byte)
uRPF Enabled per-session
AAA Accounting Start-Stop Accounting
PPP Keepalives (seconds) 30
High Availability SSO
Today (3.6S) we can do much more: • Per-session CGN NAT44, IPv6 uplink AVC (DPI), ISGv6, 6VPE VRF, 48K/64K sessions…
Cisco Confidential © 2011 Cisco and/or its affiliates. All rights reserved. 95
2011:1000 1.1.1 Interface ID Subnet-
ID
0 32 56 64
6rd IPv6 Prefix Customer IPv6 Prefix
Customer’s IPv4 prefix, without the “10.” (24 bits)
In this example, the
6rd Prefix is /32
Any number of bits may be masked off, as long as they are common for the entire domain. This is very convienent when deploying with a CGSE , but is equally applicable to aggregated global IPv4 space.
Cisco Confidential © 2011 Cisco and/or its affiliates. All rights reserved. 96
CE
6r
d
6rd Border
Relays
IPv4 + IPv6
IPv4
IPv4 + IPv6 Core / Internet
IPv4 + IPv6
IPv4 + IPv6
6r
d
“Not 2001:100…” Interface ID
2001:100 8101:0101 Interface ID
THEN Encap in IPv4 with
embedded address (using
normal 6to4 encap)
IF 6rd IPv6 Prefix
Positive Match
ELSE (6rd IPv6 Prefix
Negative Match)
ENCAP with BR IPv4
Anycast Address
Dest = Inside 6rd Domain
IPv6 Dest = Outside 6rd
Domain
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 97
IPv4 Access Network
Between Subscriber and Internet, Private IPv4 Addr
IPv6 Internet
ISP IPv6 Core
ISP IPv4 Core
Subscriber Network (v4+v6)
BNG
6rd RG
6rd BR
10.100.100.1 2001:4860:0:1001::68
Destination IPv4 Address
Destination IPv6 Address Payload
Payload
(2001:4860:0:1001::68)
3456:789:0003:0101::1
Source IPv6 Address
10.3.1.1
Source IPv4 Address
10.100.100.1 2001:4860:0:1001::68 3456:789:0003:0101::1 10.3.1.1
2001:4860:0:1001::68 3456:789:0003:0101::1
2001:4860:0:1001::68 Payload 3456:789:0003:0101::1
2001:4860:0:1001::68 Payload 3456:789:0003:0101::1 10.100.100.1 10.3.1.1
2001:4860:0:1001::68 Payload 3456:789:0003:0101::1 10.100.100.1 10.3.1.1
2001:4860:0:1001::68 Payload 3456:789:0003:0101::1
Payload
Payload
Encapsulation Legend
Address Legend
10.100.100.1 6RD BR Anycast Address
10.3.1.1 RG Private IPv4 Address, obtained via DHCPv4
2001:4860:0:1001::68 www.google.com IPv6 Address
3456:789:0003:0101::1 RG IPv6 Address, SP IPv6 Prefix 3456:789/28
obtained via DHCPv4 new option or TR69
v6 prefix derived from v4 addr
copy v4 addr from v6
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 98
Subscriber Network (v4+v6)
IPv4 Access Network
Between Subscribers, Private IPv4 Addr
IPv6 Internet
ISP IPv6 Core
ISP IPv4 Core
Subscriber Network (v4+v6)
BNG
6rd RG2
6rd BR
10.3.2.1 3456:789:0003:0201::1 Payload 3456:789:0003:0101::1 10.3.1.1
3456:789:0003:0101::1 Payload 3456:789:0003:0201::1 10.3.1.1 10.3.2.1
3456:789:0003:0101::1 Payload 3456:789:0003:0201::1
Address Legend
10.3.2.1 RG2 Private IPv4 Address
10.3.1.1 RG1 Private IPv4 Address
3456:789:0003:0202::1 RG2 IPv6 Address, SP IPv6 Prefix 3456:789/28
3456:789:0003:0201::1 RG1 IPv6 Address, SP IPv6 Prefix 3456:789/28
6rd RG1
10.3.2.1 3456:789:0003:0201::1 Payload 3456:789:0003:0101::1 10.3.2.1
BNG
v6 prefix derived from v4 addr v6 prefix derived
from v4 addr
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 99
Security
Anti-spoofing - 6RD BR checks if IPv6 source addr matches the encapsulated IPv4 address
6RD RG (CPE) also verifies if the BR anycast address matches IPv6 source
QoS
V6 DSCP is automatically copied into V4
QoS pre-classify supported
HA
6RD is stateless – no SSO needed at 6RD BR
We use Anycast (same /32’s in IGP, nearest is BR chosen)
Scale and Performance
ASR1000, 7600 (ES+ since 15.1(3)S)
512 6RD Tunnel interfaces (meaning 512 6RD domains)
VRF awareness
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 100
• Source: http://home.cisco.com/en-us/ipv6
Goal is a universal dual-stack home gateway (6RD on by default).
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 101
Motivation
World IPv6 Launch 6/6/2012
Carrier-Grade NAT
Definition and design
Dual-stack
v4v6, v6-only, NAT64, 464
IPv6 in Mobile
Role in 3G and EPS
IPv6 in Wireline
PPPoE and IPoE sessions
Cisco CGN Products
ASR1000, ASR5000, ASR9000, CRS
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 102
• CRS
CGSE PLIM + FP40 (NAT44, NAT64, 6RD, DS-Lite)
20M xlates, 1Msps, 20Gbps
• ASR9000
ISM Module (NAT44, DS-Lite); BNG NAT44 for PPPoE sessions
20M xlates, 1Msps, 15Gbps
• ASR5000
Per-subscriber GGSN/PGW NAPT, Gi Firewall, DPI, charging
120M xlates, 1Msps
• ASR1000
Integrated (NAT44, NAT64, 6RD); BNG NAT44 for PPPoE sessions
2M xlates, 100Ksps, 20Gbps
• XR12000
CGN Daughter Card for the PRP-3 (NAT44, future NAT64)
10M xlates, 250Ksps, 6Gbps
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 103
CGSE – Carrier Grade Services Engine
Introducing the new engine for massive Cisco CGv6 deployments
CGSE PLIM
20+ million sessions
1+ million sessions per second [sps]
20Gb/s of throughput
Up to 240M xlates (12 CGSE’s per chassis)
64K global IP’s (100’s of thousands of users)
Intra- or Inter-Chassis Redundancy
CGN features
Subscriber port limit
Per L4 protocol/port timers
Static port forwarding
Netflow v9 logging
RTSPv1 ALG
IPv6 preparation 6rd BR (XR 3.9.3)
Stateless NAT64 (XR 3.9.3)
Stateful NAT64 (XR 4.1.2)
DS-Lite, bulk ports alloc and syslog (4.2.1)
Destination based logging (4.2.1, 4.3)
Future: PCP, PPTP ALG, MAP…
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 104
Inside Outside
Entry1 10.12.0.29:334 100.0.0.221:18808
Entry2 10.12.0.29:856 100.0.0.221:40582
Entry.. … …
Outside
VRF
Interface
VLAN
Private IPv4 Subscribers
Public IPv4
• VRFs to Separate the Private and
Public Routing Table.
• Interfaces are associated with a VRF.
• ServiceAPP interfaces are used to
send packets to/from CGSE
Dest 0.0.0.0/0 -> AppSVI1 Dest NAT Pool-> AppSVI2
Inside
VRF
App Int
CGSE App int
Interface
VLAN
VLAN
Timers (per cgn) Default Value
ICMP 60 sec
UDP init 30 sec
UDP active 120 sec
TCP Init 120 sec
TCP active 30 min
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 105
Uses a Line Card slot – paired with FP40
M
I
D
P
L
A
N
E FabQs EgressQ
Accel
FPGA
Accel
FPGA
PLA
iPSE
ePSE
IngressQ M
I
D
P
L
A
N
E
F A B R I C
Modular Services Card FP40, MSC20, MSC40
Service Engine PLIM
Octeon CPUs
Supports 20 Gbps aggregate bandwidth
20M NAT44 Translations
15M NAT64 Translations
1M sps
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 106
ISM supports 10 Gbps aggregate bandwidth
20M NAT44 Translations (today)
15M NAT64 Translations (planned)
1M sps
Uses a line card slot – connects via fabric
B
A
C
K
P
L
A
N
E
I/O
Hub
Bridge
Application
CPUs
(Intel)
24Gb
24Gb
Application
Memory
Bridge
Fabric
ASIC
Modular
Expansion
Cards (2)
ISM Mgmt CPU
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 107
SMDC supports 10 Gbps aggregate bandwidth (~6Gbps NAT)
10M NAT44 Translations (today)
7M NAT64 Translations (planned)
250K sps
daugther card on GSR PRP-3
SMDC (Service Module Daughter Card)
PRP-3 (fast CPU, 8GB DRAM, 80GB HD)
SMDC is field replacable
Dual PRP-3 – 1:1 redundancy
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 108
• Above number are based on few nat pools.
• The maximum number of nat pools supported is 1200 on a ESP20/ESP40, 600 on ESP10, 300 on ESP5, but session scalability is unknown when nat pools scale.
• ASR 1000 support up to 16k static NAT entries – in single RP system or inter-box HA
• ASR 1000 support up to 4k static NAT entries in redundant RP system
• Support up to 1K VRFs for VRF aware NAT
• Maximum interfaces support is not limited by NAT
• Maximum ACL is not limited by NAT, but by standard TCAM ACL limit
• Route-map scaling maximum is 1024
ESP Type Session Scalability
Forwarding Performance
Translation Setup/Teardown Rate (xlat/sec)
ESP5/ASR 1001
256k 3Mpps 50k
ESP10 1M 6Mpps 100k
ESP20 2M 8Mpps 200k
ESP40 2M 9Mpps 200k
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 109
ESP Type Session
Scalability
Forwarding
Performance
Translation
Setup/Teardown Rate
(xlat/sec)
ESP5 /
ASR 1001
256k 2Mpps 70k
ESP10 1M 4.2Mpps 100k
ESP20 2M 5.5Mpps 175k
ESP40 2M 5.5Mpps 180k
Support maximum 16k static entries
Maximum interfaces support is not limited by NAT64
Maximum ACL is not limited by NAT64, but by standard TCAM ACL limit.
Stateful HA possible, by default disabled for short-lived port http tcp/80
nat64 switchover replicate http enable port 80
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 110
• World IPv6 Launch – 6/6/12
• IPv4 exhaust business continuity
• CGN role and definition, RFC4787
• CGN performance – SPS, # of sessions, logging
• Dual-stack in Mobile and Wireline networks
• NAT64 – Avoiding Dual-Stack
• Future 464 traversal technologies
• Related Cisco Products