Enterprise IPv6 Deployment Perspectives from US Government PLNOG 9 22 October, 2012 Krakow, Poland Ron Broersma DREN Chief Engineer SPAWAR Network Security Manager Federal IPv6 Task Force [email protected]
Enterprise IPv6 Deployment Perspectives from US Government
PLNOG 9 22 October, 2012 Krakow, Poland
Ron Broersma DREN Chief Engineer
SPAWAR Network Security Manager Federal IPv6 Task Force
Historical Perspectives • Early networking – the ARPAnet
– Node: “IMP” + 56Kb modems – Protocol: BBN1822 + NCP, later TCP/IP
• dual stack until Jan 1, 1983 (NCP removed) – Apps: telnet, ftp, email – Routing (IP): write your own router code
• called “gateways” in those days, later called “routers” – note the “G” in EGP and BGP
– Addressing • NCP – 8 bit addresses (2 bits for “host”, 6 for “site”)
– written as “host/site”, e.g. “0/3”, or just “3”. • IP – 32 bit addresses, split as “net” part and “local address” part,
in 3 flavors (Class A, B, C). – written as “dd.dd.dd.dd”, e.g. “10.0.0.3” – CIDR didn’t come until much later
22-Oct-2012 2
ARPANET – July 1976
22-Oct-2012 3
Historical Perspectives • 1969 – successful moon landing (July), first remote login via
ARPAnet (Oct) • 1976 – about 100 hosts on the network • January 1983 – NCP removed forever
– strong incentive to implement TCP/IP • 1983 – my first breakin over the ARPAnet
– password guessing attack from university • 1984 – DNS
– until then, we downloaded (ftp) new host table regularly, and installed in the local host table (example: /etc/hosts in Unix)
• 1984 – Ethernet – multiple ethernets (and other nets) at site drive “subnetting” (taking host
part of address, and further breaking it down into “subnet part” and “host part”.
• Early 90’s – running out of Class B addresses • 1993 – CIDR • 1994 – IP next generation working group • 1995 – IPv6 specification (revised 1998) • Today – still trying to deploy IPv6 22-Oct-2012 4
IPv6 Today • While not fully matured in all areas, IPv6 is
ready for prime time. • Security and Performance of IPv6 is
equivalent to IPv4 • IPv6 deployment does not have to be costly
– If you start early and use an incremental approach, and use tech refresh, there is almost no cost to deployment.
– If you procrastinate, it will be costly. – If you haven’t started, you may be too late.
22-Oct-2012 5
IPv6 Today • Some of your suppliers have limited IPv6
support today. – You may need to switch providers or products. – But the mainstream router/switch products, and
the major operating systems all have very good support. Some of the major carriers do not, and some residential (DSL, cable) networks do not.
• The “business case” for IPv6 deployment is business survival.
• The “killer app” for IPv6 is the Internet itself.
22-Oct-2012 6
Urgency • We’re out of IPv4 addresses
– Global (IANA) pool depleted Feb 2011 – Asia (2011) and Europe (2012) depleted
• Growth and security of the Internet is at stake
• Urgent need to IPv6-enable the public Internet soonest. – don’t make your IPv6-only customers
suffer the pain of translators in the path to your site.
• Focus on your public facing services now – worry about your Intranet later – web (80), authoritative DNS (53), MX (25) – This part is actually easy
• if your providers support IPv6
22-Oct-2012 7
U.S. Federal Mandate • Signed by U.S. CIO, Sept 28,
2010 – By Sept 2012, all public content
IPv6-enabled – By Sept 2014, all internal systems
dual-stack
• Previous OMB mandate – everything “IPv6 capable” by June
2008 – Success(?): Everyone did a
“ping6”, and then turned if off.
• “Federal IPv6 Task Force” – team working to make it happen – transition managers assigned in
every agency
22-Oct-2012 8
Status
• NIST IPv6 Deployment Monitor
22-Oct-2012
http://fedv6-deployment.antd.nist.gov/
9
Observations and Questions
• Why did much of the change came right before the deadline?
• If these metrics show only 36% completion, does this indicate a failure to meet the goal?
• After the Sept 2012 deadline, what incentive is there to… – leave things turned on – continue making progress on the other 64%
22-Oct-2012 10
Success?
• Yes, this was a big success: – A significant increase in demand signal from
the U.S. Government to industry, to deliver IPv6 services • much harder to ignore us, or give low priority to our
requirements • explodes the myth that “nobody is asking for IPv6”
– A huge increase in IPv6 awareness in the Government agencies • people holding workshops, getting training,
working with their providers, etc.
22-Oct-2012 11
Success?
– A lot of public Government content is becoming IPv6-enabled, as part of the World goal to IPv6-enable the entire public Internet • being the solution, rather than the problem • setting an example and paving the way for the rest
of the public sector – This hopefully incentivizes other countries to
do something similar
22-Oct-2012 12
Some Lessons Learned • Gain operational IPv6 experience before putting too
much effort into enterprise-wide planning • Addressing Plans
– everyone makes the same mistakes because they are coming from an IPv4 mindset.
• Go native (dual stack, no tunnels, no translators) • Start from outside, and work in
– focus now on public facing services
• There will be challenges (surprises) along the way • It doesn’t require significant resources, if you start early
and leverage tech refresh
22-Oct-2012 13
Looking forward
• What is the incentive to keep the pressure on after the deadline? – We plan to not allow .gov domains to be
renewed if that organization has not met the mandates for IPv6 (and maybe DNSSEC as well).
• Other Governments and organizations should consider similar incentives
22-Oct-2012 14
Keys to success • Clear simple achievable vision and mandate, with deadlines,
from the top (CIO) • Responsibility, accountability and authority established and
managed at the executive level • Public reporting of status along the way, both internally and
externally • Bring in experts that have IPv6 operational experience, if you
don’t have it organically in your organization. – (there are few experts available; check with the industry to
ensure who you bring in can provide what is needed) • Early (and consistent) interaction with service and technology
providers, to communicate requirements. – and be willing to switch providers to acquire IPv6 support
• Dual-stack support from ISP(s)
22-Oct-2012 15
Addressing Plans • Common mistakes
– Doing other than /64 for subnets • Didn’t read RFC 4291 nor 5375
– Thinking that the addressing plan has to be perfect the first time
• because you “believe” you can’t afford to re-address – Choosing allocations for sites based on size of site
• because /48 for all sites is too wasteful – Justification “upwards”, instead of pre-allocation
“downwards” – Host-centric allocation instead of subnet-centric
22-Oct-2012 16
Addressing Plans • Without sufficient operational experience with IPv6
deployment, you WILL get it wrong at first. – usually takes the 3rd time to get it right
• Planners are hindered by IPv4-thinking – being conservative with address space – thinking “hosts” instead of “subnets”
22-Oct-2012 17
Do’s and Don’t’s
Do • Get buy-in from corporate leadership, especially CIO • Develop a corporate culture for IPv6
– involve all parts of organization, not just the network guys – have a local champion – include IPv6 in every IT initiative
• Take baby steps – go for the low hanging fruit – get experience along the way
• Leverage tech-refresh rather than spend $$$ on fork-lift upgrades out-of-cycle.
– it doesn’t have to be very expensive • Start now
– if you haven’t, you are already quite late to the game • Start by IPv6-enabling your public facing services
– work from outside in, and from bottom up • Go native
– avoid translators, tunnels, and other transition schemes • Only choose suppliers that have a good IPv6 story
22-Oct-2012 19
Don’t • waste time developing a complete transition plan with no
operational experience • base your addressing plan on conservative IPv4
practices • waste time on a comprehensive addressing plan without
operational experience – consider the first one a throw-away
• waste time trying to develop a business case (ROI) for deploying IPv6. – it is a matter of business survival
• be afraid to break some glass – world ipv6 day validated that
22-Oct-2012 20
Updates, Observations, and other News…
World IPv6 Launch • Since the SPAWAR enterprise network (AS 22) is 100%
dual-stack, how would network utilization (traffic inbound from the Internet) be impacted by an event like this?
• Previously (5 min averages, daytime):
• After World IPv6 Launch
• Another view: overall daily average of traffic:
22-Oct-2012
1% in 2009 before Google whitelisting 2.5% after Google whitelisting just under 10% when Youtube was added (late Jan 2010) around 15% after World IPv6 Day (June 2011)
around 20% average during the day
Before: range (workdays) : 11-14% After: 14-18%
22
Top Enterprise Deployment Challenges
• Lack of IPv6/IPv4 feature parity – taking way too long to get there
• Vendors not eating own dogfood – but this is starting to change
• Rogue RAs due to Windows ICS – set router priority to “high” as workaround
• Privacy Addresses (RFC4941) break address stability – no easy way to centrally disable
• Lack of DHCPv6 client support in older OS’s • Network Management over IPv6 not quite there • Operational Complexity with dual-stack
22-Oct-2012 23
Benefits of IPv6 today (examples) • Addressing
– can better map subnets to reality – can align with security topology, simplifying ACLs – sparse addressing (harder to scan/map) – never have to worry about “growing” a subnet to hold new machines – auto-configuration, plug-n-play – universal subnet size, no surprises, no operator confusion, no bitmath – shorter addresses in some cases – at home: multiple subnets rather than single IP that you have to NAT
• Link Local implemented on every interface • Multicast is simpler
– embedded RP – no MSDP
• Mobile IPv6 is cleaner/simpler than in IPv4
22-Oct-2012 24
Playing with IPv6-only environments
(why would you want to do this?)
Case study
• Can you do all your network management using IPv6?
• Can you turn off IPv4 on your management LAN?
• How well do various products operate in this environment?
22-Oct-2012 26
Findings
• Very few products can be fully managed using IPv6
• You won’t learn what’s missing or broken unless you try it in production – remove the training wheels
• You can’t turn off IPv4 (yet) • Bugs take 6 to 12 months to get fixed • Feature requests take 18 to 48 months to
get fixed 22-Oct-2012 27
Previously (June ‘2011):
Management over IPv6 in some products
22-Oct-2012
SSH HTTPS
DNS Syslog SNMP NTP RADIUS Unified MIB RFC4293
Flow export
TFTP FTP
CDP LLDP
IPv6 MTU
No v4
Cisco3 6
Brocade1 9
Juniper 5
ALU 4
A10 8 7
SSH HTTPS
DNS Syslog SNMP NTP RADIUS Unified MIB RFC4293
Flow export TFTP FTP
CDP LLDP
Cisco
Brocade
Juniper
Now:
1. Can’t reboot using SNMP over IPv6 2. . 3. 15.2(2)TR 4. 10.0R6 (Nov 2012) 5. 12.3R1 Nov 2012 (beta in August) 6. ASR1K:3.7S (July 2012) 7. 3.0 release, 2012Q4 8. No plans 9. fix planned for Apr 2013
28
example IPv6-only bug
• when disabling IPv4 one vendor’s switches, they start responding to all ip-subnet-broadcasts, and start ARPing (from 0.0.0.0), and other strange behaviors.
• Example: echo request to x.x.x.255/24:
22-Oct-2012 29
Other IPv6-only tests • Test environment:
– network with ONLY IPv6 turned on (no IPv4 configuration or routing)
• “A” bit enabled (SLAAC) • “M” and “O” enabled (for DHCPv6)
– Many operating systems connected, to see how they behave • Windows7, MacOSX, Linux (multiple distributions), FreeBSD • iPhone, iPad, Android
• Anything without a dhcpv6-client won’t get DNS addresses – Windows XP, MacOSX before Lion, Android
22-Oct-2012 30
IPv6-only • Observation (Lion):
– You can browse OK with Safari, but Chrome and Firefox hang when trying to browse to IPv6-only web sites
• happy-eyeballs not working – tcpdump shows it ARPing for Internet addresses – … because there is a default-route-to-interface installed in the routing table – … because it assigns IPv4 link-local (RFC 3927) and implements “ARP for
everything” (paragraph 2.6.2) – … so it “thinks” it has full IPv4-internet reachability (unlike IPv6 behavior)
• Most other OS’s exhibit similar behavior • Need to fix happy-eyeballs • workaround: actually assign IPv4 addresses (like maybe from
100.64/10 space) with default IPv4 route, but have router respond to everything as net/host-unreachable.
– or just disable IPv4 on the OS (Lion has a knob to do this).
22-Oct-2012 31
END
Contact me: [email protected]