Hong Kong Network Updates – Interconnections in HK

Hong Kong Hong Kong Network UpdatesNetwork Updates

– Interconnections in HK– Interconnections in HK

Che-Hoo CHENG Che-Hoo CHENG 鄭志豪 鄭志豪 The Chinese University of Hong Kong /The Chinese University of Hong Kong /

Hong Kong Internet ExchangeHong Kong Internet Exchange

05 MAR 200905 MAR 2009

Introduction of HKIX Introduction of HKIX (1/2)(1/2)

Set up by The Chinese University of Hong Kong (CUHK) in Apr 1995Set up by The Chinese University of Hong Kong (CUHK) in Apr 1995 MLPA Internet Exchange over Layer 2 Infrastructure with BLPA MLPA Internet Exchange over Layer 2 Infrastructure with BLPA

supportsupport MLPAMLPA

• Mandatory for Hong Kong routes onlyMandatory for Hong Kong routes only• HKIX MLPA Router Server: AS4635HKIX MLPA Router Server: AS4635

AS4635 seen in AS PathAS4635 seen in AS Path• IPv4 Route filters implemented strictlyIPv4 Route filters implemented strictly

By Prefix or by Origin AS By Prefix or by Origin AS But a few trustable participants have no filters except max But a few trustable participants have no filters except max

number of prefixesnumber of prefixes Support BLPASupport BLPA

• One AS hop less than MLPAOne AS hop less than MLPA• May get more routes from your peers than MLPAMay get more routes from your peers than MLPA• HKIX encourages BLPA over HKIXHKIX encourages BLPA over HKIX

22

Introduction of HKIX Introduction of HKIX (2/2)(2/2)

202.40.161/24202.40.161/24 Port Security (one MAC address per switch port) implemented Port Security (one MAC address per switch port) implemented

strictlystrictly Our service is basically free of charge as we are Our service is basically free of charge as we are not-for-profitnot-for-profit

• But there will be charge for 10GE port or >2 x GE ports if traffic But there will be charge for 10GE port or >2 x GE ports if traffic volume is not high enough to justify the resourcesvolume is not high enough to justify the resources

Provide colo space for strategic partners such as root / TLD DNS Provide colo space for strategic partners such as root / TLD DNS servers & APNICservers & APNIC

Still located and operated by CUHKStill located and operated by CUHK Considered as Critical Internet Infrastructure in Hong KongConsidered as Critical Internet Infrastructure in Hong Kong We are confident to say that with HKIX, more than 98% of intra-HK We are confident to say that with HKIX, more than 98% of intra-HK

Internet traffic is kept within HKInternet traffic is kept within HK More info on More info on www.hkix.net

33

44

ISP DISP A ISP B ISP C

Routes of ISP A Routes of All

ISPs in HKIX

Routes of ISP B

Routes of ISP C Routes of

ISP D

Routes of All ISPs in HKIX

Routes of All ISPs in HKIX

Routes of All ISPs in HKIX

MLPARouterServer

Routes of All ISPs in HKIX

Routes from All

ISPs

Switched Ethernet

HKIX Model — HKIX Model — MLPA over Layer 2 MLPA over Layer 2 (with BLPA support)(with BLPA support)

• MLPA traffic exchanged directly over layer 2 without going through MLPA Route Server

• BLPA over layer 2 without involvement of MLPA Route Server

HKIX2HKIX2 Announced on 25 Nov 2004Announced on 25 Nov 2004 HKIX2 site in Central as redundant site of HKIXHKIX2 site in Central as redundant site of HKIX Linked up to HKIX by 2 x 10GE linksLinked up to HKIX by 2 x 10GE links

• It is It is Layer 3Layer 3 connection connection• Same AS4635 MLPASame AS4635 MLPA• Different broadcast domain from HKIXDifferent broadcast domain from HKIX

218.100.16/24218.100.16/24• Participants cannot do BLPA across HKIX and HKIX2Participants cannot do BLPA across HKIX and HKIX2

Free of charge for up to 2 GE ports unless traffic volume Free of charge for up to 2 GE ports unless traffic volume justifiesjustifies

IX portion managed by CUHKIX portion managed by CUHK

55

Quick Updates Quick Updates (1/2)(1/2) 2 x Cisco Catalyst 6513 at HKIX and 1 x Cisco Catalyst 6513 2 x Cisco Catalyst 6513 at HKIX and 1 x Cisco Catalyst 6513

at HKIX2at HKIX2 Most connected to HKIX switches without co-located routersMost connected to HKIX switches without co-located routers

• Cross-border layer 2 Ethernet connections to HKIX Cross-border layer 2 Ethernet connections to HKIX possiblepossible

Ethernet over MPLS or Ethernet over SDHEthernet over MPLS or Ethernet over SDH Officially allow overseas ISPs to connect nowOfficially allow overseas ISPs to connect now

• Those overseas ISPs may not have Hong Kong routes…Those overseas ISPs may not have Hong Kong routes…• Major overseas R&E networks connected in 2008Major overseas R&E networks connected in 2008

110 AS’es connected at HKIX + 18 AS’es at HKIX2 now110 AS’es connected at HKIX + 18 AS’es at HKIX2 now• 16 AS’es at both HKIX2 & HKIX for redundancy16 AS’es at both HKIX2 & HKIX for redundancy

18 x 10GE + 175 x GE/FE ports served18 x 10GE + 175 x GE/FE ports served >23,000 IPv4 prefixes carried by HKIX MLPA>23,000 IPv4 prefixes carried by HKIX MLPA

• More non-HK routes than HK routesMore non-HK routes than HK routes Peak 5-min traffic >75 Gbps nowPeak 5-min traffic >75 Gbps now

66

Quick Updates Quick Updates (2/2)(2/2) A small HKIX POP with Cisco 7603 (as layer 2 switch) has A small HKIX POP with Cisco 7603 (as layer 2 switch) has

been set up in Mega-i with GE link (layer 2) back to HKIX at been set up in Mega-i with GE link (layer 2) back to HKIX at CUHK but it is for academic network connections onlyCUHK but it is for academic network connections only

Basic Set-up:Basic Set-up:• First 2 GE ports at HKIX and First 2 GE ports at HKIX2 First 2 GE ports at HKIX and First 2 GE ports at HKIX2

free of charge with no question asked and no agreementfree of charge with no question asked and no agreement Advanced Set-up:Advanced Set-up:

• If 10GE port or >2 GE ports are needed, agreement is If 10GE port or >2 GE ports are needed, agreement is needed and there will be a small port charge unless needed and there will be a small port charge unless aggregate traffic volume of all ports exceeds 50% (95aggregate traffic volume of all ports exceeds 50% (95thth percentile)percentile)

See See http://www.hkix.net/hkix/connectguide.htm for details for details

77

Usage Statistics of HKIXUsage Statistics of HKIX

88

Plan for 2009Plan for 2009 Order has been placed to replace one Cisco Catalyst 6513 at HKIX Order has been placed to replace one Cisco Catalyst 6513 at HKIX

with a brand new high-end switchwith a brand new high-end switch• To support 128 line-rate 10GE portsTo support 128 line-rate 10GE ports• To support LACP with port security over GE & 10GE portsTo support LACP with port security over GE & 10GE ports

Remote participants have to check whether their tail Remote participants have to check whether their tail providers can support LACP with enough transparencyproviders can support LACP with enough transparency

• sFlow equivalentsFlow equivalent• To be in production in May 2009To be in production in May 2009

MLPA: Support daily automatic route filter updates from routing MLPA: Support daily automatic route filter updates from routing registry databaseregistry database

MLPA: Support BGP community for easier traffic load balancingMLPA: Support BGP community for easier traffic load balancing Portal for ParticipantsPortal for Participants Improve after-hour supportImprove after-hour support We continue to encourage BLPA We continue to encourage BLPA Suggestions are welcomeSuggestions are welcome

99

IPv6 at HKIXIPv6 at HKIX CUHK/HKIX is committed to help Internet development in CUHK/HKIX is committed to help Internet development in

HKHK IPv6 supported by HKIX since Mar 2004IPv6 supported by HKIX since Mar 2004

• Dual stackDual stack Today, 24 AS’es have been assigned addresses at Today, 24 AS’es have been assigned addresses at

HKIX/HKIX2 and have joined MLPAHKIX/HKIX2 and have joined MLPA• BLPA encouragedBLPA encouraged

Root server instance F supports IPv6 transport at HKIXRoot server instance F supports IPv6 transport at HKIX Dual stack so cannot know for sure how much IPv6 traffic in Dual stack so cannot know for sure how much IPv6 traffic in

totaltotal• Should be lower than 1% of the total trafficShould be lower than 1% of the total traffic• Hopefully with the new switch, we can have more Hopefully with the new switch, we can have more

detailed statisticsdetailed statistics1010

IPv6 Participants at HKIX IPv6 Participants at HKIX (1/2)(1/2)

APAN-JP (AS7660)APAN-JP (AS7660) APNIC (AS18366)APNIC (AS18366) ASCC-ASNET (AS9264)ASCC-ASNET (AS9264) Bhutan Telecom (AS17660)Bhutan Telecom (AS17660) China Mobile-Peoples (AS9231; at HKIX2 only)China Mobile-Peoples (AS9231; at HKIX2 only) CITIC1616 (AS17554; at HKIX2 only)CITIC1616 (AS17554; at HKIX2 only) CNGI-6IX (AS23911; IPv6 only) CNGI-6IX (AS23911; IPv6 only) CUHK (AS3661 & AS4641)CUHK (AS3661 & AS4641) Diyixian (AS9584)Diyixian (AS9584) Globalnet (AS17990)Globalnet (AS17990) Google (AS15169; at both HKIX & HKIX2)Google (AS15169; at both HKIX & HKIX2) Hurricane Electric (AS6939)Hurricane Electric (AS6939) Hutchison Global Communications (AS9304)Hutchison Global Communications (AS9304)

1111

IPv6 Participants at HKIX IPv6 Participants at HKIX (2/2)(2/2)

ISC (AS23709)ISC (AS23709) Internode (AS4739; to be connected soon)Internode (AS4739; to be connected soon) JUCC-HARNET (AS3662)JUCC-HARNET (AS3662) KREONET2 (AS17579)KREONET2 (AS17579) NTT Com (AS2914)NTT Com (AS2914) Reliance Globalcom / FLAG (AS15412)Reliance Globalcom / FLAG (AS15412) Samsung (AS6619)Samsung (AS6619) SCIG of HK Government (AS9732)SCIG of HK Government (AS9732) Telstra-CSL (AS38819)Telstra-CSL (AS38819) TIC (AS1836)TIC (AS1836)

Good mix of academic networks and commercial networksGood mix of academic networks and commercial networks All joined MLPAAll joined MLPA Can set up BLPA with them over HKIXCan set up BLPA with them over HKIX

1212

Recent IPv6 Work at HKIXRecent IPv6 Work at HKIX Remove route filters for IPv6 at MLPA route server Remove route filters for IPv6 at MLPA route server

• Still provides minimal protection such as bogus routesStill provides minimal protection such as bogus routes• Total number of routes on MLPA >1,000 nowTotal number of routes on MLPA >1,000 now

Add BGP community tagging to distinguish upstream routes Add BGP community tagging to distinguish upstream routes for transit purpose from downstream routes for peering for transit purpose from downstream routes for peering purposepurpose• 4635:900 - Upstream / peer routes announced by free 4635:900 - Upstream / peer routes announced by free

IPv6 transit providers for use by those HKIX participants IPv6 transit providers for use by those HKIX participants which seek for free IPv6 transit over HKIX MLPAwhich seek for free IPv6 transit over HKIX MLPA

• 4635:800 - Downstream / internal routes seeking for 4635:800 - Downstream / internal routes seeking for peering only over HKIX MLPA but not for transitpeering only over HKIX MLPA but not for transit

• 4635:700 - Downstream / internal routes seeking for 4635:700 - Downstream / internal routes seeking for transit over HKIX MLPAtransit over HKIX MLPA

See See http://www.hkix.net/hkix/route-server.htm for details for details1313

Mega-iMega-i

Located in Chai Wan in Eastern part of HK IslandLocated in Chai Wan in Eastern part of HK Island Important Carrier Hotel in HKImportant Carrier Hotel in HK Essentially all submarine / terrestrial cable Essentially all submarine / terrestrial cable

operators have presence thereoperators have presence there Good for physical interconnectionsGood for physical interconnections But colo space is running outBut colo space is running out

• Colo and Cross Connect charges increasingColo and Cross Connect charges increasing CUHK/HKIX has a small POP there to serve R&E CUHK/HKIX has a small POP there to serve R&E

networks onlynetworks only• One GE link back to HKIX should serve multiple One GE link back to HKIX should serve multiple

interconnection requirementsinterconnection requirements

1414

R&E Networks in Mega-iR&E Networks in Mega-i

ASCC/ASNET: 5/FASCC/ASNET: 5/F ASGC: 12/F inside PACNETASGC: 12/F inside PACNET CERNET/CERNET2/CNGI-6IX: 8/FCERNET/CERNET2/CNGI-6IX: 8/F CSTNET/GLORIAD: 32/F Meet-Me RoomCSTNET/GLORIAD: 32/F Meet-Me Room CUHK/HKIX: 32/F Open FarmCUHK/HKIX: 32/F Open Farm KISTI/KREONET2: 12/F inside PACNETKISTI/KREONET2: 12/F inside PACNET NICT: 10/FNICT: 10/F TEIN3: 8/FTEIN3: 8/F

1515

Fiber Cross Connect Fiber Cross Connect Inside Mega-iInside Mega-i

Same charge for MMF & SMFSame charge for MMF & SMF Ordering may be complicated if more than Ordering may be complicated if more than

one party is involvedone party is involved iAdvantage now have monthly charge iAdvantage now have monthly charge

even for Fiber Cross Connect within the even for Fiber Cross Connect within the same floorsame floor

If not a lot of traffic, HKIX switch at Mega-i If not a lot of traffic, HKIX switch at Mega-i can be used for interconnections among can be used for interconnections among R&E networks to avoid managing and R&E networks to avoid managing and paying multiple fiber cross connect cablespaying multiple fiber cross connect cables

1616

HKIX Layer 2(MLPA:AS4635)

HARNETAS3662

at Mega-i

…..

at CUHK

at CUHKat HKU

ASGCAS24167

KREONET2AS17579CSTNET

AS7497

APAN-JPAS7660

Internet2AS11537

PCCWGlobal

AS3491

CUHKAS3661

ASCCAS9264

CNGI-6IXAS23911

CERNET2AS23910

CERNETAS4538

OtherUniversities

in HK

NTTAS2914

Wharf T&TAS9381

TEIN3AS24489

Cisco 7603 of CUHKon 32/F Mega-i

on 32/F Mega-i

GE(SX)untagged

GE(SX)VLAN Trunk

NICTon 10/F Mega-i

CUHK/HKIXin 32/F Mega-i

GEuntagged

Interconnections between NICT and CSTNET in Mega-i

GE Link of HKIXGE Link of HKIXBetween Mega-i and CUHKBetween Mega-i and CUHK

1919

CUHK – PWH Hospital CUHK – PWH Hospital (1/2)(1/2) PWH is the teaching hospital of CUHKPWH is the teaching hospital of CUHK 7km away from Main Campus7km away from Main Campus

• Little chance to lay our own fibersLittle chance to lay our own fibers Leasing 300Mbps bandwidth over GE nowLeasing 300Mbps bandwidth over GE now

• Can be upgraded easily if neededCan be upgraded easily if needed Networks within PWH is complicated as Hospital Authority is Networks within PWH is complicated as Hospital Authority is

there alsothere also When doing telemedicine, CUHK network resources When doing telemedicine, CUHK network resources

(AS3661) will be used(AS3661) will be used CUHK/AS3661 has direct interconnections with APAN-JP, CUHK/AS3661 has direct interconnections with APAN-JP,

ASCC, ASGC, CNGI-6IX, CSTNET, HARNET & KREONET2 to ASCC, ASGC, CNGI-6IX, CSTNET, HARNET & KREONET2 to ensure more direct routing path is selectedensure more direct routing path is selected

2020

CUHK – PWH Hospital CUHK – PWH Hospital (2/2)(2/2)

2121

300Mbps Link of CUHK300Mbps Link of CUHKBetween Main Campus Between Main Campus

and PWHand PWH

2222

Thank YouThank You