8/6/2019 AfNOG2005 Troubleshooting BGP
1/105
8/6/2019 AfNOG2005 Troubleshooting BGP
2/105
8/6/2019 AfNOG2005 Troubleshooting BGP
3/105
8/6/2019 AfNOG2005 Troubleshooting BGP
4/105
8/6/2019 AfNOG2005 Troubleshooting BGP
5/105
8/6/2019 AfNOG2005 Troubleshooting BGP
6/105
8/6/2019 AfNOG2005 Troubleshooting BGP
7/105
8/6/2019 AfNOG2005 Troubleshooting BGP
8/105
8/6/2019 AfNOG2005 Troubleshooting BGP
9/105
8/6/2019 AfNOG2005 Troubleshooting BGP
10/105
8/6/2019 AfNOG2005 Troubleshooting BGP
11/105
8/6/2019 AfNOG2005 Troubleshooting BGP
12/105
8/6/2019 AfNOG2005 Troubleshooting BGP
13/105
8/6/2019 AfNOG2005 Troubleshooting BGP
14/105
8/6/2019 AfNOG2005 Troubleshooting BGP
15/105
8/6/2019 AfNOG2005 Troubleshooting BGP
16/105
8/6/2019 AfNOG2005 Troubleshooting BGP
17/105
8/6/2019 AfNOG2005 Troubleshooting BGP
18/105
8/6/2019 AfNOG2005 Troubleshooting BGP
19/105
8/6/2019 AfNOG2005 Troubleshooting BGP
20/105
8/6/2019 AfNOG2005 Troubleshooting BGP
21/105
8/6/2019 AfNOG2005 Troubleshooting BGP
22/105
8/6/2019 AfNOG2005 Troubleshooting BGP
23/105
8/6/2019 AfNOG2005 Troubleshooting BGP
24/105
8/6/2019 AfNOG2005 Troubleshooting BGP
25/105
8/6/2019 AfNOG2005 Troubleshooting BGP
26/105
8/6/2019 AfNOG2005 Troubleshooting BGP
27/105
8/6/2019 AfNOG2005 Troubleshooting BGP
28/105
8/6/2019 AfNOG2005 Troubleshooting BGP
29/105
8/6/2019 AfNOG2005 Troubleshooting BGP
30/105
8/6/2019 AfNOG2005 Troubleshooting BGP
31/105
8/6/2019 AfNOG2005 Troubleshooting BGP
32/105
8/6/2019 AfNOG2005 Troubleshooting BGP
33/105
8/6/2019 AfNOG2005 Troubleshooting BGP
34/105
8/6/2019 AfNOG2005 Troubleshooting BGP
35/105
8/6/2019 AfNOG2005 Troubleshooting BGP
36/105
8/6/2019 AfNOG2005 Troubleshooting BGP
37/105
8/6/2019 AfNOG2005 Troubleshooting BGP
38/105
8/6/2019 AfNOG2005 Troubleshooting BGP
39/105
8/6/2019 AfNOG2005 Troubleshooting BGP
40/105
8/6/2019 AfNOG2005 Troubleshooting BGP
41/105
8/6/2019 AfNOG2005 Troubleshooting BGP
42/105
8/6/2019 AfNOG2005 Troubleshooting BGP
43/105
8/6/2019 AfNOG2005 Troubleshooting BGP
44/105
8/6/2019 AfNOG2005 Troubleshooting BGP
45/105
8/6/2019 AfNOG2005 Troubleshooting BGP
46/105
8/6/2019 AfNOG2005 Troubleshooting BGP
47/105
8/6/2019 AfNOG2005 Troubleshooting BGP
48/105
8/6/2019 AfNOG2005 Troubleshooting BGP
49/105
8/6/2019 AfNOG2005 Troubleshooting BGP
50/105
8/6/2019 AfNOG2005 Troubleshooting BGP
51/105
8/6/2019 AfNOG2005 Troubleshooting BGP
52/105
8/6/2019 AfNOG2005 Troubleshooting BGP
53/105
8/6/2019 AfNOG2005 Troubleshooting BGP
54/105
8/6/2019 AfNOG2005 Troubleshooting BGP
55/105
8/6/2019 AfNOG2005 Troubleshooting BGP
56/105
8/6/2019 AfNOG2005 Troubleshooting BGP
57/105
8/6/2019 AfNOG2005 Troubleshooting BGP
58/105
8/6/2019 AfNOG2005 Troubleshooting BGP
59/105
8/6/2019 AfNOG2005 Troubleshooting BGP
60/105
8/6/2019 AfNOG2005 Troubleshooting BGP
61/105
8/6/2019 AfNOG2005 Troubleshooting BGP
62/105
8/6/2019 AfNOG2005 Troubleshooting BGP
63/105
8/6/2019 AfNOG2005 Troubleshooting BGP
64/105
8/6/2019 AfNOG2005 Troubleshooting BGP
65/105
8/6/2019 AfNOG2005 Troubleshooting BGP
66/105
8/6/2019 AfNOG2005 Troubleshooting BGP
67/105
8/6/2019 AfNOG2005 Troubleshooting BGP
68/105
8/6/2019 AfNOG2005 Troubleshooting BGP
69/105
8/6/2019 AfNOG2005 Troubleshooting BGP
70/105
8/6/2019 AfNOG2005 Troubleshooting BGP
71/105
8/6/2019 AfNOG2005 Troubleshooting BGP
72/105
8/6/2019 AfNOG2005 Troubleshooting BGP
73/105
8/6/2019 AfNOG2005 Troubleshooting BGP
74/105
8/6/2019 AfNOG2005 Troubleshooting BGP
75/105
8/6/2019 AfNOG2005 Troubleshooting BGP
76/105
8/6/2019 AfNOG2005 Troubleshooting BGP
77/105
8/6/2019 AfNOG2005 Troubleshooting BGP
78/105
8/6/2019 AfNOG2005 Troubleshooting BGP
79/105
8/6/2019 AfNOG2005 Troubleshooting BGP
80/105
8/6/2019 AfNOG2005 Troubleshooting BGP
81/105
8/6/2019 AfNOG2005 Troubleshooting BGP
82/105
8/6/2019 AfNOG2005 Troubleshooting BGP
83/105
8/6/2019 AfNOG2005 Troubleshooting BGP
84/105
8/6/2019 AfNOG2005 Troubleshooting BGP
85/105
8/6/2019 AfNOG2005 Troubleshooting BGP
86/105
8/6/2019 AfNOG2005 Troubleshooting BGP
87/105
8/6/2019 AfNOG2005 Troubleshooting BGP
88/105
8/6/2019 AfNOG2005 Troubleshooting BGP
89/105
8/6/2019 AfNOG2005 Troubleshooting BGP
90/105
8/6/2019 AfNOG2005 Troubleshooting BGP
91/105
8/6/2019 AfNOG2005 Troubleshooting BGP
92/105
8/6/2019 AfNOG2005 Troubleshooting BGP
93/105
8/6/2019 AfNOG2005 Troubleshooting BGP
94/105
8/6/2019 AfNOG2005 Troubleshooting BGP
95/105
8/6/2019 AfNOG2005 Troubleshooting BGP
96/105
Troubleshooting Connectivity
8/6/2019 AfNOG2005 Troubleshooting BGP
97/105
979797 2005, Cisco Systems, Inc. All rights reserved.APRICOT2005
Troubleshooting Connectivity Example IV
Checklist:Assume AS1 has done everything in thistutorial so far
L2 problem? Route Flap Damping?
All the configurations look fine, the Looking Glassoutputs look fine, life is wonderful Apart from thoseannoying traffic swings every hour or so
Since BGP is configured fine, and the net has beenstable for so long, can only be an L2 problem, orRoute Flap Damping side-effect
Troubleshooting Connectivity
8/6/2019 AfNOG2005 Troubleshooting BGP
98/105
989898 2005, Cisco Systems, Inc. All rights reserved.APRICOT2005
g yExample IV
L2 upstream somewhere has poorconnectivity between themselves and therest of the Internet
Only real solution is to impress uponupstream that this isnt good enough, and getthem to fix it
Or change upstreams
Troubleshooting Connectivity
8/6/2019 AfNOG2005 Troubleshooting BGP
99/105
999999 2005, Cisco Systems, Inc. All rights reserved.APRICOT2005
g yExample IV
Route Flap DampingMany ISPs implement route flap damping
Many ISPs simply use the vendor defaults
Vendor defaults are generally far too severe
There is even now some real concern that the morelenient RIPE-229 values are too severe
www.cs.berkeley.edu/~zmao/Papers/sig02.pdf
Again Looking Glasses come to the operators
assistance
8/6/2019 AfNOG2005 Troubleshooting BGP
100/105
100100100 2005, Cisco Systems, Inc. All rights reserved.APRICOT2005
Troubleshooting Connectivity
8/6/2019 AfNOG2005 Troubleshooting BGP
101/105
101101101 2005, Cisco Systems, Inc. All rights reserved.APRICOT2005
g yExample IV
Most Looking Glasses allow the operators tocheck the flap or damped status of theirannouncements
Many oscillating connectivity issues are usually caused
by L2 problemsRoute flap damping will cause connectivity to persistvia alternative paths even though primary paths havebeen restored
Quite often, the exponential back off of the flapdamping timer will give rise to bizarre routing
Common symptom is that bizarre routing will often clearaway by itself
8/6/2019 AfNOG2005 Troubleshooting BGP
102/105
102102102 2005, Cisco Systems, Inc. All rights reserved.APRICOT2005
Troubleshooting Summary
Most troubleshooting is about:
ExperienceRecognising the common problems
Not panicking
Logical approachCheck configuration first
Check locally first before blaming the peer
Troubleshoot layer 1, then layer 2, then layer 3, etc
8/6/2019 AfNOG2005 Troubleshooting BGP
103/105
103103103 2005, Cisco Systems, Inc. All rights reserved.APRICOT2005
Troubleshooting Summary
Most troubleshooting is about:
Using the available tools
The debugging tools on the router hardwareInternet Looking Glasses
Colleagues and their knowledge
Public mailing lists where appropriate
8/6/2019 AfNOG2005 Troubleshooting BGP
104/105
104104104 2005, Cisco Systems, Inc. All rights reserved.APRICOT2005
Closing Comments
Presentation has covered the most commontroubleshooting techniques used by ISPs today
Once these have been mastered, more complexor arcane problems are easier to solve
Feedback and input for future improvements isencouraged and very welcome
8/6/2019 AfNOG2005 Troubleshooting BGP
105/105
Troubleshooting BGPPhilip SmithPhilip Smith
AfNOGAfNOG
20052005
Maputo, MozambiqueMaputo, Mozambique