© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 1 10 Things I Wish I Knew About Networking Before I Needed Them Jimmy Ray Purser, Lead Fried Chicken Eater and Toast Butterer
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 1
10 Things I Wish I Knew About Networking Before I Needed Them
Jimmy Ray Purser, Lead Fried Chicken Eater
and Toast Butterer
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 2
Required By Law Agenda Slide
Journey though my 20 some odd years of networkin’
Geek tips I wish I learned 10 minutes before I needed them
Cool tips and tricks
Other stuff
Plenty of Sci-Fi references
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 3
Lessons learned – Tip 1
!, Remark and DESC it!
Always design your networks for the engineer that follows you.
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 4
Labeling = Awesome
! Are offline config remarks
Remark is for ACL’s
Desc is for interfaces
Don’t forget physical cables!! Label where it goes to. No adhesive labels!!!!
Color Coding…It’s a good thing
Black - From Servers to patch
Blue - From patch to Switch
Gray - From Switch to workstation switches/hubs
Yellow - From Switch to Firewall/Router
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 5
Lessons learned – Tip 2
HSRP and WAAS
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 6
Toggling for fun and profit
HSRP address is a virtual addresses, and it may flip between the routers depending on demand.
When you configure WCCP for use with the Hot Standby Router Protocol (HSRP), you must configure the WAE with the HSRP or the Virtual Router Redundancy Protocol (VRRP) virtual router address as its default gateway, and the WAE WCCP router-list with the primary address of the routers in the HSRP group.
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 7
Turning Point!!!!!!
(A) Logs
(B) Documentation
(C) Lawyers
(D) Poor Chain of Custody
Leading cause of lost court cases?
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 8
Lessons learned – Tip 3
Logging
Only looking for what I know I want to find instead of just looking to see what I find
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 9
Logs are just data… Processed and analyzed, they become information.
- Marcus Ranum
http://www.ranum.com/security/computer_security/archives/logging-notes.pdf
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 10
I’m a Lumberjack and I’m OK
Time stamping logs
- service timestamps log datetime localtime show-timezone msec year.
- service sequence-numbers
- logging server-arp
- BONUS: login delay seconds and/or login block-for <seconds> attempts <tries> within <seconds>
Save Syslogs with GZ/ZIP. They can compress up to 10th orignal size
Unix Syslog D SUCKS! I
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 11
I’m a Lumberjack and I’m OK
Grrrrr…..
TWTVRouter#clo
00:07:31: %SYS-5-CONFIG_I: Configured from console by console
% Incomplete command.
- Don’t turn off logging console just sync it!
TWTVRouter(config)#line con 0
TWTVRouter(config-line)#logging synchronous
TWTVRouter(config-line)#^Z
Router#clo
00:08:39: %SYS-5-CONFIG_I: Configured from console by console
TWTVRouter#clo ! router retyped this
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 12
Lessons learned – Tip 4
Erase START or RUN is normally not a good idea
Cisco devices have specific start up configs. The higher end the device the more this is true.
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 13
To the chopper!
Defaulting is much easier and you don’t look like a knob starting from scratch.
default interface command
default interface Fa0/0.
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 14
Turning Point!!!!!!
(A) Call a sitter/friend
(B) Take him but leave in the car
(C) Take him to the DC
(D) Call wife
On call and just got called in…but I have the kid??
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 15
Lessons learned – Tip 5
The network is broke!
What was the LAST thing done to the network before it broke?
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 16
Two, Two, two lessons in One!
Always, always always save a digital and hardcopy version of your config with the date wrote on it!
Contextual Config Diff command
- show archive config differences start-configuration target-configuration
- show archive config differences nvram:startup-config system:running-
config
Change Management
- archive
- log config
- logging enable
- logging size 200
- notify syslog
- hidekeys
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 17
Turning Point!!!!!!
(A) Tell ‘um to call support
(B) Talk about the physical impossibility of this feat to CU
(C) Ask them to send the logs, packet trace and topology to you
(D) Jump on plane and go see it
10Meg is faster then 1Gb
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 18
Lessons learned – Tip 6
DNS…TTL…The Mother In Law’s of troubleshooting
It works, oh now it doesn’t…oh now it does…no wait….
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 19
ASA FQDN ACL’s woot woot!
Supported as of 8.4(2)
Config a TRUSTED DNS server
Define a FQDN object
Add the FQDN to the ACL
access-list inside_in deny ip any object obj-hr.techwisetv.com access-list inside_in
permit ip any any
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 20
But….my CPU is spiking..
Some sites set a very low TTL <20secs (social media sites; FB, OKRA, Sena, Path…)
Mainly because many folks use Content Delivery Network to serve it up (Amazon CWS, Akamai, etc..)
By default (and at a minimum) the ASA always adds one minute to the TTL
Bottom line: FQDN functionality in ACLs is not a replacement for HTTP Filtering. It cannot distinguish what content is being sent.
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 21
Turning Point!!!!!!
(A) Test it in the lab
(B) Research it
(C) Contact the Psychic Friends Network
(D) Damn the torpedoes Full Speed Ahead!
New Feature Do you??
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 22
Lessons learned – Tip 7
Who keeps calling Cuba?
Can ya get them to ship some Cohiba Singlo III’s ?
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 23
Toll Fraud is for posers
IOS voice gateways default is to accept call setups from all sources
IOS 15.1(2)T
Hey new improvements to Toll Fraud protection BONUS!!!
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 24
On by default!!
Blocks all inbound VoIP call setups
Two stage dialing is off by default (YEAH!)
The GW has to be properly configured to trust these sources
Check logs and look for:
TOLLFRAUD_APP is rejecting the call, it will generate a Q.850 disconnect cause value of 21
Yes you can roll back to pre 15.1(2)T behavior if ya wanna
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 25
Turning Point!!!!!!
(A) Just plug it in goober
(B) Ask the staff to do it
(C) Invent a room temp super conducting battery
(D) Work faster!
Laptop battery is dying!!
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 26
Lessons learned – Tip 8
In The Garden of Debugging
….ummm….what???
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 27
Debug in a Production Network!!!
Chill out man. Typically only 10-20% CPU overhead is needed for even the most verbose debugs
Monitor show processor cpu history
Enable debugs one at a time
Thanks Jimmy Ray but I use syslog for this and so should you…goober…
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 28
The Problem with Syslog and Debugs…
they contain dropped messages and render an accurate analysis impossible
Check it out! Enable sequence numbers on the IOS side. Are they in order?
This is because syslog is UDP by default, and the messages usually end up being rate-limited heavily
Goober??
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 29
This works for me Router(config)# service sequence-numbers
Router(config)# service timestamps debug datetime localtime msec Router(config)# logging buffered 10000000 debug Router(config)# no logging console Router(config)# no logging monitor
Router(config)# default logging rate-limit
Router(config)# default logging queue-limit
<Enable debugs, then wait for issue to occur.>
<Enable session capture to txt file in terminal program.>
Router# terminal length 0
Router# show logging
My Favorite Terminal Progs; PuTTy , SecureCRT. Mac's Terminal app
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 30
OK, But…How do I become a….
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 31
ALPHA GEEK!!!!!
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 32
Alpha Geek Warfare Training: – Tip 1
Be the packet
Wireshark is cool but so is EPC
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 33
Alpha Geek: Wireshark Shortcuts
Boolean logic goes a long ways in WireShark
Plus Wireshark is self correcting!
My Favs:
- tcp.flags.fin == 1 - ftp.request.command== "USER”
-not arp and not stp - ip.src == 192.168.1.114
- icmp.type== 8 - tcp.srcport == 31337
My all time fav: The network is slow filter
tcp.analysis.retransmission
- Also remember stream reassembly!!
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 34
Alpha Geek: Embedded Packet Capture Awesome IOS packet capture right in the CEF path!
Two parter; Config Capture Buffer and Capture Point
Capture Buffer:
- TWTVrouter# monitor capture buffer iospcap1 size 58 max-size 256 circular
Capture Point:
- TWTVrouter# monitor capture point ip cef ipGE0/7 GigabitEthernet 0/7 both
Map ‘um up
- TWTVrouter# monitor capture point associate ipGE0/7 iospcap1
Engage!
- TWTVrouter# monitor capture point start ipGE0/7
- TWTVrouter# monitor capture buffer iospcap1 export tftp://192.168.1.99/iospcap1
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 35
Turning Point!!!!!!
(A) Stay
(B) Go
(C) Use the offer as leverage
(D) Just go fishin’
Should I stay or should I go?
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 36
Alpha Geek Warfare Training: – Tip 2
Hey Man, Nice shot. And other Filter Hits
A little Unix knowledge goes a long ways…
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 37
Alpha Geek: Filtering Commands
Sorting thru configs and logs sucks. Pipe for what you are looking for with: Include, Begin, Exclude
- TWTVRouter# show running-config | include service
- TWTVRouter# show ip route | include 172.16.
- TWTVRouter# show run | begin router rip.
Try this; Tracking a user
-TWTV3750# sh arp | include 192.168.1.44
-TWTV3750# sh mac-address | include 1cf00.4522.e433
- This starts getting you ready for Regular Expressions which are used a lot in UC and BGP
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 38
Alpha Geek Warfare Training: – Tip 3
Fix 92% of network problems.
Make cable test your new bestest pal
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 39
Alpha Geek: Cable Testing!
Time Delay Reflectometer for tri-speed Cu ports!
TWTV3750# test cable-diagnostics tdr interface gigabitethernet 1/0/4
- Switch echo back: “TDR test started on interface Gi1/0/2”
TWTV3750#show cable-diagnostics tdr interface gigabitEthernet 1/0/4
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 40
Cable Fault—TDR
Router#test cable-diagnostics tdr interface GigabitEthernet3/1
Link state may be affected during TDR test
TDR test started on interface Gi3/1
A TDR test can take a few seconds to run on an interface
Use 'show cable-diagnostics tdr' to read the TDR results.
Router#show cable-diagnostics tdr int g3/1
TDR test last run on: April 27 1:29:58
Interface Speed Pair Cable length Distance to fault Channel Pair status
--------- ----- ---- ------------------- ------------------- ------- -----------
-
Gi3/1 100 1-2 N/A N/A Pair A Terminated
3-4 N/A N/A Pair B Terminated
5-6 N/A 5 +/- 2 m Invalid Short
7-8 N/A 5 +/- 2 m Invalid Short
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 41
Alpha Geek Warfare Training: – Tip 4
The ultimate and coolest Cisco feature ever.
EEM is so cool, it should come with it’s own pair of sunglasses
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 42
Alpha Geek: Embedded Event Manager
DEMO
www.cisco.com/go/eem
www.techwisetv.com
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 43
Alpha Geek Warfare Training: – Bonus Tip!
Dark side of network…VOIP
You like your POTS line will be MINE….
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 44
Alpha Geek: My Fav Voice Debugs
H.323
debug voip ccapi inout
debug h225 asn1
debug h245 asn1
debug cch323 all
debug ip tcp transaction
SIP
debug voip ccapi inout
debug ccsip messages
debug voip rtp session named-event
MGCP
debug voip ccapi inout
debug mgcp packet
debug ip tcp transaction
<Be sure to enable appropriate POTS
debugs, too.>
ISDN
debug voip ccapi inout
debug isdn q931
Analog or Non-ISDN POTS
debug voip ccapi inout
debug vpm signal
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 45
http://www.techwisetv.com