Advanced 802.1X Design and Troubleshootingd2zmdbbm9feqrf.cloudfront.net/2011/las/pdf/BRKSEC-3005.pdf · Design and Troubleshooting. ... Use RADIUS proxy to send requests from *.mycorp.com

BRKSEC-3005

Advanced 802.1XDesign and Troubleshooting

© 2011 Cisco and/or its affiliates. All rights reserved. Cisco PublicBRKSEC-3005 2

Agenda

Deployment Considerations

Authentication

Authorization

Optimizing Deployment Scenarios

Low Impact Mode

High Security Mode

Troubleshooting

Methodology

Flows

For Your Reference

Real World Example

Deployment ConsiderationsAuthentication


Authorization

Authentication

Policy

Teamwork & Organization

Credentials,

DBs, EAP,

Supplicants,

Agentless,

Order/Priority

Windows GPO,

machine auth,

PXE, WoL, VM

Network,

IT,

Desktop

Desktops

Multiple Endpoints

Confidentiality

Authentication Considerations


EAPoL Start

EAP-Response Identity: Alice

EAPoL Request Identity

RADIUS Access Request

[AVP: EAP-Response: Alice]

EAP-Request: PEAP

EAP-Response: PEAP

EAP Success

RADIUS Access-Accept

[AVP: EAP Success]

[AVP: VLAN 10, dACL-n]

RADIUS Access-Challenge

[AVP: EAP-Request PEAP]


[AVP: EAP-Response: PEAP]

Multiple

Challenge-

Request

Exchanges

Possible

Beginning

Middle

End

IEEE 802.1X Provides Port-Based Access Control Using Authentication

Layer 2 Point-to-Point Layer 3 Link

Authenticator AAA ServerSupplicant EAP over LAN

(EAPoL)

RADIUS


Choosing Credentials for 802.1X

Username/PwdDirectory

alicec1sC0L1v Certificate

Authority

TokenServer

Deployment Best PracticesReuse Existing Credentials

Understand the Implications of Existing Systems

Common Types

Passwords

Certificates

Tokens

Deciding Factors

Security Policy

Validation

Distribution & Maintenance


Passwords: Not Always As Simple as They Seem

Possible Solutions To Multiple-Domain Issues:1. Establish two-way trust between mycorp.com & mycorp.uk

2. Use RADIUS proxy to send requests from *.mycorp.com to US ACS

3. Use certs with global Enterprise CA and don’t check AD

mycorp.com mycorp.uk1) Two-way trust

2) RADIUS proxyalice.mycorp.com

3) mycorp root CA

alicec1sC0L1v

√Root Cause: Alice is not

a member of mycorp.uk

Directory Structure Can Impact Network Access


Users and Machines Can Have Credentials

alice

User Authentication Machine Authentication

host\XP2 host\XP2

• Enables Devices To Access

Network Prior To (or In the

Absence of) User Login

• Enables Critical Device Traffic

(DHCP, NFS, Machine GPO)

• Is Required In Managed Wired

Environments

• Enables User-Based Access

Control and Visibility

• If Enabled, Should Be In

Addition To Machine

Authentication


Power On

Kernel LoadingWindows HAL LoadingDevice Driver Loading

Why You Must Enable Machine Auth In A Managed EnvironmentEasy

Obtain Network Address(Static, DHCP)

Determine Site and DC(DNS, LDAP)

Establish SecureChannel to AD

(LDAP, SMB)

Kerberos Authentication(Machine Account)

Computer GPOs Loading (Async)

GPO based StartupScript Execution

Certificate Auto EnrollmentTime SynchronizationDynamic DNS Update

GINA

Components that depend on network connectivity

Kerberos Auth(User Account)

User GPOs Loading(Async)

GPO based LogonScript Execution (SMB)

Machine Authentication

UserAuthentication


Example 1: Call Center Objective: Differentiated Access for Agents

Conditions: Shared Use PCs (desktop)

Business Case & Security Policy Determines Whether You Need User Auth

Machine + User

Example 2: Enterprise CampusObjective: Access for Corporate Assets OnlyConditions: One Laptop = One User

Machine Only

Bonus Question:

Could this customer enable

user auth if they wanted to?


Massive Outage After OS Upgrade

Understanding Your Supplicant is EssentialMake Friends With Your Desktop Team

• XP SP2: single service & profile for all

802.1X (wired/wireless)

• XP SP3/Vista/Win7: separate services and

profiles for wired and wireless.

• wired service is disabled by default

• http://support.microsoft.com/kb/953650

• Switch expects 3 failures by default

• XP SP3, Vista, Win7: 20 minute block timer

on first auth fail

• http://support.microsoft.com/kb/957931

• (config-if)#authentication event fail retry 0

Auth Fail VLAN Doesn’t Work

Open Source

Hardware

Native

Premium

http://support.microsoft.com/kb/953650

http://support.microsoft.com/kb/957931


EAPoL: EAP Request-Identity

Any Packet


RADIUS Access-Request

[AVP: 00.0a.95.7f.de.06 ]

Switch RADIUS Server

IEEE 802.1X

Timeout1

MAB2



MAC Authentication Bypass (MAB)“Authentication” for Clientless Devices

00.0a.95.7f.de.06

How Are MACs “Authenticated” ?


MAB is PAP…or you can optimize


MAB as PAP•works with any RADIUS server

•password = username

MAB as “Host Lookup”•ACS optimization

•no need for fake passwords

Differentiates MAB Request


Building Your MAB Database

Export Phone MACs From CUCM


Sample Script To Convert MACs

#!C:\Perl\bin\perl.exe

#script name: convert.pl

#Description: Converts MAC address files exported from Cisco Call Manager# to a format that can be imported into ACS 5

#usage: convert.pl InputFile OutputFile EnableFlag IdentityGroup#alternative usage: convert.pl InputFile OutputFile EnableFlag#alternative usage: convert.pl InputFile OutputFile

if ($#ARGV < 1) {die "Insufficient arguments.\nUsage:convert.pl InputFile OutputFile EnableFlag IdentityGroupconvert.pl InputFile OutputFile EnableFlagconvert.pl InputFile OutputFile\n";

} elsif ($#ARGV < 2) {$EnableFlag = "true";

} else {$EnableFlag = $ARGV[2];

}

open(InFile, $ARGV[0]) or die "Can't open input file $ARGV[0]\n";open(OutFile, ">$ARGV[1]") or die "Can't open output file $ARGV[1]\n";

#print Required ACS Template Header to OutFileprint OutFile 'MACAddress:String(64):Required,description:String(1024),"enabled:Boolean(true,false):Required",HostIdentityGroup:String(256)', "\n";

#Reformat fields and print to OutFilewhile (<InFile>) {

if (s/^SEP//) {@field = split /,/;$field[0] =~ s/(..)(..)(..)(..)(..)/\1-\2-\3-\4-\5-/ ;print OutFile ($mac,$field[0],",",$field[1],",",$EnableFlag,",",$ARGV[3],"\n");

}}close(InFile);close(OutFile);

CUCM -> ACS 5 Format


SNMP, Netflow, DHCP

Building Your MAB DatabaseProfiling Tool

Profiler


LDAP

ISE


Building Your MAB Database

Wildcard Rules Based on MAC Prefixes

00-04-0D-9D-BE-59

Organizationally Unique Identifier (OUI)• Assigned by IEEE

• Identifies device vendor and possible device type


Where Will You Store MACs Once You Get Them?

• Centralized Repository

• 50 K Limit

• Administrative Security Model

ACS Internal Hosts

• Dedicated database

• Distributed Admin Domains

• Mgmt / Failover / RedundancyLDAP

• Username/password (Pre 2003-RC2)

• ieee802Device (2003-RC2/2008):

Active Directory


To Fail or Not to Fail MAB?Two options for unknown MAC addresses

1)No Access

2)Switch-based Web-Auth

3)Guest VLAN

RADIUS-Access Request (MAB)

RADIUS-Access Reject

2) MAC is Unknown and MAB Fails

RADIUS-Access Request (MAB)

RADIUS-Access Accept

Guest Policy

Unknown MAC. Apply Guest Policy

1) MAC is Unknown but MAB “Passes”

• AAA server determines policy for unknown endpoints (e.g. network

access levels, re-authentication policy)

• Good for centralized control & visibility of guest policy (VLAN, ACL)

Deployment ConsiderationsAuthorization


Authorization

Authentication

Policy


Pre-Auth,

VLAN, ACL,

Failed Auth,

AAA down

Desktops

Multiple Endpoints

Phones,

Link State,

VMs,

Desktop Switches

Confidentiality

Authorization Considerations


Authorization Summary

Authentication

Status

Default

Authorization

Alternative 1 Alternative 2

Pre-802.1X / MAB Closed Open Selectively

Open

Successful 802.1X Open Dynamic

VLAN

Dynamic

ACL

Successful MAB Open Dynamic

VLAN

Dynamic

ACL

Failed 802.1X Closed Auth-Fail

VLAN

Next

Method

Failed MAB Closed Guest

VLAN

Next

Method

No 802.1X

(no client)

Closed Guest

VLAN

Next

Method

No 802.1X, MAB

(server down)

Closed Critical

VLAN


RADIUS Access-AcceptAV: url-redirect=http://192.168.10.55/Use_VPN.htm

Web Server

URL Redirect Is Another Authorization Option for MAB and IEEE 802.1X

HTTP://www.google.com

HTTP: //192.168.10.55/Use_VPN.htm

Use_VPN.htm

“Please VPN to

your home network

before accessing

the Internet.”

URL Redirect:• Is NOT Web Authentication

• Allows Custom Notifications

• Persists Until Termination or Reauth

HTTP://Redirect

http://www.google.com/

http://redirect/


ISE

RADIUS Change of Authorization (CoA)Overview

Internet

Example Use Case

1)Guest gets assigned URL

redirect via “Unknown MAC”

2)Guest registers MAC address

via web portal

3)RADIUS “Reauthenticate” CoA

is issued

4)Client passes 802.1X/MAB

and URL redirect is removed

CoA Begins Where 802.1X/MAB Leaves Off

Enables Central,Dynamic Session Control includes failed sessions

RADIUS Server

Web Portal

√ Guest

BRKSEC-2041


Configuration commands

aaa server radius dynamic-authorSwitch(config)#

• Configure the switch as AAA server to facilitate interaction with an

external policy server.

Switch(config-locsvr-da-radius)#

client {ip-address | name} [vrf vrfname] [server-key string]

• Enter dynamic authorization local server configuration mode and

specify a RADIUS client from which a device will accept CoA and

disconnect requests.

Switch(config-locsvr-da-radius)#

port radius-server-port

• The switch defaults to port 1700. ACS 5.1 defaults to port 3799. This

must be set to port 3799 on the switch to use with ACS 5.


Triggering CoA from ACS 5.1Select Session and CoA Type


Security Group (SG) Tags are Another Form of Authorization

SGACL

Security Group Based Access Control:

Provides topology independent policy

Flexible and scalable policy based on user role

Centralised Policy Management for Dynamic policy provisioning

Egress filtering results to reduce TCAM impact

802.1X/MAB/Web Auth

Database (SGT=4)

IT Server (SGT=10)

I’m a contractor

My group is IT Admin

Contactor

& IT Admin

SGT = 100

SGT = 100

SGT capable device


Phones Need To Be Authorized on a Multi-Domain (or Multi-Auth) Port

RADIUS-Access Request


device-traffic-class=voice

VLAN: Purple

Voice VLAN Enabled

Single Host Multi-Domain Multi-Auth

Deployment ScenariosOptimizing Phased Deployments


Authorization

Authentication

Policy


Credentials,

DBs, EAP,

Supplicants,

Agentless,

Order/Priority

Pre-Auth,

VLAN, ACL,

Failed Auth,

AAA down

Windows GPO,

machine auth,

PXE, WoL, VM

Definition,

Enforcement,

RolloutNetwork,

IT,

Desktop

Desktops

Multiple Endpoints

Phones,

Link State,

VMs,

Desktop Switches

Confidentiality

Encryption

Considering Deployment Scenarios


Three Deployment Scenarios

Monitor Mode

• Authentication Without Access Control

Low Impact Mode

• Minimal Impact to Network and Users

High Security Mode

• Logical Isolation of User Groups / Device Types


Low Impact Mode

Begin to control/differentiate network access

Minimize Impact to Existing Network Access

“Low Impact” == no need to re-architect your network

Keep existing VLAN design

Minimize LAN changes

Start with Monitor Mode

Add PreAuth ACL

Dynamically download ACL after authentication

Low Impact Mode Uses ACLs for Tunable Access Control

Before

After


Pre-Auth Port ACL Considerations

Pre-auth port ACL is arbitrary and can progress as you better

understand the traffic on your network

Recommendations: use least restrictive ACL that you can, time-

sensitive traffic is a good candidate for ACL.

Approach 1: Selectively block traffic

Selectively protect certain assets/subnets

Low risk of inadvertently blocking wanted traffic

Example: Block unauthenticated users from Finance servers

Approach 2: Selectively allow traffic

More secure, better control

May block wanted traffic

Example: Only allow pre-auth access for PXE devices to boot


ACL Configuration Notes 802.1X/MAB Web-Auth

Downloadable

ACL• On ACS • Centralized

• No size limitation*

• Requires ACS

• 3K: 12.2(50)SE

• 4K: 12.2(50)SG

• 6K: 12.2(33)SXI

• 3K: 12.2(50)SE

• 4K: 12.2(50)SG

• 6K: 12.2(33)SXI

PerUser • On AAA server • Centralized

• Length limited to

RADIUS packet size*

• Supports 3rd party AAA

servers

• 3K: 12.2(50)SE

• 4K: 12.2(52)SG

• 6K: 12.2(33)SXI3

• Not Supported

Filter-id • ACL name on

AAA server

• ACL contents on

switch

• Distributed

• No size limitation*


servers

• 3K: 12.2(50)SE

• 4K: 12.2(52)SG

• 6K: 12.2(33)SXI3

• 3K: 12.2(50)SE

• 4K: Not Supported

• 6K: Not Supported

Proxy • On AAA server • Centralized

• Web-Auth only

• Length limited to

RADIUS packet size*


servers

• Not Supported • 3K: 12.2(35)SE

• 4K: 12.2(50)SG

• 6K: Not supported

Dynamic ACL Types for Authentication

*Size refers to defined length of ACL. TCAM limits on switch still apply.


Use downloadable ACLs

If no ACS, use per-user ACLs (centralized)

If no ACS, use Filter-ID ACLs (distributed)

Try to avoid WebAuth Proxy ACLs

ACL Rules of Thumb


Two Deployment Concerns with ACLs

Which Comes First?

• Migrating from Monitor Mode requires adding Port ACLs and dACLs.

• Port ACLs restrict all traffic

• dACLs alone can cause problems

dACLs, dACLs everywhere

• Because of the Port ACL, everybody has to have a dACL…

• …even if they don’t “need” it

• Lots of dACLs to configure


Transition Gracefully from Monitor Mode

Handling dACLs without PACL

SSC

%AUTHMGR-5-FAILPrior to12.2(54)SG and12.2(55)SE, a switch that receives a dACL for a port without a PACL will fail authorization.

After 12.2(54)SG and12.2(55)SE, the switch will automatically attach a default PACL called “Auth-Default-ACL” and then apply dACL.

%EPM-6-AUTH_ACL: POLICY Auth-Default-ACL

dACL-n


permit udp any any eq bootpspermit udp any host 10.100.10.116 eq domainpermit udp any host 10.100.10.117 eq tftp

SSC

port

ACL

Reduce dynamic ACL configurationOpen Directive obsoletes “permit ip any any”

If no dynamic ACL is downloaded, Pre-Auth Port ACL controls the port

Every endpoint must be assigned a dynamic ACL

Switch(config)#epm access-control open

If the RADIUS server returns a dynamic ACL, dynamic ACL is applied.

If no dynamic ACL returned, switch automatically creates a “permit ip host any” entry for the authenticated host

Default behavior:

With “open directive” configured:

12.2(54)SG

12.2(55)SE

permit ip any any


Low Impact In a Nutshell

• Default open + pre-auth ACL

• Differentiated access control using dynamic ACLs

Summary

• Minimal Impact to Endpoints

• Minimal Impact to Network

• No L2 Isolation

• Some access prior to authentication

Benefits & Limitations

• Start with least restrictive port ACLs

• Use downloadable ACLs if you have ACS

• Use Open Directive to reduce dACL config

• Use transient control on sSW for NEAT

Recommendations


High Security: How To

Return to default “closed” access

Timers or authentication order change

Implement identity-based VLANassignment

High Security Mode Goals

No access before authentication

Rapid access for non-802.1X-capable corporate assets

Logical isolation of traffic at the access edge

High Security Mode Uses VLANs for Logical Isolation

Network Virtualization Solution

See BRKRST-2033 for more on Network Virtualization


802.1X and Dynamic VLANsNetwork Deployment Considerations

VLAN 10: DATA

VLAN 20: VOICE

VLAN 30: MACHINE

VLAN 40: ENG

VLAN 50: UNAUTH

10.10.10.x/24

10.10.20.x/24

10.10.30.x/24

10.10.40.x/24

10.10.50.x/24

Network Interface

10.10.10.x/24 G0/1

10.10.20.x/24 G0/2

10.10.30.x/24 G0/3

10.10.40.x/24 G0/4

10.10.50.x/24 G0/5

Every Assignable VLAN Must Be Defined on Every Access Switch More VLANs To Trunk (Multi-Layer* Deployments)

More Subnets to Route (mitigated by VSS*)

Every Assignable VLAN Must Be Defined on Every Access Switch

Best Practice: Use the Fewest Possible Number of VLANs

*For More Details on Campus Design, see BRKCRS-2031: Multilayer Campus Architectures and Design Principles


802.1X and Dynamic VLANsEndpoint Deployment Considerations

Non-802.1X Endpoints

• Unaware of VLAN changes, no mechanism to change IP address

• Best Practice: Dynamic VLAN in High Security Mode only

Older 802.1X Endpoints (e.g. Windows XP)

• Supplicants can renew IP address on VLAN change but OS and underlying processes may not handle IP address change gracefully

• Best Practice: Use same VLAN for User and Machine Authentication (Windows)

Newer 802.1X Endpoints (e.g. Windows Vista, 7)

• Supplicant and OS can handle VLAN/IP address changes

• Best Practice: Use the VLAN policy that best matches your security policy.


XP: Microsoft Remote Desktop logs off the local user and drops the machine into machine mode which results in a machine auth.

If machine authentication and user authentication result in the same VLAN then there are no problems

If machine authentication puts the machine in a different VLAN, then RDC breaks.

SSC / AnyConnect on XP can be configured to extend the connection

Vista / Win 7: Leaves the local user logged onto the system, so it does not trigger 802.1X.

Remote Desktop and Windows XP For Your

Reference


802.1X, Dynamic VLANs, and WoL

interface fastEthernet 3/48

dot1x pae authenticator

authentication port-control auto

authentication control-direction in

Unidirectional Access Control

802.1X + WoL Challenge:

• Device flaps link when sleeping

• 802.1X session cleared

• No network access (closed mode)

• WoL packet can’t get through

802.1X + WoL + dVLAN:

• Devices flap link when they sleep

• 802.1X Session Cleared

• VLAN reverts to access VLAN

• WoL packet goes to dVLAN subnet

• Don’t assign VLANs to WoL devices

• Use Low Impact Mode

• Use hardware (Intel AMT) supplicant

• Build VLAN awareness into WoL

server

Dynamic VLAN + WoL Solutions


Avoid VLAN Name Changes with User Distribution

Access-Accept:

VLAN: corporate30

switch1

switch231

vlan 30

name corporate

vlan 31

name corporate-1

vlan group corporate vlan-list 31

Traditional VLAN Assignment

Is by VLAN Name

User Distribution Assigns

by VLAN Group (or Name)

• Allows Flexible Adaption in Existing Environments

• No Need to Reconfigure Existing VLANs

• Also Enables Load Balancing


<groupname>: Name for the VLAN group starting with an

alphabet

<list of VLANs>: Comma separated VLANs or a range of

VLANs or a single VLAN

Configuring User Distribution

Switch(config)# vlan group <groupname> vlan-list <list of vlans>

Switch(config)#vlan group corporate vlan-list 4

Switch(config)#vlan group corporate vlan-list 40-50

Switch(config)#vlan group corporate vlan-list 12,52,75


Limited Dynamic VLAN Assignment Now Available for Multi-Auth

Access-Accept:

VLAN: BLUE

VM

Access-Accept:

VLAN: BLUE

Access-Accept

12.2(55)SE15.0(2)SG3.2.0SG

• First successful authentication “locks” the Data VLAN

• Subsequent endpoints must get assigned same VLAN

or no VLAN


switch(config-if)#authentication event server dead action authorize vlan 52Critical VLAN

switch(config-if)#authentication event server dead action reinitialize vlan 52

12.2(52)SE15.0(2)SG

Critical VLAN Now Supported With Multi-Auth


Phones Rely on RADIUS Server

00.18.ba.c7.bc.ee

RADIUS-Access

Request: 00.18.ba.c7.bc.ee


device-traffic-class=voiceVoice VLAN Enabled“Only the VSA

can save the

phone!”

00.18.ba.c7.bc.ee

DataVLAN Enabled




authentication event server dead action authorize

Does Not Save

Phones


Critical Voice VLAN Saves Phones When AAA Server Dies

00.18.ba.c7.bc.ee

DataVLAN Enabled




authentication event server dead action authorize

authentication event server dead action authorize voice

Voice VLAN Enabled

#show authentication session int f3/48

…

Critical Authorization is in effect for domain(s) DATA and VOICE

15.0(2)SG


Feature Update:MACSec Adds Confidentiality, Integrity

Without MACSec, IEEE 802.1X cannot ensure

confidentiality or integrity of the traffic after

authentication.

Alice Rogue AP can extend

attack outside physical

perimeter. Rogue users with

physical access can

monitor and spoof.

SSC


Feature Update:MACSec Adds Confidentiality, Integrity

Alice

AC 3

MACSec protects the port

after IEEE 802.1X

Even with physical access,

rogue users cannot monitor

or spoof encrypted traffic



EAPoL: EAP-Response: AliceRADIUS Access-Request

[AVP: EAP-Response: Alice]

EAP Success


[AVP: EAP Success]

[AVP: EAP Key Name]

[AVP: CAK]

RADIUS Access-Challenge

[AVP: EAP-Request: PEAP]

Authentication

and Master

Key

Distribution

Session

Key

Agreement

Authenticator Authentication ServerSupplicant

1

2

Session

Secure

3

EAPoL-MKA: Key Server

EAPoL-MKA: MACSec Capable

EAPoL-MKA: Key Name, SAK

EAPoL-MKA: SAK Installed

Encrypted Data

Encrypted Data

MACSec Functional Sequence

AES-GCM-128

IEE

E 8

02

.1X

MK

AM

AC

Se

c


MACSec Configuration

interface GigabitEthernet1/0/25switchport access vlan 20switchport mode accessswitchport voice vlan 21authentication port-control automacsecmka default-policydot1x pae authenticatorspanning-tree portfast

AnyConnect 3.03560-X/3750-X12.2(53)SE1

(config-if)#authentication linksec policy ?

must-not-secure Never secure sessions

must-secure Always secure sessions

should-secure OPTIONALLY secure sessions

Required

Op

tional


MACSec Summary

MACSec secures communication on the LAN when you need it

MACSec requires new hardware

MACSec offers confidentiality and integrity while preserving network intelligence


High Security In a Nutshell

• Default closed

• Differentiated access control using dynamic VLANs

Summary

• Logical Isolation at L2

• No Access for Unauthorized Endpoints

• Impact to Network

• Impact to Endpoints

Benefits & Limitations

• Use fewest VLANs possible

• Know which devices can’t change VLANs

• User Distribution helps with VLAN names

• Enable Critical Voice VLAN

• Consider MACSec as needed

Recommendations

Troubleshooting Failed Authorizations Failed Authentications Timeout-related Issues Server-dead Issues IP Telephony Issues


Troubleshooting In Perspective

Enterprise Customer

70,000 Endpoints

Windows Native Supplicant

PEAP-MSCHAPv2

Additional Support Staff:

< 5 Hours / Week

“The typical user is unaware of the 802.1X implementation.”


Troubleshooting MethodologyGeneral Recommendations

Develop & Document a Methodology

Be aware of role dependencies

Start where info density is highest

Good AAA server can diagnose most failed authentications

Switch (CLI, SNMP, syslog) helps with:

Failed authorizations

Current port status

Client side info sometimes helpful

Sniffer Traces Often Definitive

1

4

7 8

5

2 3

6

9

SSC

C:\Documents And Settings\All Users\Application

Data\Cisco\Cisco Secure Services Client

C:\ProgramData\Cisco\Cisco Secure Services Client

netsh ras set tracing eapol enable

netsh ras set tracing rastls enable

%systemroot%\tracing\EAPoL.log

Mic

rosoft

Native

SS

C


Components & Roles

SSC

RADIUS Authentication

RADIUS Accounting

syslog

syslog

show commands

debug commands

SNMP

sys tray icon

logs / event viewer

Component Instrumentation Troubleshooting Role

Bird’s eye view

Central Node

Authentication Status

Central Policy Definition

Port Policy Definition

Local Authorization Definition

Policy Enforcement Status

status messages Status Verification

Client Side Authentication

Au

tho

riza

tion

Acco

un

ting

Au

the

nticatio

n

Syslo

g,

SN

MP

*

*ACS 5.1


802.1X Passed Authentication: Expected

Start 802.1X

AAA-based Authz?

Switch config’d

for authz?

Rcv’d dACL?

R’cv’d dynamic VLAN?

Port ACL defined on

switch?

VLAN defined on

switch?

Port ACL + dACL Dynamic VLANStatic Port Config:Switchport VLAN + Port ACL (if any)F

inal P

ort

Sta

tus

N

Y

Y

Y

N

Y

Y Y

Au

then

ticati

on

Pro

cess

802.1XPass

High Security Mode

Low Impact Mode


Helpful IOS Show Commands

#show authentication session inter g1/13

Interface: GigabitEthernet1/13

MAC Address: 0014.5e95.d6cc

IP Address: 10.100.60.200

User-Name: Administrator

Status: Authz Success

Domain: DATA

Oper host mode: multi-domain

Oper control dir: both

Authorized By: Authentication Server

Vlan Policy: 60

Session timeout: N/A

Idle timeout: N/A

Common Session ID: 0A640A050000147711045C10

Acct Session ID: 0x00001479

Handle: 0x5E000477

Runnable methods list:

Method State

dot1x Authc Success

mab Not run

webauth Not run

show ip access-list interface gi1/13

show run interface g1/13

show epm session ip 10.100.60.200

show ip access-list <xACSACLx-xxxxxx-xxxxx>

12.2(33)SXI

12.2(50)SE

12.2(50)SG

Earlier Versions: show dot1x interface g1/13 details

General Diagnostic

downloadable ACLs

show tcam interface g1/13 acl in ip

dACL + port ACL elements

Input actual dACL name from epm session output

Shows actual dACL ACEs with source substitution

Will be “N/A” if not

dynamically assigned

show vlan

Current VLAN (if not dynamic)

For port ACL


802.1X Passed Authentication ProblemsDynamic Authorization Not Enabled

Start 802.1X

AAA-based Authz?

Switch config’d

for authz?

Static Port Config:Switchport VLAN + Port ACL (if any)

Fin

al P

ort

Sta

tus

Y

N

Au

then

ticati

on

Pro

cess

802.1XPass


Problem 1: Port Authorized but Dynamic Authz Not Applied

Detection: Difficult to detect (no indication that 802.1X is to blame)

Root Cause: Incomplete Switch Config

Resolution: (config)# aaa authorization network default group radius

End User

• Access: default port config

• “I don’t have enough access” or “I have too much access”

AAA Server

• Authentication Passed

Access Switch

• Port is authorized but without dynamic VLAN or dACL

• No syslog -- this is not an error


802.1X Passed Authentication ProblemsACL Not Configured

Start 802.1X

AAA-based Authz?

Switch config’d

for authz?

Rcv’d dACL?

Port ACL defined on

switch?

Authz Fail:Quiet Period

Static Port Config:Switchport VLAN + Port ACL (if any)

Fin

al P

ort

Sta

tus

Y

N

Y

Y

N

Au

then

ticati

on

Pro

cess

802.1XPass

ACL Enhance

ment?N


Problem 2: Authentication Passed but ACL Authorization Failed

Detection: End User, Switch syslogs & epm logging, no Accting

Root Cause: Incorrect Switch Config, pre-12.2(54)SG

Resolution: (config-if)# ip access-group PRE-AUTH in

End User

• Pre-Authentication Access only

AAA Server


Access Switch

• %AUTHMGR-7-RESULT: Authentication result 'success' from 'dot1x' for client (0014.5e95.d6cc) on Interface Gi1/13

• %AUTHMGR-5-FAIL: Authorization failed for client (0014.5e95.d6cc) on Interface Gi1/13

• With “epm logging” configured:

• %EPM-4-POLICY_APP_FAILURE: IP=0.0.0.0| MAC=0014.5e95.d6cc|POLICY_TYPE=Named ACL| POLICY_NAME=xACSACLx-IP-PERMIT-ANY-4999ced8 | RESULT=FAILURE| REASON=Interface ACL not configured


802.1X Passed Authentication ProblemsBad VLAN Assignment

Start 802.1X

AAA-based Authz?

Switch config’d

for authz?

Rcv’d dACL?


VLAN defined on

switch?


Fin

al P

ort

Sta

tus

Y

Y

N

Y

N

Au

then

ticati

on

Pro

cess

802.1XPass


Problem 3: Authentication Passed but VLAN Authorization Failed

Detection: End User, Switch syslogs, no Accting

Root Cause: Incorrect Switch Config

Resolution: (config-vlan)# name Employee

End User


AAA Server


Access Switch

• %AUTHMGR-7-RESULT: Authentication result 'success' from 'dot1x' for client (0014.5e95.d6cc) on Interface Gi1/13

• %DOT1X_SWITCH-5-ERR_VLAN_NOT_FOUND: Attempt to assign non-existent or shutdown VLAN Employee to 802.1x port GigabitEthernet1/13

• %AUTHMGR-5-FAIL: Authorization failed for client (0014.5e95.d6cc) on Interface Gi1/13


ACS 5.1 Syslog Collector Can Help Here!


When Syslogs Are Too Much of A Good Thing

• Device-level syslog filtering & programmable framework

• Limited platform support

Embedded Syslog Manager (ESM)

• #no [authentication | dot1x | mab] syslog verbose

• limited filtering

Syslog suppression CLI

• #logging trap 5

• Filters all syslogs (not just authentication syslogs)

Filter by severity


802.1X Failed Authentication Flow

Start 802.1X

Event fail action

config’d?

Auth Fail VLAN

conf’d?

MABpass?

Web-Auth config’d?

Auth Fail VLAN1,4Pre-Auth Access2

Fin

al P

ort

Sta

tus

Y

Y

Au

then

ticati

on

Pro

cess 802.1X

Fail

Restart Timer

config’d?

RestartTimer

Expires

AAABased

Authz 2,3,4

1Subject to change on receipt of EAPoL-Logoff2All subsequent EAP traffic will be dropped until reauth or link down3See 802.1X Passed Flowchart for details4May be impacted by supplicant behavior

Valid username

/ pwd?

Valid dACL & priv-lvl=15?

dACL + fallback ACL2,4

fallback ACL2

N

Y

N

Y

Y

N

N

> Max Attempt?

Y

N

Y

N

N

QuietPeriod Expires

Y

NHigh Security Mode

Low Impact Mode


How to Get Out of the Auth-Fail States

• Failed Authorization States are deliberately

hard to escape.

• EAPoL-Starts ignored in all Auth Fail states

• Methods to get out of Auth-Fail States:1. Unplug (Link down clears 802.1X session)

Endpoint must be directly connected OR connected to

CDP-Second-Port Capable Phone.

2. Endpoint sends EAPoL-Logoff

Varies by supplicant. Doesn’t work for MAB or Web-

authorized states after 802.1X failure.

3. Local re-authentication timer on switch expires

Local re-auth has other consequences that may not be

desirable.


Syslogs and Accounting for Auth-Fail VLAN

*Mar 27 12:25:16: %AUTHMGR-7-RESULT: Authentication result 'fail' from 'dot1x' for client (0014.5e95.d6cc) on Interface Gi1/13

*Mar 27 12:25:17: %AUTHMGR-5-SUCCESS: Authorization succeeded for client (0014.5e95.d6cc) on Interface Gi1/13

*Mar 27 12:25:16: %AUTHMGR-SP-5-VLANASSIGN: VLAN 10 assigned to Interface Gi1/13

*Mar 27 12:25:17: RADIUS(00000480): Send Accounting-Request to 10.100.10.150:1813 id 1646/23, len 233

*Mar 27 12:25:17: RADIUS: authenticator E4 E7 62 2B 34 63 5A 6E - C8 7E D9 35 55 86 E2 D2

*Mar 27 12:25:17: RADIUS: Acct-Session-Id [44] 10 "0000047D"

*Mar 27 12:25:17: RADIUS: Vendor, Cisco [26] 49

*Mar 27 12:25:17: RADIUS: Cisco AVpair [1] 43 "audit-session-id=0A640A050000047B159C623C"

*Mar 27 12:25:17: RADIUS: Acct-Authentic [45] 6 Local [2]

*Mar 27 12:25:17: RADIUS: Framed-IP-Address [8] 6 10.100.10.240

*Mar 27 12:25:17: RADIUS: User-Name [1] 15 "Administrator"

*Mar 27 12:25:17: RADIUS: Vendor, Cisco [26] 32

*Mar 27 12:25:17: RADIUS: Cisco AVpair [1] 26 "connect-progress=Call Up"

*Mar 27 12:25:17: RADIUS: Acct-Status-Type [40] 6 Start [1]

*Mar 27 12:25:17: RADIUS: NAS-Port-Type [61] 6 Ethernet [15]

*Mar 27 12:25:17: RADIUS: NAS-Port [5] 6 50113

*Mar 27 12:25:17: RADIUS: NAS-Port-Id [87] 21 "GigabitEthernet1/13"

*Mar 27 12:25:17: RADIUS: Called-Station-Id [30] 19 "00-19-AA-7A-8B-4C"

*Mar 27 12:25:17: RADIUS: Calling-Station-Id [31] 19 "00-14-5E-95-D6-CC"

*Mar 27 12:25:17: RADIUS: Service-Type [6] 6 Framed [2]

*Mar 27 12:25:17: RADIUS: NAS-IP-Address [4] 6 10.100.10.5

*Mar 27 12:25:17: RADIUS: Acct-Delay-Time [41] 6 0

*Mar 27 12:25:17: RADIUS: Received from id 1646/23 10.100.10.150:1813, Accounting-response, len 20

*Mar 27 12:25:17: RADIUS: authenticator 03 95 5C 7D B1 3B D1 02 - D7 49 C3 F1 44 D5 03 E6

Auth-Fail VLAN:authentication event fail authorize vlan 10

Switch has locally authorized this session


802.1X Failed Authentication Overview

Detection: End User, AAA records, Switch syslogs

Root Cause: EAP negotiation or credential issue

Resolution: depends on root cause

End User


AAA Server

• Best source of info for 802.1X failures

• Start Troubleshooting here!

Access Switch

• *Mar 5 11:31:41: %AUTHMGR-7-RESULT: Authentication result 'fail' from 'dot1x' for client (0014.5e95.d6cc) on Interface Gi1/13


802.1X FailuresIncompatible EAP Methods

Applies to:

All 802.1X authenticationsBonus Question:

Why is there a passed auth

record after the failure?

Resolution:Configure at least one common EAP method (inner & outer) on ACS and supplicant

Error: Supplicant configured for PEAP, AAA for EAP-TLS

Error: Supplicant configured for PEAP-GTC, AAA for PEAP-MSCHAPv2


Error: Known User, Password Expired

Error: Unknown User

802.1X Credential FailuresPasswords

Applies to: Password-based EAP methods (PEAP-MSCHAPv2, MD5, EAP-FAST)

Bonus Question:Why is there a

passed auth record after this failure?

Error: Known User, Bad Password


802.1X Credential FailuresServer Certificates

Applies to: EAP methods that use server-side

TLS tunnel: e.g. EAP-TLS, PEAP

Typical Error Message:12321 PEAP failed SSL/TLS handshake

because the client rejected the ACS local-

certificate

server

CA

Most Common Root Causes:•AAA server cert signed by a CA

chain that client doesn’t trust

•AAA server cert disallowed by

client’s trusted server rules

•AAA server cert expired

•AAA server cert lacks Server Auth

EKU

EAP-Response

TLS-Alert:

“Unknown CA”

Note: Server won’t know why its cert was rejected

unless client provides info in optional TLS Alert and

server makes Alerts visible (Alerts are supported by

SSC & ACS 5).

Windows Tip:If Authentication passes

when you unclick this box, the

supplicant doesn’t trust the server cert!


802.1X Failures: Client Certificate

Applies to:

EAP methods that use

client-side TLS tunnel:

e.g. EAP-TLS

Typical Error Message:12514 EAP-TLS failed SSL/TLS

handshake because of an unknown CA in

the client certificates chain

12515 EAP-TLS failed SSL/TLS

handshake because of an expired CRL

associated with a CA in the client

certificates chain

12516 EAP-TLS failed SSL/TLS

handshake because of an expired

certificate in the client certificates chain

serverCA

Server Cert Authentication:Signed by trusted CABelongs to allowed server

client CA

Most Common Root Causes:•Client cert signed by a CA chain

that AAA server doesn’t trust

•Client cert expired

•Client cert CRL expired


802.1X Failure vs. 802.1X Timeout

An 802.1X failure occurs when the AAA server rejects the

request:

A timeout occurs when an endpoint can’t speak 802.1X:

EAPoL Start

EAPoL Response Identity



EAP FailureRADIUS Access Reject

SSC


EAP Who?


Guest VLAN†

802.1X Timeout Authentication Flow

Start 802.1X

MAB config’d?

MAB pass?

Fin

al P

ort

Sta

tus

Au

then

ticati

on

Pro

cess 802.1X

Time out

AAABasedAuthz*

Web-Auth config’d?

Event no-responseconfig’d?

Valid username

/ pwd?

Valid dACL & priv-lvl=15?

dACL + fallback ACL

fallback ACL

N

Y

Y

N

Y

Y

Y

N

N

N

Pre-Auth Access †

RestartTimer

Expires

Y

N

Restart Timer

config’d?

N

Y

High Security Mode

Low Impact Mode

*See 802.1X Passed Flowchart for details

†Subject to change on receipt of EAPoL-Start if 802.1X has priority


Common Timeout-Related Problems

Too long

Symptoms

• No IP address

• PXE fail

Root Cause

• DHCP timeout < 802.1X timeout

Solutions

• Shorten timers, MAB first.

• Low Impact Mode.

Too short

Symptoms

• Wrong access levels

• Excessive control traffic

Root Cause

• Switch gives up on 802.1X too soon

Solutions

• Enable EAPoL-Starts

• 802.1X has priority

Just right

Requirement

• Testing in your network

Alternatives

• Low Impact Mode

• MAB first


802.1X Server Dead Flow

Start 802.1X

Event server dead

config’d?

Fin

al P

ort

Sta

tus

Au

then

ticati

on

Pro

cess AAA dead

N

Pre-Auth Access

Y

Critical VLAN

Re-auth 802.1X

AAA dead

N

Pre-Auth Access

Y

Existing Auth

Event server dead

config’d?


802.1X Server Alive Flow

Critical state

Event server alive config’d?

Fin

al P

ort

Sta

tus

Au

then

ticati

on

Pro

cess AAA alive

Y

N

Existing Auth

Start 802.1X


Misconfigurations Can Lead to Appearance of Dead Server

Symptoms ACS5 Log / Root Cause / Resolution

All authentications fail from a

switch or groups of switches.

Switch declares a functioning

AAA server dead.

Switch may deploy Critical

VLAN.

Root Cause: AAA server does not accept RADIUS requests from

this switch

Resolution: Configure AAA server to accept requests from this

switch.

Root Cause: Shared secret is not the same on switch and AAA

server

Resolution: Configure same shared secret on switch and AAA

server


Other Server-Dead Causes:

1. Server is actually dead

Should never happen in HA deployment of AAA server

2. IP connectivity dead

Should never happen in campus, maybe in branch office

3. AAA server is fine but backend database (AD, LDAP) is non-responsive

AAA server has two choices – send a Reject to switch or send nothing (behavior is configurable on ACS v5)

If send a Reject, switch will continue to use this AAA server (cannot distinguish between Reject due to bad credentials and Reject due to process failure)

If send nothing, switch can use local failover mechanisms (e.g. try next server in AAA server group or deploy critical VLAN)


802.1X Passed Auth for IP Phones:Expected Behavior with Multi-Domain Authentication (MDA)

Start 802.1X

AAA-based Authz?

Switch config’d

for authz?

Rcv’d dACL?


Port ACL defined on

switch?

VLAN defined on

switch?

Static Voice VLAN,Port ACL + dACL

Dynamic Voice VLAN

Fin

al P

ort

Sta

tus

Y

Y

Y

N

Y

Y Y

Au

then

ticati

on

Pro

cess

802.1XPass

Rcv’d device-traffic-

class=voice?

Static Voice VLAN

High Security Mode

Low Impact Mode

N

Y


Y

N

Rcv’d device-traffic-

class=voice?

802.1X Passed Authentication for IP Phones: Authorization Problems with MDA

Start 802.1X

AAA-based Authz?

Switch config’d

for authz?

Rcv’d dACL?


Port ACL defined on

switch?

VLAN defined on

switch?

Fin

al P

ort

Sta

tus

Y

Y

N

Y

Au

then

ticati

on

Pro

cess

802.1XPass

Access to DATA VLAN only

N N


N

Y

PC behind phone?

N

Security Violation

YPC

behind phone?

YN

N


Passed 802.1X IP Phones Summary

Problem Cause

Phone in data VLAN Switch did not receive or process the

device-traffic-class=voice VSA from

AAA server

Security violation Phone (with attached PC) either:

• Failed to authenticate

• Failed to get authorized

• Failed to receive voice VSA

In each case, switch assumes phone

is data device. Switch expects 1 data

device & 1 voice device per port.


802.1X Failure Flow for IP Phones with MDA

Start 802.1X

Event fail action next-

method?

MAB pass?

Pre-Auth Access

Fin

al P

ort

Sta

tus

Y

Au

then

ticati

on

Pro

cess

802.1XFail?

Restart Timer

config’d?

RestartTimer

Expires

AAABasedAuthz*

*See 802.1X IP Phone Passed Flowchart for details

Y

N

PC Behind Phone?

SecurityViolation

N

N

Y

N

Event fail action VLAN?

Auth-Fail VLAN

Y

Web-Auth

config’d?

data VLAN, fallback ACL

YY

NN


802.1X Timeout Flow for IP Phones

Start 802.1X

MAB config’d?

MAB pass?

Pre-Auth Access

Fin

al P

ort

Sta

tus

Y

Au

then

ticati

on

Pro

cess

802.1XTime Out

Restart Timer

config’d?

RestartTimer

Expires

AAABasedAuthz*

*See 802.1X IP Phone Passed Flowchart for details

Y

N

PC Behind Phone?

Security Violation

NN

Y

N

Event no-

response VLAN?

Guest VLAN

Y

Web-Auth

config’d?

data VLAN, fallback ACL

YY

N

N


Conclusion


Authorization

Authentication

Policy


Credentials,

DBs, EAP,

Supplicants,

Agentless,

Order/Priority

Pre-Auth,

VLAN, ACL,

Failed Auth,

AAA down

Windows GPO,

machine auth,

PXE, WoL, VM

Definition,

Enforcement,

RolloutNetwork,

IT,

Desktop

Desktops

Multiple Endpoints

Phones,

Link State,

VMs,

Desktop Switches

Confidentiality

Encryption

Think at the System-Level


Key Takeaways

Port-based access control is a system

Multiple protocols, multiple features, multiple products

Start Simple and Evolve

Monitor mode before access control

Least restrictive ACLs, fewest VLANs

Optimize Deployment Scenarios With New Features

Document expected flows for your implementation

Know where every device & user should / could end up

Start at a central point, work outward as required – a good AAA server is invaluable


Where To Find Out More

Whitepapers

Deployment Scenario Design Guide http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps65

86/ps6638/whitepaper_C11-530469.html

Deployment Scenario Config Guide http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps65

86/ps6638/Whitepaper_c11-532065.html

IEEE 802.1X Deep Dive http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps65

86/ps6638/guide_c07-627531.html

MAB Deep Dive http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps65

86/ps6638/config_guide_c17-663759.html

Web Auth Deep Dive http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps65

86/ps6638/app_note_c27-577494.html

http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps65

86/ps6638/app_note_c27-577490.html

Flex Auth App Note http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps65

86/ps6638/application_note_c27-

573287_ps6638_Products_White_Paper.html

IP Telephony Deep Dive http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps65

86/ps6638/config_guide_c17-605524.html

MACSec Deep Dive http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps65

86/ps6638/deploy_guide_c17-663760.html

www.cisco.com/go/ibns

http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps6586/ps6638/whitepaper_C11-530469.html




http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps6586/ps6638/Whitepaper_c11-532065.html




http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps6586/ps6638/guide_c07-627531.html




http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps6586/ps6638/config_guide_c17-663759.html




http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps6586/ps6638/app_note_c27-577494.html








http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps6586/ps6638/application_note_c27- 573287_ps6638_Products_White_Paper.html









http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps6586/ps6638/deploy_guide_c17-663760.html





Receive 25 Cisco Preferred Access points for each session evaluation you complete.

Give us your feedback and you could win fabulous prizes. Points are calculated on a daily basis. Winners will be notified by email after July 22nd.

Complete your session evaluation online now (open a browser through our wireless network to access our portal) or visit one of the Internet stations throughout the Convention Center.

Don’t forget to activate your Cisco Live and Networkers Virtual account for access to all session materials, communities, and on-demand and live activities throughout the year. Activate your account at any internet station or visit www.ciscolivevirtual.com.

Complete Your Online Session Evaluation

http://www.ciscolivevirtual.com/


Visit the Cisco Store for Related Titles

http://theciscostores.com

http://theciscostore.com/


Thank you.