Overview of IETF work on IP traffic flow measurement and current developments Dr. Jürgen Quittek General Manager Network Research Division, NEC Europe.

Overview of IETF work on IP traffic flow measurement

and current developments

Dr. Jürgen Quittek

General ManagerNetwork Research Division, NEC Europe Ltd.

Heidelberg, Germany

ITU-T Workshop onIP Traffic Flow Measurement

(Geneva, Switzerland, 24 March 2011)

Geneva, 24 March 2011

… …Flows can be long lasting...

… or have a limited lifetime...

… …

… and packets may belong to more than one flow

Typical reported flow information:•start time•end time•#packets•#bytes

tPeriodically reported for long lasting flows

IP packets and flowsGroups of IP packets sharing common characteristics (e.g IP src/dst address, TOS field, protocol, transport layer ports, etc.)

2

Classification &Flow Recording

PAYLOAD HEAD PAYLOAD HEAD

PAYLOAD HEAD PAYLOAD HEAD

PacketCapturing

Filtering

Samplingpackets

Filtering

Samplingflow records

flow records

packets

packets

flow records flow records

packetreports

both steps may be trivial (1:1 sampling, no filtering)

The general (passive) IP traffic measurement process

Exportingprocess

ObservationPoint

(router, probe, etc.)

Metering process

3

………

…Meter:

Filters packets,timestamps

themand associatesPkts to flow(s)

Flow cache:Creates/Removes/Updates

flow records

• Flow Key• Flow start time• Flow last update time• # Pkts• # Bytes

•….

•….

Collector:Receives export packets, interfaces to applications

info info infoExp HDDatabase

Exporter:Reads Flow cache,

prepares and sends export packets

info info infoExp HD

Router functionalityor dedicated Probe

The flow monitoring process

4

IETF IPFIX(Netflow v9)

Flow monitoring issues

Flows have very different characteristics long-/short-lived, high/low volume, etc.

Creating/updating flow record at high speed links

packet sampling fast memory for flow cache, flow sampling

Timing out flows (TCP FIN/RST vs. timeout)Reporting

flow cache reading effort, reporting frequencyselective report

Reporting formatfixed format: Netflow 5template based: Netflow 9, IPFIX

5

IETF activities on IP traffic measurement

Three working groupsIPPM: IP Performance Metrics

defines metrics for performance measure-ments (delay, roundtrip time, loss, etc.)

IPFIX: IP Flow Information eXportdefines protocol for export of flow data

PSAMP: Packet Sampling (concluded)defines protocol for export of packet databased on IPFIX

6

IPFIX protocolIP Flow Information eXport

Established 2001Main goal: Develop common IP traffic flow reporting protocol to be available on most future routers

meeting requirements of many applicationslow hardware/software costssimple,Scalableextensible

http://datatracker.ietf.org/wg/ipfix/ 7

Distinguishing flows by5-tuple (IP addresses, protocol, port)MPLS label, TOS fieldsinterface & direction

Flexible aggregation of flowsMetering Process

timestampsflow timeouts

Further requirements for IPFIX I

8

Extensible information/data modelflow properties and statistics

many header fields

anonymization

Reliable and secure data transfercongestion awarenesspush model reporting

Configuration

Further requirements for IPFIX II

9

IPFIX architecture

Application

Flow Record

Observation Point

Flow Information Export

PAYLOAD HEAD PAYLOAD HEAD

PAYLOAD HEAD PAYLOAD HEAD

PAYLOAD HEAD PAYLOAD HEAD

PAYLOAD HEAD PAYLOAD HEAD

MeteringProcess

ExportingProcess

CollectingProcess

10

OM

E

Probe

OM

E

Simple Router

O OO OM

E

Complex Router

O OO OMO OO O

M

E

Multiple Exporters

O OO OM

E

O OO

OM

E

Protocol Converter

(Meter MIB)

OM

E

OM

E

OM

E

M E

Concen-trator

C E

Proxy

C …

IPFIX devices

11C

EM

OMetering ProcessExporting Process

Collecting Process

Observation Point

IPFIX protocol design

Based on NetFlow version 9Binary-coded flow record arraysTemplates for flow record formats

first send a templatethen send data records with the format defined by the template

Runs over SCTP, TCP, UDP

12

IPFIX information model

A flow record containsheader fields (transport, IP, sub-IP)

"flow keys" used for distinguishing flows

counters for packets, bytes, etc.time stamps further flow properties

min/max values, duration, directionnext hop IP addressBGP source AS, destination AS, next hop AS

may also be used as flow keys

All defined as "Information Elements"

13

IPFIX normative documents

RFC 5101: Specification of the IPFIX Protocol for the Exchange of IP Traffic Flow Information, 2008RFC 5102: Information Model for IPFIX, 2008RFC 5103: Bidirectional Flow Export Using IPFIX, 2008RFC 5473: Reducing Redundancy in IPFIX and PSAMP Reports, 2009RFC 5610: Exporting Type Information for IPFIX Information Elements, 2009RFC 5655: Specification of the IPFIX File Format, 2009RFC 5815: Definitions of Managed Objects for IPFIX, 2010

core protocol specification

14

IPFIX informational documents

RFC 3917: Requirements for IPFIX, 2004

RFC 3955: Evaluation of Candidate Protocols for IPFIX, 2004

RFC 5153: IPFIX Implementation Guidelines, 2008

RFC 5470: Architecture for IPFIX, 2009

RFC 5471: Guidelines for IPFIX Testing, 2009

RFC 5472: IPFIX Applicability, 2009

RFC 5982: IPFIX Mediation: Problem Statement, 2010

15

Current issues in the IPFIX WG

Configurationinterface for configuring IPFIX devicesdefined as YANG module

Mediationparticularly for large networks

driven by NTTaggregationanonymization

Flow selectionStructuring flow records

extending IPFIX capabilities

Using IPFIX for reporting other informationMIB variables, SIP server logs, etc. 16

PSAMP

Established in Summer 2002Focus on sampling and capturing packets and on transferring them to data collectorsTarget applications

traffic profilingmonitoring network behavior

Extends IPFIX export Defines packet sampling with much more detail

packet filtering and sampling information model

17

IPPM"The IPPM WG will produce documents that define specific metrics and procedures for accurately measuring and documenting these metrics:"

connectivityone-way delay and lossround-trip delay and lossdelay variationloss patternspacket reorderingbulk transport capacity (BTC = data_sent / elapsed_time)link bandwidth capacity

Refer to WG official page for list of already published RFCs and ID

http://datatracker.ietf.org/wg/ippm/ 18

Final remarksThe IETF developed IPFIX as standard protocol for reporting IP flow informationTechnology is mature

many implementationsseveral interoperability testing eventsmajor router vendors expected to release IPFIX soon as part of standard installation

IPFIX is extensibleBGP-related flow info can already be reportedadditional information elements can be added

IPFIX can be used to report measurements at peering points appropriate metering hardware required

19