Moving VoIP beyond the phone Henning Schulzrinne Dept. of Computer Science Columbia University September 2005.

Moving VoIP beyond the Moving VoIP beyond the phonephone

Henning SchulzrinneDept. of Computer Science

Columbia UniversitySeptember 2005

September 2005 2

OverviewOverview The big transitions in VoIP Voice, media meta data

rich presence, caller preferences, events, … Programmable VoIP services

servers and end systems Spam and spit Emergency calling (“9-1-1”)

beyond PSTN replacement Maintaining reliable systems

administratively distributed systems

September 2005 3

Philosophy transitionPhilosophy transition

One computer/phone,many users

One computer/phone,one user

Many computers/phones,one user

anywhere,any time

any media

right place (device),right time,right media

~ ubiquitous computing embedded VoIP

mainframe erahome phone

party line

PC eracell phone era

September 2005 4

Collaboration in transitionCollaboration in transition

intra-organization;

small number of systems

(meeting rooms)

inter-organization

multiple technology generationsdiverse end

points

proprietary (single-vendor)

systems

standards-based solutions

September 2005 5

Evolution of VoIPEvolution of VoIP

“amazing – thephone rings”

“does it docall transfer?”

“how can I make itstop ringing?”

1996-2000 2000-2003 2004-

catching upwith the digital PBX

long-distance calling,ca. 1930 going beyond

the black phone

September 2005 6

Internet services – the Internet services – the missing entrymissing entry

Service/delivery

synchronous asynchronous

push instant messagingpresenceevent notificationsession setupmedia-on-demand

messaging

pull data retrievalfile downloadremote procedure call

peer-to-peer file sharing

September 2005 7

Filling in the protocol gapFilling in the protocol gap

Service/delivery

synchronous asynchronous

push SIPRTSP, RTP

SMTP

pull HTTPftpSunRPC, Corba, SOAP

(not yet standardized)

September 2005 8

An eco system, not just a An eco system, not just a protocolprotocol

SIP

XCAP(config)

RTSP

SIMPLEpolicyRPID

….

SDP

XCON(conferencing)

STUNTURN

RTP

configures

initiates carries

carriescontrols provide addresses

September 2005 9

SIP – a bi-cultural protocolSIP – a bi-cultural protocol

• overlap dialing• DTMF carriage• key systems• notion of lines• per-minute billing• early media• ISUP & BICC interoperation• trusted service providers

• multimedia• IM and presence• location-based service• user-created services• decentralized operation• everyone equally suspect

September 2005 10

SIP is PBX/Centrex readySIP is PBX/Centrex readycall waiting/multiple calls

RFC 3261

hold RFC 3264

transfer RFC 3515/Replaces

conference RFC 3261/callee caps

message waiting message summary package

call forward RFC 3261

call park RFC 3515/Replaces

call pickup Replaces

do not disturb RFC 3261

call coverage RFC 3261

from Rohan Mahy’s VON Fall 2003 talk

simultaneous ringing

RFC 3261

basic shared lines

dialog/reg. package

barge-in Join

“Take” Replaces

Shared-line “privacy”

dialog package

divert to admin RFC 3261

intercom URI convention

auto attendant RFC 3261/2833

attendant console

dialog package

night service RFC 3261

centr

ex-s

tyle

featu

res

boss/admin features

attendant features

September 2005 11

SIP design objectivesSIP design objectives new features and services

support features not available in PSTN e.g., presence and IM, session mobility

not a PSTN replacement not just SS7-over-IP even similar services use different models (e.g., call

transfer) client heterogeneity

clients can be smart or dumb (terminal adapter) mobile or stationary hardware or software

client multiplicity one user – multiple clients – one address

multimedia nothing in SIP assumes a particular media type

Rosenberg/Schulzrinne: draft-rosenberg-sipping-sip-arch-00

September 2005 12

Interconnection Interconnection approachesapproaches

Property NGN, 3GPP “Internet”

interconnection per service service neutral

end device control carrier-controlled user-provided

end device type mostly hardware software, maybe hardware

state preference call state-full statelesstransaction-stateful

interconnect arrangement

pre-arranged serendipitous

interconnect discovery

pre-configured DNS

billing preference per serviceusage-based

bandwidth-basedservices fixed-rate or ad-supported

billing arrangement clearing house sender keepsindependent

September 2005 13

The role of presenceThe role of presence

Guess, ring and annoy high probability of failure:

“telephone tag” inappropriate time (call

during meeting) inappropriate media (audio

in public place) current solutions:

voice mail tedious, doesn’t scale, hard to search and catalogue, no indication of when call might be returned

automated call back rarely used, too inflexible

most successful calls are now scheduled by email

Presence-based facilitates unscheduled

communications provide recipient-specific

information only contact in real-time if

destination is willing and able

appropriately use synchronous vs. asynchronous communication

guide media use (text vs. audio)

predict availability in the near future (timed presence)

Prediction: almost all (professional) communication will be presence-initiated or

pre-scheduled

September 2005 14

Context-aware Context-aware communicationcommunication

context = “the interrelated conditions in which something exists or occurs”

anything known about the participants in the (potential) communication relationship

both at caller and callee

time CPL

capabilities caller preferences

location location-based call routinglocation events

activity/availability presence

sensor data (mood, bio)

privacy issues similar to location data

September 2005 15

Basic presenceBasic presence Role of presence

initially: “can I send an instant message and expect a response?”

now: “should I use voice or IM? is my call going to interrupt a meeting? is the callee awake?”

Yahoo, MSN, Google, Skype presence services: on-line & off-line

useful in modem days – but many people are (technically) on-line 24x7

thus, need to provide more context + simple status (“not at my desk”)

entered manually rarely correct if user has time to update presence, they are not busy

enough to use presence does not provide enough context for directing interactive

communications

September 2005 16

Presence data modelPresence data model

“calendar” “cell” “manual”

[email protected], video, text

[email protected]

person(presentity)

(views)

services

devices

September 2005 17

Presence data architecturePresence data architecture

rawpresencedocument

createview

(compose)

privacyfiltering

draft-ietf-simple-presence-data-model

compositionpolicy

privacypolicy

presence sources

XCAP XCAP

(not defined yet)

depends on watcherselect best sourceresolve contradictions

PUBLISH

September 2005 18

Presence data architecturePresence data architecture

candidatepresencedocument

watcherfilter

rawpresencedocument

post-processingcomposition(merging)

finalpresencedocument

differenceto previous notification

SUBSCRIBE

NOTIFY

remove data not of interest

watcher

September 2005 19

Rich presenceRich presence More information automatically derived from

sensors: physical presence, movement electronic activity: calendars

Rich information: multiple contacts per presentity

device (cell, PDA, phone, …) service (“audio”)

activities, current and planned surroundings (noise, privacy, vehicle, …) contact information composing (typing, recording audio/video IM, …)

September 2005 20

RPID: rich presenceRPID: rich presence<person>

<tuple>

<device>

<activities>

<class>

<mood>

<place-is>

<place-type>

<privacy>

<relationship>

<service-class>

<sphere>

<status-icon>

<time-offset>

<user-input>

September 2005 21

The role of presence for call The role of presence for call routingrouting Two modes:

watcher uses presence information to select suitable contacts

advisory – caller may not adhere to suggestions and still call when you’re in a meeting

user call routing policy informed by presence

likely less flexible – machine intelligence

“if activities indicate meeting, route to tuple indicating assistant”

“try most-recently-active contact first” (seq. forking)

LESS

translateRPID

CPL

PA

PUBLISH

NOTIFY

INVITE

September 2005 22

Presence and privacyPresence and privacy

All presence data, particularly location, is highly sensitive

Basic location object (PIDF-LO) describes

distribution (binary) retention duration

Policy rules for more detailed access control

who can subscribe to my presence

who can see what when

<tuple id="sg89ae">

<status>

<gp:geopriv>

<gp:location-info>

<gml:location>

<gml:Point gml:id="point1“

srsName="epsg:4326">

<gml:coordinates>37:46:30N 122:25:10W

</gml:coordinates>

</gml:Point>

</gml:location>

</gp:location-info>

<gp:usage-rules>

<gp:retransmission-allowed>no

</gp:retransmission-allowed>

<gp:retention-expiry>2003-06-23T04:57:29Z

</gp:retention-expiry>

</gp:usage-rules>

</gp:geopriv>

</status>

<timestamp>2003-06-22T20:57:29Z</timestamp>

</tuple>

September 2005 23

Location-based servicesLocation-based services Finding services based on location

physical services (stores, restaurants, ATMs, …) electronic services (media I/O, printer, display,

…) not covered here

Using location to improve (network) services communication

incoming communications changes based on where I am configuration

devices in room adapt to their current users awareness

others are (selectively) made aware of my location security

proximity grants temporary access to local resources

September 2005 24

Location-based SIP Location-based SIP servicesservices Location-aware inbound routing

do not forward call if time at callee location is [11 pm, 8 am]

only forward time-for-lunch if destination is on campus do not ring phone if I’m in a theater

outbound call routing contact nearest emergency call center send [email protected] to nearest branch

location-based events subscribe to locations, not people Alice has entered the meeting room subscriber may be device in room our lab stereo

changes CDs for each person that enters the room

September 2005 25

Program location-based Program location-based servicesservices

September 2005 26

Service creationService creation

programmer, carrier

end user

network servers

SIP servlets, sip-cgi

CPL

end system VoiceXMLscripting, RPC

VoiceXML (voice),LESS

Tailor a shared infrastructure to individual users traditionally, only by vendors (and sometimes carriers) learn from web models: killer app vertical apps

September 2005 27

A different view of A different view of presencepresence Presence is problematic for facilitating calls

rapid changes + large watcher lists high notification volume

privacy: don’t want to tell whole life story to all watchers

on-demand presence information just before communication = SUBSCRIBE with zero duration

aggregate presence synchronized calendars either centralized or peer-to-peer

Likely uses shifting to true events environment changes (traffic, weather, …) people changes (visitor stuck in traffic)

September 2005 28

Programming VoIP clientsProgramming VoIP clients Precursor: CTI

but rarely used outside call centers Call external programs

e.g., Google/MSN maps, local search Scripting APIs

e.g., call Tcl or PHP scripts sip-cgi Controllable

COM, XML RPC used for media agents in sipc

Embeddable no UI, just signaling and media

September 2005 29

Automating media Automating media interaction – service interaction – service examplesexamples If call from my boss, turn off the stereo call

handling with device control As soon as Tom is online, call him call

handling with presence information Vibrate instead of ring when I am in movie

theatre call handling with location information

At 9:00AM on 09/01/2005, find the multicast session titled “ABC keynote” and invite all the group members to watch call handling with session information

When incoming call is rejected, send email to the callee call handling with email

September 2005 30

LESS: simplicityLESS: simplicity Generality (few and simple concepts) Uniformity (few and simple rules)

Trigger rule Switch rule Action rule Modifier rule

Familiarity (easy for user to understand)

Analyzability (simple to analyze)

switchestrigger actions

modifiers

September 2005 31

LESS: Decision treeLESS: Decision tree

No loopsLimited variablesNot necessarily Turing-complete

September 2005 32

LESS: SafetyLESS: Safety Type safety

Strong typing in XML schema Static type checking

Control flow safety No loop and recursion One trigger appear only once, no feature interaction for a

defined script Memory access

No direct memory access LESS engine safety

Ensure safe resource usage Easy safety checking

Any valid LESS scripts can be converted into graphical representation of decision trees.

September 2005 33

LESS snapshotLESS snapshot<less> <incoming> <address-switch> <address is=“sip:[email protected]"> <device:turnoff device=“sip:[email protected]”/> <media media=“audio”> <accept/> </media> </address> </address-switch> </incoming></less>

incoming call

If the call from my boss

Turn off the stereo

Accept the call with only audio

trigger, switch, modifier, action

September 2005 34

Device agent

x10 vcr

SIP user agent

SIP

LESS packagesLESS packages

Basic user agent

Generic Media UI

conference

email web

calendar

im

Presence agent

presence

Event

Use packages to group elements

locationsession

September 2005 35

When Tom is online, …When Tom is online, …<less><EVENT:notification> <address-switch> <address is="sip:[email protected]"> <EVENT:event-switch> <EVENT:event is="open"> <location url="sip:[email protected]"> <IM:im message="Hi, Tom"/> </location> </EVENT:event> </EVENT:event-switch> ………</less>

September 2005 36

When I am in a movie When I am in a movie theatre, …theatre, …

<less><incoming> <location-switch> <location placetype=“quiet”> <alert sound=“none”

vibrate=“yes”/> </location> </location-switch></incoming></less>

September 2005 38

Interfacing with GoogleInterfacing with Google911: caller locationIM/presence: location of friendscall: “I’m here”

September 2005 39

Interfacing with GoogleInterfacing with Googleshow all files from caller Xiaotao Wu

September 2005 40

Embedding VoIP: FAA Embedding VoIP: FAA trainingtraining

controls pilot and ATC

agents – using multicast and

unicast (“landlines”)

September 2005 41

Open issues: application Open issues: application sharingsharing Current: T.120

doesn’t integrate well with other conference control mechanisms

hard to make work across platforms (fonts) ill-defined security mechanisms

Current: web-based sharing hard to integrate with other media, control and record generally only works for Windows mostly limited to shared PowerPoint

Current: vnc whole-screen sharing only can be coerced into conferencing, but doesn’t

integrate well with control protocols

September 2005 42

IETF effort: standardized IETF effort: standardized application sharingapplication sharing Remote access = application sharing Four components:

window drawing ops PNG keyboard input mouse input window operations (raise, lower, move)

Uses RTP as transport synchronization with continuous media but typically, TCP allow multicast large group sessions

September 2005 43

SIP unsolicited calls and SIP unsolicited calls and messagesmessages Possibly at least as

large a problem more annoying (ring,

pop-up) Bayesian content

filtering unlikely to work

identity-based filtering

PKI for every user unrealistic

Use two-stage authentication

SIP identity work

home.comDigest

mutualPK authentication (TLS)

September 2005 44

Domain ClassificationDomain Classification Classification of domains based on their identity instantiation and

maintenance procedures plus other domain policies. Admission controlled domains

Strict identity instantiation with long term relationships Example: Employees, students, bank customers

Bonded domains Membership possible only through posting of bonds tied to a expected

behavior Membership domains

No personal verification of new members but verifiable identification required such as a valid credit card and/or payment

Example: E-bay, phone and data carriers Open domains

No limit or background check on identity creation and usage Example: Hotmail

Open, rate limited domains Open but limits the number of messages per time unit and prevents account

creation by bots Example: Yahoo

September 2005 45

Reputation service (“Trust Reputation service (“Trust paths”)paths”)

Alice Bob

CarolDavid

Emily Frank

has sentemail to

has sentIM to

is this a spammer?

for people and domains

September 2005 46

Emergency callingEmergency calling FCC mandate: all interconnected VoIP

providers must provide E911 service by 11/05 switch calls to right PSAP (via ESGW attached to

switches) deliver location information to PSAP for dispatch location entered manually

Problems: user-entered location no nomadic, mobile users PSAP infrastructure brittle (38 PSAPs out after

Katrina) fully IP-based allow relocation

September 2005 47

What makes VoIP 112/911 What makes VoIP 112/911 hard?hard?

POTS PSTN-emulation VoIP end-to-end VoIP

(landline) phone number limited to limited area

landline phone number anywhere in US (cf. German 180)

no phone number or phone number anywhere around the world

regional carrier national or continent-wide carrier

enterprise “carrier” or anybody with a peer-to-peer device

voice provider = line provider (~ business relationship)

voice provider ≠ ISP voice provider ≠ ISP

national protocols and call routing

probably North America + EU

international protocols and routing

location = line location mostly residential or small business

stationary, nomadic, wireless

September 2005 48

Location, location, locationLocation, location, location Location locate right PSAP & speed

dispatch In the PSTN, local 9-1-1 calls remain

geographically local In VoIP, no such locality for VSPs

most VSPs have close to national coverage Thus, unlike landline and wireless, need

location information from the very beginning

Unlike PSTN, voice service provider doesn’t have wire database information VSP needs assistance from access provider

(DSL, cable, WiMax, 802.11, …)

September 2005 49

Options for location Options for location deliverydelivery L2: LLDP-MED (standardized version of

CDP + location data) periodic per-port broadcast of configuration

information L3: DHCP for

geospatial (RFC 3825) civic (draft-ietf-geopriv-dhcp-civil)

L7: proposals for retrievals by IP address by MAC address by identifier (conveyed by LLDP, DHCP or

PPP) via HTTP or maybe SIP

September 2005 50

ArchitectureArchitecture

DHCP

DHCP

home

ISP

SIPconfig

outboundproxy

VSP

PSAPs

September 2005 51

LUMP mapping LUMP mapping architecturearchitecture

floodingnj.usny.us

bergen.nj.us

leonia.bergen.nj.us

R R

R R

R

R

knows alltrees;

caches results

carrier X customers

generalizes to otherlocation-based services

September 2005 52

Emergency call conferencingEmergency call conferencing

INVITE

3rd partycall control

INVITE

INVITE

REFER

REFER

REFER

Conferenceserver

PSAP

Recorder

Firedepartment

HospitalPSAP brings all related parties into a conference call

INVITE

media info

INVITEmedia info

Caller

INVITE

Managing (VoIP) Managing (VoIP) Applications – DYSWISApplications – DYSWIS

Henning SchulzrinneDept. of Computer Science

Columbia UniversityJuly 2005

September 2005 54

OverviewOverview User experience for VoIP still inferior Existing network management doesn’t

work for VoIP and other modern applications

Need user-centric rather than operator-centric management

Proposal: peer-to-peer management “Do You See What I See?”

Also use for reliability estimation and statistical fault characterization

September 2005 55

VoIP user experienceVoIP user experience Only 95-99.5% call attempt success

“Keynote was able to complete VoIP calls 96.9% of the time, compared with 99.9% for calls made over the public network. Voice quality for VoIP calls on average was rated at 3.5 out of 5, compared with 3.9 for public-network calls and 3.6 for cellular phone calls. And the amount of delay the audio signals experienced was 295 milliseconds for VoIP calls, compared with 139 milliseconds for public-network calls.” (InformationWeek, July 11, 2005)

Mid-call disruptions Lots of knobs to turn

Separate problem: manual configuration

September 2005 56

Traditional network Traditional network management modelmanagement model

SNMP

X

“management from the center”

September 2005 57

AssumptionsAssumptions Single provider (enterprise, carrier)

has access to most path elements professionally managed

Typically, hard failures or aggregate problems element failures substantial packet loss

Mostly L2 and L3 elements switches, routers rarely 802.11 APs

Indirect detection MIB variable vs. actual protocol performance

September 2005 58

Managing the protocol Managing the protocol stackstack

RTP

UDP/TCP

IP

SIP

no routepacket loss

TCP neg. failureNAT time-outfirewall policy

protocol problem

playout errors

mediaecho

gain problemsVAD action

protocol problem

authorizationasymmetric conn (NAT)

September 2005 59

Call lifecycle viewCall lifecycle viewge

t ad

dres

ses

SIP

INVIT

E

get 20

0 O

K

REG

ISTE

R

exch

ange

med

iate

rmin

ate

call

STUN failur

e

auth?registra

r?

outbound

proxy?dest.

proxy?

loss?gain?

silence suppressio

n?

September 2005 60

Types of failuresTypes of failures Hard failures

connection attempt fails no media connection NAT time-out

Soft failures (degradation) packet loss (bursts)

access network? backbone? remote access? delay (bursts)

OS? access networks? acoustic problems (microphone gain, echo)

September 2005 61

Diagnostic undecidabilityDiagnostic undecidability symptom: “cannot reach server” more precise: send packet, but no

response causes:

NAT problem (return packet dropped)? firewall problem? path to server broken? outdated server information (moved)? server dead?

5 causes very different remedies no good way for non-technical user to tell

Whom do you call?

September 2005 62

Additional problemsAdditional problems

ping and traceroute no longer works reliably WinXP SP 2 turns off ICMP some networks filter all ICMP

messages Early NAT binding time-out

initial packet exchange succeeds, but then TCP binding is removed (“web-only Internet”)

September 2005 63

““Do You See What I See?”Do You See What I See?” Each node has a set of active

measurement tools Nodes can ask others for their view

possibly also dedicated “weather stations” Iterative process, leading to:

user indication of cause of failure in some cases, work-around (application-

layer routing) TURN server, use remote DNS servers

Nodes collect statistical information on failures and their likely causes

September 2005 64

Failure detection toolsFailure detection tools

STUN server what is your IP address?

ping and traceroute Transport-level

liveness open TCP connection to

port send UDP ping to port

media

RTP

UDP/TCP

IP

September 2005 65

Failure statisticsFailure statistics Which parts of the network are most

likely to fail (or degrade) access network network interconnects backbone network infrastructure servers (DHCP, DNS) application servers (SIP, RTSP, HTTP, …) protocol failures/incompatibility

Currently, mostly guesses End nodes can gather and accumulate

statistics

September 2005 66

How to find management How to find management peers?peers?

Use carrier-provided bootstrap list Previous session partners

e.g., address book Watcher list

September 2005 67

What’s missing?What’s missing? Request diagnostic

“send this message”; return result do specific high-level operation (ping,

traceroute, DNS resolution) Failure statistics protocol and data

exchange format Algorithm specification for steps

“if no response to REGISTER, check server liveness”

“if bad voice QoS, ask subnet neighbor; then ask somebody close to destination”

September 2005 68

Security issuesSecurity issues

Indirect denial-of-service attacks limit per-requestor rate return cached results to querier

Lying Non-participation (“leechers”)

usual P2P mechanisms such as blacklists

September 2005 69

ConclusionConclusion Slow transition from emulating PSTN to new

services presence-based embedded (e.g., games)

Emphasis moving from protocol mechanics to architecture

slow transition to open systems different combinations of software vendors, IAP/ISP, VSP,

hardware vendors Still need to fill out infrastructure for collaboration

and presence Protocols systems infrastructure replacement Management from the center management

from the edges