ND The research group on Networks & Distributed systems.

ND

The research group onNetworks & Distributed systems

2

ND activities

• ICON – Interconnection Networks– Interconnection networks are tightly coupled/short distance networks

with extreme demands on bandwidth, latency, and delivery

– Problem areas: Effective routing/topologies, fault-tolerance/dynamic reconfiguration, and Quality of Service

• VINNER – End-to-end Internet communications – Problem area: Network resilience – as a set of methods and

techniques that improve the user perception of network robustness and reliability.

3

ND activities

• QuA - Support of Quality of Service in component architectures– Problem area: How to develop applications that are sensitive to QoS

on a component architecture platform and how dynamic QoS management and adaptation can be supported

• Relay – Resource utilization in time-dependent distributed systems– Problem area: Reduce the effects of resource limitations and

geographical distances in interactive distributed applications – through a toolkit of kernel extensions, programmable subsystems, protocols and decision methods

Assessment of Data Path Assessment of Data Path Implementations for Download and Implementations for Download and

StreamingStreaming

Pål Halvorsen1,2, Tom Anders Dalseng1 and Carsten Griwodz1,2

1Department of Informatics, University of Oslo, Norway

2Simula Research Laboratory, Norway

5

Overview

• Motivation

• Existing mechanisms in Linux

• Possible enhancements

• Summary and Conclusions

6

Delivery Systems

Network

bus(es)

7

file systemcommunication

system

application

user space

kernel space

bus(es)

Delivery Systems

8

Pentium 4Processor

registers

cache(s)

I/Ocontroller

hub

memorycontroller

hub

RDRAM

RDRAM

RDRAM

RDRAM

PCI slots

PCI slots

PCI slots

network card

disk

file system

communication system

application

file systemcommunication

system

application

disk network card

Intel Hub Architecture several in-memory data movements and context switches

9

Motivation

• Data copy operations are expensive

– consume CPU, memory, hub, bus and interface resources (proportional to data size)

– profiling shows that ~40% of CPU time is consumed by copying data between user and kernel space

– gap between memory and CPU speeds increase

– different access times to different banks

• System calls make a lot of switches between user and kernel space

10

file systemcommunication

system

application

user space

kernel space

bus(es)

data_pointer data_pointer

Zero–CopyData Paths

11

Motivation

• Data copy operations are expensive

– consume CPU, memory, hub, bus and interface resources (proportional to data size)

– profiling shows that ~40% of CPU time is consumed by copying data between user and kernel

– gap between memory and CPU speeds increase

– different access times to different banks

• System calls make a lot of switches between user and kernel space

• A lot of research has been performed in this area

• BUT, what is the status today of commodity operating systems?

Existing Linux Existing Linux Data PathsData Paths

13

file systemcommunication

system

application

user space

kernel space

bus(es)

ContentDownload

14

Content Download: read / sendapplication

kernel

page cache socket buffer

applicationbuffer

read send

copycopy

DMA transfer DMA transfer

2n copy operations 2n system calls

15

Content Download: mmap / sendapplication

kernel

page cache socket buffer

mmap send

copy

DMA transfer DMA transfer

n copy operations 1 + n system calls

16

Content Download: sendfileapplication

kernel

page cache socket buffer

sendfile

gather DMA transfer

append descriptor

DMA transfer

0 copy operations 1 system calls

17

Content Download: Results

UDP TCP

• Tested transfer of 1 GB file on Linux 2.6

• Both UDP (with enhancements) and TCP

18

file systemcommunication

system

application

user space

kernel space

bus(es)

Streaming

19

Streaming: read / sendapplication

kernel

page cache socket buffer

application buffer

read send

copycopy

DMA transfer DMA transfer

2n copy operations 2n system calls

20

Streaming: read / writevapplication

kernel

page cache socket buffer

application buffer

read writev

copycopy

DMA transfer DMA transfer

3n copy operations 2n system calls

copy

One copy more than previous solution

21

Streaming: mmap / sendapplication

kernel

page cache socket buffer

application buffer

mmap uncork

copy

DMA transfer DMA transfer

2n copy operations 1 + 4n system calls

copy

sendsendcork

22

Streaming: mmap / writevapplication

kernel

page cache socket buffer

application buffer

mmap writev

copy

DMA transfer DMA transfer

2n copy operations 1 + n system calls

copy

Three calls less than previous solution

23

Streaming: sendfileapplication

kernel

page cache socket buffer

application buffer

DMA transfer

n copy operations 4n system calls

gather DMA transfer

append descriptor

copy

uncorksendfilesendcork

24

Streaming: Results• Tested streaming of 1 GB file on Linux 2.6

• RTP over UDP

TCP sendfile (content download)

Compared to not sending an RTP header over UDP, we get an increase of 29%(additional send call)

More copy operations and system calls required potential for improvements

Enhanced Streaming Enhanced Streaming Data PathsData Paths

26

Enhanced Streaming: mmap / msendapplication

kernel

page cache socket buffer

application buffer

DMA transfer

n copy operations 1 + 4n system calls

gather DMA transfer

append descriptor

copy

msend allows to send data from anmmap’ed file without copy

mmap uncorksendsendcork msend

copy

DMA transfer

One copy less than previous solution

27

Enhanced Streaming: mmap / rtpmsendapplication

kernel

page cache socket buffer

application buffer

DMA transfer

n copy operations 1 + n system calls

gather DMA transfer

append descriptor

copy

mmap uncorksendsendcork rtpmsend

RTP header copy integrated intomsend system call

Three calls less than previous solution

28

Enhanced Streaming: mmap/krtpmsendapplication

kernel

page cache socket buffer

application buffer

DMA transfer

0 copy operations 1 system call

gather DMA transfer

append descriptor

copy

krtpmsend

One call less than previous solution

An RTP engine in the kernel adds RTP headers

rtpmsend

RTP engine

One copy less than previous solution

29

Enhanced Streaming: rtpsendfileapplication

kernel

page cache socket buffer

application buffer

DMA transfer

n copy operations n system calls

gather DMA transfer

append descriptor

copy

rtpsendfile

existing solution requires three more calls per packet

uncorksendfilesendcork

RTP header copy integrated intosendfile system call

30

Enhanced Streaming: krtpsendfileapplication

kernel

page cache socket buffer

application buffer

DMA transfer

0 copy operations 1 system call

gather DMA transfer

append descriptor

copy

krtpsendfile

One call less than previous solution

An RTP engine in the kerneladds RTP headers

rtpsendfile

RTP engine

One copy less than previous solution

31

Enhanced Streaming: Results• Tested streaming of 1 GB file on Linux 2.6

• RTP over UDP

TCP

send

file

(con

tent

dow

nlo

ad)Ex

isting

mec

hani

sm

(str

eam

ing)

mmap based mechanisms sendfile based mechanisms

~27%

impr

ovem

ent

~25%

impr

ovem

ent

32

Conclusions

• Current commodity operating systems still pay a high price for streaming services

• However, small changes in the system call layer might be sufficient to remove most of the overhead

• Conclusively, commodity operating systems still have potential for improvement with respect to streaming support

• What can we hope to be supported?

• Road ahead: optimize the code, make patch and submit to kernel.org

33

Questions??