Server Resources 6/9 - 2004 INF5070 – Media Storage and Distribution Systems:

Server ResourcesServer Resources

6/9 - 2004

INF5070 – Media Storage and Distribution Systems:

2004 Carsten Griwodz & Pål Halvorsen

INF5070 – media storage and distribution systems

Overview

Resources, real-time, …

“Continuous” media streams

(CPU) Scheduling

Memory management for streaming

Resources and Real–Time



Resources Resource:

“A resource is a system entity required by a task for manipulating data” [Steimetz & Narhstedt 95]

Characteristics: active: provides a service, e.g., CPU, disk or network adapter passive: system capabilities required by active resources, e.g.,

memory or bandwidth

exclusive: only one process at a time can use it, e.g., CPU shared: can be used by several concurrent processed, e.g.,

memory

single: exists only once in the system, e.g., loudspeaker multiple: several within a system, e.g., CPUs in a multi-

processor system



Real–Time Real-time process:

“A process which delivers the results of the processing in a given time-span”

Real-time system:“A system in which the correctness of a computation depends not only on obtaining the result, but also upon providing the result on time”

Many real-time applications, e.g.: temperature control in a nuclear/chemical plant

driven by interrupts from an external device these interrupts occur irregularly

defense system on a navy boat driven by interrupts from an external device these interrupts occur irregularly

control of a flight simulator execution at periodic intervals scheduled by timer-services which the application requests from the OS

...



Real–Time Deadline:

“A deadline represents the latest acceptable time for the presentation of the processing result”

Hard deadlines: must never be violated system failure too late results

have no value, e.g., processing weather forecasts

means severe (catastrophic) system failure, e.g., processing of an incoming torpedo signal in a navy boat scenario

Soft deadlines: in some cases, the deadline might be missed

not too frequently not by much time

result still may have some (but decreasing) value, e.g., a late I-frame in MPEG



Real–Time and Multimedia Multimedia systems

typically have soft deadlines (may miss a frame) are non-critical (user may be annoyed, but …) have periodic processing requirements

(e.g., each 33 ms in a 30 fps video) require large bandwidths

(e.g., average of 3.5 Mbps for DVD video only)

need predictability (guarantees) adapt real-time mechanisms to continuous media exploit resource-specific properties

(like real-time resource allocation schemes, preemption, ...)

priority-based schemes are of special importance



Admission and Reservation To prevent overload, admission may be performed:

schedulability test: “are there enough resources available for a new stream?” “can we find a schedule for the new task without disturbing the existing workload?” a task is allowed if the utilization remains < 1

yes – allow new task, allocate/reserve resources no – reject

Resource reservation is analogous to booking(asking for resources) pessimistic

avoid resource conflicts making worst-case reservations potentially under-utilized resources guaranteed QoS

optimistic reserve according to average load high utilization overload may occur

perfect must have detailed knowledge about resource requirements of all processes too expensive to make/takes much time



Real–Time and Operating Systems The operating system manages local resources

(CPU, memory, disk, network card, busses, ...) In a real-time, multimedia scenario, support is needed for:

real-time processing efficient memory management

This also means support for proper … scheduling –

high priorities for time-restrictive multimedia tasks timer support –

clock with fine granularity and event scheduling with high accuracy kernel preemption –

avoid long periods where low priority processes cannot be interrupted

memory replacement – prevent code for real-time programs from being paged out

fast switching – both interrupts and context switching should be fast

...

Continuous Media Streams



Start playback at t1

Consumed bytes (offset) variable rate constant rate

Must start retrieving data earlier

Data must arrive beforeconsumption time

Data must be sent before arrival time

Data must be read from disk before sending time

Streaming Data

t1

time

data offset

consume function

arrive function

send functionread function



Need buffers to hold data between the functions, e.g., client B(t) = A(t) – C(t), i.e., t : A(t) ≥ C(t)

Latest start of data arrival is given by min[B(t,t0,t1) ; t B(t,t0,t1) ≥ 0],

i.e., the buffer must at all times t have more data to consume

Streaming Data

time

data offset

t1

consume function

arrive function

t 0



file systemcommunication

system

application

“Continuous Media” and “Streaming” are ILLUSIONS retrieve data in blocks from disk transfer blocks from file

system to application send packets to communication system

split packets into appropriate MTUs

... (intermediate nodes) ... (client)

different optimal sizes

pseudo-parallel processes (run in time slices)

need for scheduling(to have timing and appropriate resource allocation)

Streaming Data

(CPU) Scheduling



Scheduling A task is a schedulable entity

(a process/thread executing a job, e.g., an packet through the communication system or a disk request through the file system)

In a multi-tasking system, several tasks may wish to use a resource simultaneously

A scheduler decides which task that may use the resource, i.e., determines order by which requests are serviced, using a scheduling algorithm

Each active (CPU, disk, NIC) resources needs a scheduler(passive resources are also “scheduled”, but in a slightly different way)

resource

requests

scheduler



Scheduling Scheduling algorithm classification:

dynamic make scheduling decisions at run-time flexible to adapt considers only actual task requests and execution time parameters large run-time overhead finding a schedule

static make scheduling decisions at off-line (also called pre-run-time) generates a dispatching table for run-time dispatcher at compile time needs complete knowledge of task before compiling small run-time overhead

preemptive currently executing task may be interrupted (preempted) by higher priority

processes preempted process continues later at the same state potential frequent contexts switching (almost!?) useless for disk and network cards

non-preemptive running tasks will be allowed to finish its time-slot (higher priority processes

must wait) reasonable for short tasks like sending a packet (used by disk and network

cards) less frequent switches



Scheduling Preemption:

tasks waits for processing scheduler assigns priorities task with highest priority will be

scheduled first preempt current execution if a higher

priority (more urgent) task arrives

real-time and best effort priorities(real-time processes have higher priority - if exists, they will run)

to kinds of preemption: preemption points

o predictable overheado simplified scheduler accounting

immediate preemptiono needed for hard real-time systemso needs special timers and fast interrupt

and context switch handling

resource

requests

scheduler preemption



Scheduling Scheduling is difficult and takes time

(both to find a schedule and to switch between threads/processes – not shown):

process 1 process 2 process 3 process 4 process N RT process…

RT process

request

round-robin

process 1 process 2 process 3 process 4 process N…

RT process

requestpriority,non-preemtive

delay

RT process

delay

process 1 process 2 process 3 process 4 process N…

requestpriority,preemtive p 1 p 1 process 2 process 3 process 4 process N…

RT process

RT process p 1 process 2 process 3 process 4 process N…

only delay switching and interrupts

NOTE: preemption may also be limited to preemption points (fixed points where the scheduler is allowed to interrupt a running process) giving larger delays



Priorities and Multimedia Multimedia streams need predictable access to

resources – high priorities:

Within each class one could have a second-level scheduler 1 and 2: real-time scheduling and fine grained

priorities 3: may use traditional approaches as round-robin

1. multimedia traffic with guaranteed QoS

2. multimedia traffic with predictive QoS

3. other requests

may not exist

must not starve



Scheduling in Windows 2000 Preemptive kernel Schedules threads individually

Time slices given in quantums 3 quantums = 1 clock interval

different values used for different clock frequencies, e.g., x86 uniCPU: 10 ms x86 multiCPU: 15 ms

defaults: Win2000 server: 36 quantums Win2000 workstation: 6 quantums (professional)

may manually be increased between threads (1x, 2x, 4x, 6x)

foreground quantum boost (add 0x, 1x, 2x): active window can get longer time slices (assumed needs fast response)



Scheduling in Windows 2000 32 priority levels:

Round Robin (RR) within each level

Interactive and throughput-oriented: “Real time” – 16 system levels

fixed priority may run forever

Variable – 15 user levels priority may change:

thread priority = process priority ± 2 uses much drops user interactions, I/O completions increase

Idle/zero-page thread – 1 system level runs whenever there are no other processes to

run clears memory pages for memory manager

31

30

...

17

16

15

14

...

2

1

0

Real Time (system thread)

Variable (user thread)

Idle (system thread)



Scheduling in Linux Preemptive kernel Threads and processes used to be equal,

but Linux uses (in 2.6) thread scheduling

SHED_FIFO may run forever, no timeslices may use it’s own scheduling algorithm

SHED_RR each priority in RR timeslices of 10 ms (quantums)

SHED_OTHER ordinary user processes uses “nice”-values: 1≤ priority≤40 timeslices of 10 ms (quantums)

Threads with highest goodness are selected first:

realtime (FIFO and RR):goodness = 1000 + priority

timesharing (OTHER): goodness = (quantum > 0 ? quantum + priority : 0)

Quantums are reset when no ready process has quantums left:quantum = (quantum/2) + priority

1

2

...

126

127

1

2

...

126

127

default (20)

-20

-19

...

18

19

SHED_FIFO

SHED_RR

SHED_OTHER

nice



Scheduling in AIX Similar to Linux, but has

always only used thread scheduling SHED_FIFO SHED_RR SHED_OTHER

BUT, SHED_OTHER may change “nice” values running long (whole

timeslices) penalty – nice increase

interrupted (e.g., I/O) gives initial “nice” value back

1

2

...

126

127

1

2

...

126

127

default

-20

-19

...

18

19

SHED_FIFO

SHED_RR

SHED_OTHER

nice



Multimedia streams are usually periodic (fixed frame rates and audio sample frequencies)

Time constraints for a periodic task: s – starting point

(first time the task require processing) e – processing time d – deadline p – period (r – rate (r = 1/p))

0 ≤ e ≤ d (often d ≤ p: we’ll use d = p – end of period, but Σd ≤ Σp is enough)

the kth processing of the task is ready at time s + (k – 1) p must be finished at time s + (k – 1) p + d

the scheduling algorithm must account for these properties

Real–Time Scheduling

s time

ed

p



Real–Time Scheduling Resource reservation

QoS can be guaranteed relies on knowledge of tasks no fairness origin: time sharing operating systems e.g., earliest deadline first (EDF) and rate monotonic (RM)

(AQUA, HeiTS, RT Upcalls, ...)

Proportional share resource allocation no guarantees requirements are specified by a relative share allocation in proportion to competing shares size of a share depends on system state and time origin: packet switched networks e.g., Scheduler for Multimedia And Real-Time (SMART)

(Lottery, Stride, Move-to-Rear List, ...)



Earliest Deadline First (EDF) Preemptive scheduling based on dynamic task priorities

Task with closest deadline has highest priority stream priorities vary with time

Dispatcher selects the highest priority task

Assumptions: requests for all tasks with deadlines are periodic the deadline of a task is equal to the end on its period (starting

of next) independent tasks (no precedence) run-time for each task is known and constant context switches can be ignored



Earliest Deadline First (EDF)

Example:

Task A

Task Btime

Dispatching

deadlines

priority A > priority B

priority A < priority B



Rate Monotonic (RM) Scheduling Classic algorithm for hard real-time systems with one

CPU [Liu & Layland ‘73]

Pre-emptive scheduling based on static task priorities

Optimal: no other algorithms with static task priorities can schedule tasks that cannot be scheduled by RM

Assumptions: requests for all tasks with deadlines are periodic the deadline of a task is equal to the end on its period (starting of

next) independent tasks (no precedence) run-time for each task is known and constant context switches can be ignored any non-periodic task has no deadline



Process priority based on task periods task with shortest period gets

highest static priority task with longest period gets

lowest static priority dispatcher always selects task requests with highest priority

Example:

Rate Monotonic (RM) Scheduling

pri

ori

ty

period length

shortest period, highest priority

longest period, lowest priority

Task 1

p1

Dispatching

Task 2

p2 P1 < P2

P1 highest prioritypreemption



EDF Versus RM It might be impossible to prevent deadline misses in a strict, fixed priority system:

Task A

Task B

Fixed priorities,A has priority, no dropping

Fixed priorities,B has priority, no dropping

Fixed priorities,A has priority, dropping

Fixed priorities,B has priority, dropping

time

deadline miss

deadline miss

deadline miss

deadline miss

Earliest deadline first

deadlines

waste of time

waste of time

waste of time

Rate monotonic (as the first)

deadline miss

RM may give somedeadline violationswhich is avoided by EDF



NOTE: this means that EDF is usually more efficient than RM, i.e., if switchesare free and EDF uses resources ≤ 1, then RM may need ≤ ln(2) resources to schedule the same workload

EDF Versus RM EDF

dynamic priorities changing in time overhead in priority switching QoS calculation – maximal throughput:

Ri x ei ≤ 1, R – rate, e – processing time

RM static priorities based on periods may map priority onto fixed OS priorities (like Linux) QoS calculation:

Ri x ei ≤ ln(2), R – rate, e – processing time

all streams i

all streams i



SMART (Scheduler for Multimedia And Real–Time applications)

Designed for multimedia and real-time applications

Principles

priority – high priority tasks should not suffer degradation due to presence of low priority tasks

proportional sharing – allocate resources proportionally and distribute unused resources (work conserving)

tradeoff immediate fairness – real-time and less competitive processes (short-lived, interactive, I/O-bound, ...) get instantaneous higher shares

graceful transitions – adapt smoothly to resource demand changes

notification – notify applications of resource changes

No admission control



Tasks have importance and urgency urgency – an immediate real-time constraint, short deadline

(determine when a task will get resources) importance – determine the overall resource allocation

expressed by a tuple: [ priority p , biased virtual finishing time bvft ]

static priority: supplied by user or assigned a default value virtual finishing time: degree to which the share was consumed bias: bonus for interactive tasks

Best effort schedule based on urgency and importance find most important tasks – compare tuple:

T1 > T2 (p1 > p2) (p1 = p2 bvft1 > bvft2) sort after urgency (EDF based sorting) iteratively select task from candidate set as long as schedule is

feasible

SMART (Scheduler for Multimedia And Real–Time applications)



Evaluation of a Real–Time Scheduling

Tests performed by IBM (1993) executing tasks with and without EDF on an 57 MHz, 32 MB RAM, AIX Power 1

Video playback program: one real-time process read compressed data decompress data present video frames via X server to user process requires 15 timeslots of 28 ms each per

second 42 % of the CPU time




task numberevent number

lax

ity [

s]3 Load Processes

-0.05

-0.04

-0.03

-0.02

-0.01

0

0.01

0.02

0.03

0.04

0.05

0 20 40 60 80 100 120 140 160 180 200

without real-time schedulingwith real-time scheduling

laxit

y (

rem

ain

ing t

ime t

o d

eadlin

e)

several deadlineviolations by thenon-real-timescheduler

the real-time scheduler reaches all its deadlines

3 load processes(competing with the video playback)




0.026

0.028

0.03

0.032

0.034

0.036

0.038

0.04

0.042

0 20 40 60 80 100 120 140 160 180 200task number

laxit

y (

rem

ain

ing t

ime t

o d

eadlin

e)

Varied the number of load processes(competing with the video playback)

NB! The EDF scheduler kept its deadlines

4 other processes

16 other processes

Only video process




Tests again performed by IBM (1993) on an 57 MHz, 32 MB RAM, AIX Power 1

“Stupid” end system program: 3 real-time processes only requesting CPU cycles each process requires 15 timeslots of 21 ms each per

second 31.5 % of the CPU time each 94.5 % of the CPU time required for real-time tasks




1 Load Process

event number

laxi

ty [

s]

-0.08

-0.06

-0.04

-0.02

0

0.02

0.04

0.06

0 20 40 60 80 100 120 140 160 180 200

without real-time scheduling

with real-time scheduling

0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0.045

0.05

0 20 40 60 80 100 120 140 160 180 200

with real-time scheduling – process 1with real-time scheduling – process 2with real-time scheduling – process 3

16 Load Processes

laxi

ty [

s]

event number

1 load process(competing with the real-time processes)

task number

laxit

y (

rem

ain

ing t

ime t

o d

eadlin

e)

the real-time scheduler reaches all its deadlines




16 load process(competing with the real-time processes)

task number

laxit

y (

rem

ain

ing t

ime t

o d

eadlin

e)

1 Load Process

event number

laxi

ty [

s]

-0.08

-0.06

-0.04

-0.02

0

0.02

0.04

0.06

0 20 40 60 80 100 120 140 160 180 200



0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0.045

0.05

0 20 40 60 80 100 120 140 160 180 200


16 Load Processesla

xity

[s]

event number

Regardless of other load, the EDF-scheduler reach its deadlines(laxity almost equal as in 1 load process scenario)

process 1

process 2

process 3NOTE: Processes are scheduled in same order

1 Load Process

event number

laxi

ty [

s]

-0.08

-0.06

-0.04

-0.02

0

0.02

0.04

0.06

0 20 40 60 80 100 120 140 160 180 200



0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0.045

0.05

0 20 40 60 80 100 120 140 160 180 200


16 Load Processes

laxi

ty [

s]

event number

Memory Management



Pentium 4Processor

registers

cache(s)

I/Ocontroller

hub

memorycontroller

hub

RDRAM

RDRAM

RDRAM

RDRAM

PCI slots

PCI slots

PCI slots

network card

disk

file system

communication system

application


system

application

disk network card

Copying on the Intel Hub Architecture



Streaming Modes Using Copying Traditional applications:

Streaming applications:

device driver device driver

independent abstraction layer(s)

HW device HW device

read writeapplication-specific data modificationsuser

kernel

OS

device driver device driver


HW device HW device

read writeuserkernel

OS



Cost of Data Transfers – Example I First generation router built with 133 MHz Intel Pentium

mean packet size 500 B interrupt time of 10 µs, word access 50 ns per packet processing of 200 instructions (1.504 µs)

copy loop: 4 instructions 2 memory accesses 130.08 ns (per 4 byte)

per packet: processing + copy + interrupt = 1.504 µs + [(500/4) * 130 ns] + 10 µs = 27.754 µs 144 Mbps

register memory[read_ptr]memory[write_ptr] registerread_ptr read_prt + 4write_ptr write_prt + 4counter counter – 1if (counter not 0) goto top of loop



Cost of Data Transfers – Example II Copying in NetBSDv1.5

by UniK/IFI (2000) copyin(), copyout(), and memcpy()

933 MHz P3 CPU theoretical max.:

25.6 Gbps

INTEL:larger is better

BUT: max at 2 – 8 KB decrease at larger

sizes caching effects



Cost of Data Transfers – Example II (cont.)

Assume sending 1 GB data whole operation, reading from disk and sending to network,

takes about 10 s reading 64 KB blocks from disk 137.10 µs per copyout() sending 4 KB packets 1.65 µs per copyin() in total: read + send =

(16384 * 137.10 µs) + (262144 * 1.65 µs) =2.679 s for copying only

THUS; data movement costs should be kept small careful management of contiguous media data avoid unnecessary physical copy operations apply appropriate buffer management schemes

reduce overhead by removing physical in-memory copy operation, i.e., ZERO-COPY data pathsZERO-COPY data paths




system

application

user space

kernel space

bus(es)

mbufbuf

b_data m_data

Basic Idea of Zero–Copy Data Paths



Application streaming using zero-copy: read data into kernel

buffer and send from there

application responsible for timing

send: explicit send automatic send

Kernel streaming using zero-copy: thread per stream perform read and

write operations application specifies

timing, but it is ensured by the tread

stream is only created – controlled by kernel

userkernel

OS device driver device driver


HW device HW device

read & send

Streaming Modes NOT Using Copying

read write

create streamuserkernel

OS device driver device driver


HW device HW device

thread

read write



Existing Zero–Copy Streaming Mechanisms

Linux: sendfile() between two descriptors (file and TCP-socket) bi-directional: disk-network and network-disk need TCP_CORK

AIX: send_file() only TCP uni-directional: disk-network

INSTANCE (MMBUF-based, in NetBSDv1.5): by UniK/IFI (2000) uni-directional: disk-network

(network-disk ongoing work) stream_read() and stream_send()

(zero-copy 1) stream_rdsnd()

(zero-copy 2)

splice(), stream(), IO-Lite, MMBUF, …

Kernel streaming using zero-copy

Application streaming using zero-copy



INSTANCE CPU Time Transfer 1 GB Used disk blocks of 64 KB Used UDP packets of 1–8 KB

Results in seconds:

Gain larger than expected: removed other operations as

well like buffer cache look-up(simplified the chain of functions)

some packet drop at server saved about 0.2 s

0

2

4

6

8

10

12

1 2 4 8

Traditional Zero-Copy 1

tim

e in s

eco

nds

packet size in KB

Removing copy

Measured

1 KB 2.80 3.392 KB 2.75 4.094 KB 2.68 3.988 KB 2.98 3.31



INSTANCE Zero–Copy Transfer Rate

Throughput increase of ~2.7 times per stream (can at least double the number of streams)

Zero-copy transfer rate limited by network cardand storage system

saturated a 1 Gbps NIC and 32-bit, 33 MHz PCI

reduced processing time by approximately 50 %

huge improvement in number of concurrent streams

approx. 12 Mbps

approx. 6 Mbps

read, write, with copy

read, write, no copy

read, automatic write, no copy

The End:Summary



Summary All (active) resources needs to be scheduled Scheduling algorithms for multimedia tasks have to…

… consider real-time requirements … provide good resource utilization (… be implementable)

Memory management is an important issue pinning frequently used data – or at least keep as long as

possible (replacement algorithms later) reservation of memory buffers copying is expensive

Rule of thumb: watch out for bottlenecks copying data touching operations frequent context switches scheduling of slow devices (disk) ...



Some References1. Halvorsen, P.: “Improving I/O Performance of Multimedia Servers”, Thesis for the Dr. Scient.

degree at University of Oslo, Unipub forlag, ISSN 1501-7710, No. 161, Oslo, Norway, August 2001

2. Liu, C.L., Layland, J.W.: "Scheduling Algorithms for Multi-Programming in a Hard Real-Time Environment“, Journal of the Association for Computing Machinery 20, 1 (January 1973): 40-61

3. Nieh, J., Lam, M.S.: “The Design, Implementation and Evaluation of SMART: A Scheduler for Multimedia Applications”, Proc. of 16th ACM Symp. on Operating System Principles (SOSP’97), St. Malo, France, October 1997, pp. 184-197

4. Plagemann, T., Goebel, V., Halvorsen, P., Anshus, O.: "Operating System Support for Multimedia Systems", The Computer Communications Journal, Elsevier, Vol. 23, No. 3, February 2000, pp. 267-289

5. Solomon, D.A., Russinovich, M.E.: “Inside Microsoft Windows2000”, 3rd edition, Microsoft Press, 2000

6. Steinmetz, R., Nahrstedt, C.: “Multimedia: Computing, Communications & Applications”, Prentice Hall, 1995

7. Tanenbaum, A.S.: “Modern Operating Systems” (2nd ed.), Prentice Hall, 2001 8. Wolf, L.C., Burke, W., Vogt, C.: “Evaluation of a CPU Scheduling Mechanism for Multimedia

Systems”, Software - Practice and Experience, Vol. 26, No. 4, 1996, pp. 375 - 398

Server Resources 6/9 - 2004 INF5070 – Media Storage and Distribution Systems:

Documents

realtime system

system capabilities

system entity

distribution systemsadmission

new task

system failuretoo late

realtime mechanisms

realtime applications