Real-Time Linux (RT-Preempt) + SCHED DEADLINE (Linux ...

Real-Time Linux (RT-Preempt)

+

SCHED DEADLINE (Linux Kernel 3.14)

Josef Widder

BRDS 2015

Misconceptions about Real-Time Computing [Sta88]

I Advances in supercomputer hardware will take care ofreal-time requiremens

I Real-time computing is equivalent to fast computing

I Real-time programming is assembly coding

I Real-time is meaningless if hardware will fail or softwarecontains bugs

I Real-time systems function in static environments

2







2







2







2







2

Real-Time vs. general-purpose OS

real-timealways predictable behavior

I timing guarantees

I under worst case behavior

general-purposemost of the time good behavior

I high throughput

I under normal (avg. case) behavior

3

Real-Time vs. general-purpose OS

real-timealways predictable behavior

I timing guarantees

I under worst case behavior

general-purposemost of the time good behavior

I high throughput

I under normal (avg. case) behavior

3

Why (Vanilla) Linux is not Real-time (bird’s eye view)

I general-purpose

I high functionality

I optimized for average case

I latency my vary and may not be bounded

(even with patch, convenient Linux features should not be touched)

4

Why (Vanilla) Linux is not Real-time (bird’s eye view)

I general-purpose

I high functionality

I optimized for average case

I latency my vary and may not be bounded

(even with patch, convenient Linux features should not be touched)

4

Why Linux is not Real-time (a closer look)

I Applications run in user space

I Hardware interaction is in kernel

I kernel not preempted by user threads ⇒

I event triggers high-priority thread

I thread cannot preempt kernel⇒ large latency possible > 100ms

I “classic” time sharing scheduling

5

CONFIG PREEMPT RT patch

to improve RT behavior, “reduce special status” of kernel

I kernel code outside of

I spinlocks protected areas

I interrupt handlers

may be preempted

I WC latency drops to (single digit) ms

Unlike semaphores, spinlocks may be used in code that cannotsleep, such as interrupt handlers. When properly used, spinlocksoffer higher performance than semaphores in general.

6

How does it work? [faq07]

The patch converts Linux into a fully preemptible kernel:

I in-kernel locking-primitives (using spinlocks) preemptiblethrough reimplementation with rtmutexes.RT-mutexes extend the semantics of simple mutexes by thepriority inheritance protocol

I citical sections protected by spinlock t and rwlock t arenow preemptible.rwlock t allows multiple readers in critical section

I priority inheritance for in-kernel spinlocks and semaphores.

I converting interrupt handlers into preemptible kernel threads

I user space POSIX timers with high resolution.

7








7








7








7

What sort of real-time? [faq07]

it depends. . .

I speed of the CPU and the architecture

I device drivers

I hardware

e.g., DMA activity can introduce significant latencies

e.g., some firmwares can stop the system for housekeeping(SMI, Service Management Interrupts)SMIs can not be trapped by the OS

. . . and it is still Linux ⇒ certain features must not be used. . .

8

What sort of real-time? [faq07]

it depends. . .

I speed of the CPU and the architecture

I device drivers

I hardware

e.g., DMA activity can introduce significant latencies

e.g., some firmwares can stop the system for housekeeping(SMI, Service Management Interrupts)SMIs can not be trapped by the OS

. . . and it is still Linux ⇒ certain features must not be used. . .

8

Guidelines for writing RT applications [faq07]

I Call mlockall() as soon as possible from main().causes all of the pages mapped by the address space of aprocess to be memory residenti.e., preventing that memory is paged to the swap area

I Create all threads at startup time of the applicationi.e., never start threads dynamically during mission time

I Touch each page of the entire stack of each thread

I Never use system calls that are known to generate page faults,such as fopen().opening files does mmap() system call ⇒ page fault

9






9






9

How to not build RT applications [how07]

Don’t use fancy stuff!

Direct memory access

I latency in microseconds by

I SATA/PATA/SCSI devices

I network adapters

I video cards

I can sometimes be controlled by device driversthroughput vs. latency

I this problem independent of OS

Power managementWatts vs. latency

Hyperthreading and out-of-order executionperformance vs. latency

10






I network adapters

I video cards





10






I network adapters

I video cards





10

More Guidelines. . . [how07]

I do not use VGA text consolevery large latency

I use serial console, ssh, telnet

I take care of page faults (swapping) during start-up

I Call directly from the main() entry the mlockall() call

I Create all threads at startup time of the application.

I Reserve a pool of memory to do new/delete or malloc/free

I Never use system calls that are known to generate pagefaults.

I do not configure application with priority 99

some watchdog threads need higher priority than appl.

I use mmap to pass data around

11












11












11

Install [rth06]

it is a kernel patch. . . thus something like this:

getting the sources:

# wget ftp://ftp.kernel.org/pub/linux/kernel/v2.6/linux-2.6.23.1.tar.bz2

# wget http://www.kernel.org/pub/linux/kernel/projects/rt/patch-2.6.23.1-rt11.bz2

Patching the Sources

# tar xfj linux-2.6.23.1.tar.bz2

# cd linux-2.6.23.1

# bzcat ../patch-2.6.23.1-rt11.bz2 | patch -p1

configuring and build. . .

let’s see “hello, world.” and kernel detection [rth06]

12

Install [rth06]

it is a kernel patch. . . thus something like this:

getting the sources:

# wget ftp://ftp.kernel.org/pub/linux/kernel/v2.6/linux-2.6.23.1.tar.bz2

# wget http://www.kernel.org/pub/linux/kernel/projects/rt/patch-2.6.23.1-rt11.bz2

Patching the Sources

# tar xfj linux-2.6.23.1.tar.bz2

# cd linux-2.6.23.1

# bzcat ../patch-2.6.23.1-rt11.bz2 | patch -p1

configuring and build. . .

let’s see “hello, world.” and kernel detection [rth06]

12

SCHED DEADLINE

“Classic” Linux Schedulers

Normal scheduling policies

I SCHED OTHER

standard round-robin time-sharing policy

I SCHED BATCH

for ”batch” style execution of processes“middle priority”

I SCHED IDLE

for running very low priority background jobs

Linux real-time schedulers with static priorities:

I SCHED FIFO: first in first outno time slicing

I SCHED RR: round robintime slices with preemption

14

“Classic” Linux Schedulers

Normal scheduling policies

I SCHED OTHER

standard round-robin time-sharing policy

I SCHED BATCH

for ”batch” style execution of processes“middle priority”

I SCHED IDLE

for running very low priority background jobs

Linux real-time schedulers with static priorities:

I SCHED FIFO: first in first outno time slicing

I SCHED RR: round robintime slices with preemption

14

SCHED DEADLINE

I real-time scheduling class in Linux

I earliest deadline first (EDF)

I dynamic priorities

I shortest deadline = highest priority

I optimal (restrictions apply)

I taskset is schedulable iff U ≤ 1

I released in kernel 3.14 on March 30, 2014.

15

There are some issues with EDF

I critical sectionpriority inheritance (deadline inheritance still a to-do

I precendence constraintsshifting deadlines w.r.t. dependent tasks

I overloadsadmission control

I context switch for free“in theory there is no differencebetween theory and practice. . . ”

. . . see RT Scheduling lecture

let’s see Linux implementation of EDF. . .

16

There are some issues with EDF

I critical sectionpriority inheritance (deadline inheritance still a to-do

I precendence constraintsshifting deadlines w.r.t. dependent tasks

I overloadsadmission control

I context switch for free“in theory there is no differencebetween theory and practice. . . ”

. . . see RT Scheduling lecture

let’s see Linux implementation of EDF. . .

16

sched-deadline.txt [EDF]

0. WARNING

===========

Fiddling with these settings can result in an

unpredictable or even unstable system behavior. As

for -rt (group) scheduling, it is assumed that root

users know what they’re doing.

17

Scheduling Guarantee

3 parameters

I runtime

I period

I deadline

The task receives runtime time units within deadline if a properadmission control strategy is used.

WCET vs. runtime?

In the example: deadline = period

18

Constant Bandwidth Server Scheduling (CBS) [EDF]

EDF-based scheduling algorithm that allows:

I periodic (sporadic) tasks with hard deadlinesutilization Up

I a CB server s with given utilization Us for “soft tasks”CBS ensures bounded utilization even in overload

I schedulable iff Us + Up ≤ 1

Tasks in CBS:

I Task cannot exceed its registered budget,

I Task cannot be unblocked when a ratio between remainingbudget and remaining deadline is higher than bandwidth.

19

Constant Bandwidth Server Scheduling (CBS) [EDF]

EDF-based scheduling algorithm that allows:

I periodic (sporadic) tasks with hard deadlinesutilization Up

I a CB server s with given utilization Us for “soft tasks”CBS ensures bounded utilization even in overload

I schedulable iff Us + Up ≤ 1

Tasks in CBS:

I Task cannot exceed its registered budget,

I Task cannot be unblocked when a ratio between remainingbudget and remaining deadline is higher than bandwidth.

19

Task interface

Specifying a periodic/sporadic task requires:

I a (maximum/typical) instance execution time,

I a minimum interval between consecutive instances,

I a time constraint by which each instance must be completed.

Therefore:

I struct sched attr with all the necessary fields

I sched setattr() and sched getattr()

scheduling related syscalls that manipulate sched attr

20

Task interface

Specifying a periodic/sporadic task requires:

I a (maximum/typical) instance execution time,

I a minimum interval between consecutive instances,

I a time constraint by which each instance must be completed.

Therefore:

I struct sched attr with all the necessary fields

I sched setattr() and sched getattr()

scheduling related syscalls that manipulate sched attr

20

Admission control

I interface parameters are only used at admission control timewhen user calls sched setattr()

I system-wide settings under proc file system

I for a root domain comprising M CPUs, -deadline tasks canbe created while the sum of their bandwidths stays below:M * (sched rt runtime us / sched rt period us)

I disable by writing -1 in/proc/sys/kernel/sched rt runtime us

21

minimal main. . . [EDF]

22

Thanks!

References I

Deadline task scheduling.https:

//www.kernel.org/doc/Documentation/scheduler/sched-deadline.txt.Accessed: Oct, 2015.

Frequently Asked Questions: RTwiki.https://rt.wiki.kernel.org/index.php/Frequently_Asked_Questions,2007.Accessed: Oct, 2015.

HOWTO: Build an RT-application Jump to: navigation, search.https://rt.wiki.kernel.org/index.php/HOWTO:_Build_an_RT-application,2007.Accessed: Oct, 2015.

RT PREEMPT HOWTO.https://rt.wiki.kernel.org/index.php/RT_PREEMPT_HOWTO, 2006.Accessed: Oct, 2015.

John A. Stankovic.Misconceptions about real-time computing.IEEE Computer, 21(10):10–19, 1988.

24

https://www.kernel.org/doc/Documentation/scheduler/sched-deadline.txt

https://www.kernel.org/doc/Documentation/scheduler/sched-deadline.txt

https://rt.wiki.kernel.org/index.php/Frequently_Asked_Questions

https://rt.wiki.kernel.org/index.php/HOWTO:_Build_an_RT-application

https://rt.wiki.kernel.org/index.php/RT_PREEMPT_HOWTO

Real-Time Linux (RT-Preempt) + SCHED DEADLINE (Linux ...

Documents