04 Kernel Prog Mechanisms-V2.6

8/11/2019 04 Kernel Prog Mechanisms-V2.6

1/25

Copyright 2003 - ATR Labs - All Rights Reserved.

Kernel Programming Mechanisms

For Version 2.6 kernel


2/25

Kernel topics covered

1. Interrupts & Interrupt Strategies

a) Concepts

b) Interrupt Strategies - Reason for deferralc) APIs

2. Bottom Halves / deferred procedures

a) History - BH mechanism, Task Queues

b) Softirqs

c) Taskletsd) Workqueues

3. Kernel Synchronization

a) Critical Regions, Race Conditions, Deadlocks

b) Atomic Ops

c) Spinlocksd) Kernel Semaphores

e) Completion Variables

f) Seq Locks

g) Barriers

4. Kernel Timers & Timing Mechanisms


3/25

Interrupts & Interrupt Strategies

Concepts

1. CPU INT-Line, IRQ Lines, The PIC, Device Interrupts2. Acknowledging an interrupt

3. System degradation

4. Types of interrupts & Interrupt Strategies

a) The timer interruptb) The UART 16550A ( Serial Port ) interrupt / FIFO Buffers

c) The Network Card ( NIC ) interrupt

d) Edge Triggered versus Level Triggered / Shared Interrupts

e) PCI Bus - Device Interrupts

f) USB Bus - Interrupt Handling by Host Controller.5. Fast and Slow Interrupts - Running with interrupts enabled.

6. Interrupt context - Stack used by an interrupt, irrelevance of the

currentmacro, cannot use semaphores ( which can sleep ).


4/25

APIs for Interrupt Handlers

#include

int request_irq(unsigned int irq,

irqreturn_t (*handler)(int irq, void *devid, struct pt_regs *),unsigned long irqflags,

const char *devname,

void *dev_id);

irqflags= 0 (normal) or ORRED with SA_INTERRUPT, SA_SAMPLE_RANDOM, SA_SHIRQ

devname is shown in /proc/interrupts

dev_idneeds to be unique for shared interrupts.

void free_irq(unsigned int irq, void *dev_id);

local_irq_disable(); /* Disable interrupts ONLY for the Current Processor */

local_irq_enable(); /* Dangerous, if originally interrupts were disabled !! */

unsigned long flags; /* To restore back to original status, whether enabled

local_irq_save(flags); or disabled originally */

local_irq_restore(flags);

int in_interrupt();int in_irq();


5/25

APIs for Interrupt Handlers

disable_irq(unsigned int irq); /* enable or disable the given interrupt line */

enable_irq(unsigned int irq);

#include

int in_interrupt(); /* boolean - returns TRUE if inside an interrupt context,

including bottom half handlers */

int in_irq(); /* returns TRUE only if inside an Interrupt Handler */


6/25

Bottom Halves / Deferred Procedures

History :

Original Bottom Halves :

Initially, statically created list of 32 bottom-half handlers

The top half would mark a BH by setting a corresponding bit in a 32-bit integer.

Each BH was globally synchronized.

No two could run at same time even on different processors.

Task Queues : Introduced in 2.2.

Kernel defined a series of predefined queues

Scheduler Queue, Timer Queue, Immediate Queue,

Custom Queue etc.

Each queue contained a linked list of functions to call.

Softirqs and Tasklets - Introduced in 2.3

During 2.5 -> BH and Task Queues were dumped, Workqueues introduced.

Currently in 2.6 -> Three Deferred Procedure Mechanisms - softirqs, tasklets and

workqueues.

Bottom Half Status

BH Removed in 2.5

Task Queues Removed in 2.5

Softirq Available since 2.3

Tasklet Available since 2.3

Work queues Available since 2.5


7/25

Softirqs

Rarely used.

Statically allocated at compile time.

Reserved for most timing critical and important functions.

struct softirq_action {

void (*action)(struct softirq_action *);

void *data;

};

static struct softirq_action softirq_vec[32];

Maximum of 32 elements in this array of structures. Only 6 used in 2.6 kernel :

HI_SOFTIRQ 0 High priority tasklets

TIMER_SOFTIRQ 1 Timer bottom halfNET_TX_SOFTIRQ 2 Sending packets on the network

NET_RX_SOFTIRQ 3 Receiving packets on the network

SCSI_SOFTIRQ 4 SCSI bottom half

TASKLET_SOFTIRQ 5 Tasklets


8/25

Softirqscontinued

Softirq handler :

void softirq_handler(struct softirq_action *);

/* The kernel passes the entire structure and not just the data value, since this

allows for future expansion of the structure without breaking existing code */

Registering softirqs:

open_softirq(NET_TX_SOFTIRQ, net_tx_action, NULL); /* e.g networking */

Executing softirqs :

Registered softirq must be marked before it is executed.

Known as raising the softirq. Usually interrupt handlers raise softirqs.

raise_softirq(NET_TX_SOFTIRQ);

Softirqs are checked for and executed :

a) After processing a hardware interrupt

b) By the ksoftirqd kernel thread

c) By code that explicitly checks and executes (e.g Networking)


9/25

Tasklets

Deferred procedure mechanisms built on top of softirqs.

Represented by two special softirqs - HI_SOFTIRQ and TASKLET_SOFTIRQ.

HI_SOFTIRQ based tasklets run before TASKLET_SOFTIRQ.Simpler interface and relaxed locking rules.

struct tasklet_struct {

struct tasklet_struct *next;

unsigned long state;

atomic_t count;

void (*func)(unsigned long);

unsigned long data;

};

tasklet_schedule(struct tasklet_struct *);

tasklet_hi_schedule(struct tasklet_struct *);

See example 04_tasklet.c


10/25

Tasklets

Deferred procedure mechanisms built on top of softirqs.

Represented by two special softirqs - HI_SOFTIRQ and TASKLET_SOFTIRQ.

HI_SOFTIRQ based tasklets run before TASKLET_SOFTIRQ.Simpler interface and relaxed locking rules.

struct tasklet_struct {

struct tasklet_struct *next;

unsigned long state;

atomic_t count;

void (*func)(unsigned long);

unsigned long data;

}; /* You can initialize a struct like this with value */

ORDECLARE_TASKLET(name, function, data);

DECLARE_TASKLET_DISABLED(name, function, data);

tasklet_schedule(struct tasklet_struct *);

tasklet_hi_schedule(struct tasklet_struct *);


11/25

Taskletscontinued

As with softirqs tasklets cannot sleep.

Cannot use semaphores or other blocking functions inside a tasklet.

Tasklets run with all interrupts enabled, so kernel synchronization is important if

data is shared with other interrupts.

Unlike softirqs, two of the same tasklets NEVER run concurrently.

Two different tasklets can run simultaneously on different processors.

softirqs ( and therefore tasklets ) are aided by a set of per-processor kernel

threads.

If the number of softirqs increases heavily, kernel spawns a set of kernel threads

to handle the load. These threads run with lowest priority ( nice = 19 ). User

processes therefore are not starved of CPU time.


12/25

Workqueues

Different methodology - uses kernel threads

work always runs in process context.

Are schedulable entities - can sleep.

If deferred work needs to sleep -> Workqueues are used, NOT tasklets.

Useful for situations where you need a lot of memory, obtain a semaphore or

perform blocking I/O.

Basically, an interface for creating kernel threads to handle work.

Implemented using worker_thread() function.

After initial setup, this function enters an infinite loop. When work is queued, the

thread is awakened and processes the work. When queue is empty, it sleeps.


13/25

Workqueues continued

#include

struct work_struct {

unsigned long pending;struct list_head entry;

void (*func)(void *);

void *data;

void *wq_data; /* used internally */

struct timer_list timer; /* used by delayed work queues */};

Using the default work queue ( events/n queue ) where n = processor number.

DECLARE_WORK(workname, void (*func)(void *), void *data);

schedule_work(&workname);schedule_delayed_work(&workname, delay);

flush_scheduled_work(void);


14/25


15/25

Kernel Synchronization

Critical regions

Concurrency

Reentrancy

Race conditions

Deadlocks

Preemption

Symmetric multiprocessing


16/25

Atomic Operations - Integers

#include

atomic_t i; /* define i */

atomic_t v = ATOMIC_INIT(0); /* define and initialize */

atomic_set(&v, 4);

atomic_add(2, &v);

atomic_sub(2, &v);

atomic_inc(&v);atomic_dec(&v);

int atomic_read(&v); /* converts an atomic_t value to an int */

int atomic_dec_and_test(atomic_t *v);


17/25

Atomic Operations - Bit Operations

#include

void set_bit(int nr, void *addr);

void clear_bit(int nr, void *addr);void change_bit(int nr, void *addr); /* toggle value */

int test_and_set_bit(int nr, void *addr); /* Atomically set the nr-th bit starting

from addr, and return the previous value */

int test_and_clear_bit(int nr, void *addr);

int test_and_change_bit(int nr, void *addr);int test_bit(int nr, void *addr);


18/25

Spinlocks

Can be held by at most one thread of execution.

Only one thread is allowed inside a critical region.

Useful when more than 1 processor exists, and 2nd processor can spin rather thanto do a context switch and return.

On uniprocessor machines, the spinlock compiles away and do not exist.

The same lock mechanism is used to enable/disable kernel preemption.

Spin locks are NOT recursive. If you attempt to acquire a lock you already hold,

the kernel will hang.

Can be used in interrupt_handlers.

#include

spinlock_t mylock;

spin_lock(&mylock);

spin_unlock(&mylock);


19/25

Kernel semaphores

#include

struct semaphore sem;

sema_init(struct semaphore *sem, int);

down_interruptible(struct semaphore *);

up(struct semaphore *);

See example 04_semaphore.c


20/25

Completion variables

Easy way to synchronize between two tasks in the kernel.

Used when one task needs to signal another task that a n event has occurred.

One task waits on the completion variable, while 2nd task performs some work.

When 2nd task has completed, it uses the completion variable to wake up any

waiting tasks.

#include

DECLARE_COMPLETION(my_comp);

init_completion(struct completion *my_comp);

wait_for_completion(struct completion *); /* waits for completion signal */

complete(struct completion *); /* signals any waiting tasks to wake up */


21/25


22/25

Memory barriers

To ensure memory reads and writes are not optimized by the hardware or the

compiler.

rmb(;

wmb();

mb();


23/25

Kernel timers and timing mechanisms

The concept of the HZ /* declared in asm/param.h>

In the 2.6 kernel, HZ = 1000 on the x86 architecture.

extern unsigned long jiffies ; /* linux/jiffies.h>

jiffies gets incremented HZ times per second. ( 1000 ).

#include

void udelay(unsigned long usecs);void mdelay(unsigned long msecs);

set_current_state(TASK_INTERRUPTIBLE);

schedule_timeout(s * HZ ); /* task sleeps for s seconds */


24/25


struct timer_list {

struct list_head entry;unsigned long expires; /* in jiffies */

spinlock_t lock;

void (*function)(unsigned long); /* timeout handler */

unsigned long data;

/* some other internal fields */};

The timeout value is in jiffies.

Thus, timer -> function will run when jiffies >= timer-> expires.

The timeout is usually generated by taking the current value of jiffies and adding the

amount of the desired delay.

The function add_timer() inserts the timer into a sorted list which is then polled around

1000 times per second.


25/25


Functions to deal with timers :

void init_timer(struct timer_list *timer);

/* Use this instead of tampering with the structure directly. This zeroes prev and next andrunning flag on SMP systems. */

void add_timer(struct timer_list *timer);

int mod_timer(struct timer_list *timer, unsigned long expires);

int del_timer(struct timer_list *timer);

int del_timer_sync(struct timer_list *timer);

/* This function works like del_timer but ensures that NO CPU is running it */

04 Kernel Prog Mechanisms-V2.6

Documents

04 Kernel Prog Mechanisms-V2.6