I/O Devices and Drivers

I/O Devices and Drivers

Vivek Pai / Kai LiPrinceton University

2

Gaining FlexibilityQuestion: how do you make a file descriptor refer to non-files?Answer: treat it as an object

System calls have a shared part of codeActual work done by calls to function ptrsEach type of object exports a structure of func ptrs that handle all file-related syscalls

3

Where Have We Seen This?

Internals of read( ) system call descending down to fop_read methodOther places where this might be good?

Filesystems – want to support local, network, CD-ROM, legacyOther I/O Devices

4

Using “Virtual Nodes”struct vnode { u_long v_flag; /* vnode flags (see below) */ int v_usecount; /* reference count of users */ int v_writecount; /* reference count of writers */ int v_holdcnt; /* page & buffer references */ u_long v_id; /* capability identifier */ struct mount *v_mount; /* ptr to vfs we are in */ vop_t **v_op; /* vnode operations vector */ TAILQ_ENTRY(vnode) v_freelist; /* vnode freelist */ TAILQ_ENTRY(vnode) v_nmntvnodes; /* vnodes for mount point */[…] enum vtype v_type; /* vnode type */[…] struct vm_object *v_object; /* Place to store VM object */[…] enum vtagtype v_tag; /* type of underlying data */ void *v_data; /* private data for fs */[…]}

5

More Vnode Infoenum vtype { VNON, VREG, VDIR, VBLK, VCHR, VLNK, VSOCK, VFIFO, VBAD };enum vtagtype {

VT_NON, VT_UFS, VT_NFS, VT_MFS, VT_PC, VT_LFS, VT_LOFS, VT_FDESC, VT_PORTAL, VT_NULL, VT_UMAP, VT_KERNFS, VT_PROCFS, VT_AFS, VT_ISOFS, VT_UNION, VT_MSDOSFS, VT_TFS, VT_VFS, VT_CODA, VT_NTFS, VT_HPFS, VT_NWFS, VT_SMBFS };

6

Definitions & General Method

OverheadCPU time to initiate operation (cannot be overlapped)

LatencyTime to perform 1-byte I/O operation

BandwidthRate of I/O transfer, once initiated

General methodAbstraction of byte transfersBatch transfers into block I/O for efficiency to prorate overhead and latency over a large unit

7

Programmed I/O “Slow” Input Device

DeviceData registersStatus register(ready, busy, interrupt, … )

A simple mouse designPut (X, Y) in data registers on a moveInterrupt

Perform an inputOn an interrupt

reads values in X, Y registerssets ready bitwakes up a process/thread or execute a piece of code

CPU

Memory L2Cache

I/O Bus

InterfaceX Y

8

Programmed I/O Output Device

DeviceData registersStatus registers (ready, busy, … )

Perform an outputPolls the busy bitWrites the data to data register(s)Sets ready bitController sets busy bit and transfers dataController clears the ready bit and busy bit

9

Direct Memory Access (DMA)

Perform DMA from host CPUDevice driver call (kernel mode)Wait until DMA device is freeInitiate a DMA transaction(command, memory address, size)Block

DMA interfaceDMA data to device(size--; address++)Interrupt on completion(size == 0)

Interrupt handler (on completion)

Wakeup the blocked process

CPU

Memory L2Cache

I/O BusDMA

Interface

Free to movedata during

DMA

10

Device Drivers

Rest of theoperating

system

Devicedriver

Devicedriver

...

Devicedriver

I/O System

Devicecontroller

Devicecontroller

...Device

controller

Device

Device

Device

Device

11

Device Driver Design Issues

Operating system and driver communicationCommands and data between OS and device drivers

Driver and hardware communicationCommands and data between driver and hardware

Driver operationsInitialize devicesInterpreting commands from OSSchedule multiple outstanding requestsManage data transfersAccept and process interruptsMaintain the integrity of driver and kernel data structures

12

Device Driver InterfaceOpen( deviceNumber )

Initialization and allocate resources (buffers)Close( deviceNumber )

Cleanup, deallocate, and possibly turnoffDevice driver types

Block: fixed sized block data transfer Character: variable sized data transferTerminal: character driver with terminal controlNetwork: streams for networking

13

Block Device Interfaceread( deviceNumber, deviceAddr, bufferAddr )

transfer a block of data from “deviceAddr” to “bufferAddr”

write( deviceNumber, deviceAddr, bufferAddr )transfer a block of data from “bufferAddr” to “deviceAddr”

seek( deviceNumber, deviceAddress )move the head to the correct positionusually not necessary

14

Character Device Interfaceread( deviceNumber, bufferAddr, size )

reads “size” bytes from a byte stream device to “bufferAddr”

write( deviceNumber, bufferAddr, size )write “size” bytes from “bufferSize” to a byte stream device

15

Unix Device Driver Interface Entry Points

init( ): Initialize hardwarestart( ): Boot time initialization (require system services)open(dev, flag, id): initialization for read or writeclose(dev, flag, id): release resources after read and writehalt( ): call before the system is shutdownintr(vector): called by the kernel on a hardware interruptread/write calls: data transferpoll(pri): called by the kernel 25 to 100 times a secondioctl(dev, cmd, arg, mode): special request processing

16

What Was That Last One?The system call “ioctl”

SYNOPSIS #include <sys/ioctl.h>

int ioctl(int d, unsigned long request, ...);

DESCRIPTION The ioctl() function manipulates the underlying device

parameters of special files. In particular, many operating characteristics of character special files (e.g. terminals) may be controlled with ioctl() requests. The argument d must be an open file descriptor.

17

Any Counterparts?“fcntl” – operations on files

Duplicating file descriptorsGet/set/clear “close-on-exec” flagGet/set/clear flags – nonblocking, appending, direct (no-cache), async signal notification, locking & unlocking

Also available as dup( ), dup2( ), lockf( ), flock( ) and others

18

Any Other Non-Orthogonality

Sending data, credentials, file descriptors over sockets

SYNOPSIS ssize_t send(int s, const void *msg, size_t len, int flags); ssize_t sendto(int s, const void *msg, size_t len, int flags, const struct sockaddr *to, socklen_t tolen); ssize_t sendmsg(int s, const struct msghdr *msg, int flags);

DESCRIPTION Send(), sendto(), and sendmsg() are used to transmit a message to

another socket. Send() may be used only when the socket is in a connected state, while sendto() and sendmsg() may be used at any time.

19

Why BufferingSpeed mismatch between producer and consumer

Character device and block device, for exampleAdapt different data transfer sizes

Packets vs. streamsSupport copy semanticsDeal with address translation

I/O devices see physical memory, but programs use virtual memory

SpoolingAvoid deadlock problems

CachingAvoid I/O operations

20

Asynchronous I/OWhy do we want asynchronous I/O?

Life is simple if all I/O is synchronousHow to implement asynchronous I/O?

On a readcopy data from a system buffer if the data is thereOtherwise, initiate I/OHow does process find out about completion?

On a writecopy to a system buffer, initiate the write and return

21

Other Design IssuesBuild device drivers

staticallydynamically

How to download device driver dynamically?

load drivers into kernel memoryinstall entry points and maintain related data structuresinitialize the device drivers

22

Dynamic Binding with An Indirect Table

Open( 1, … );

Driv

er-k

erne

l int

erfa

ce Driver for device 0

…

open(…) {}

read(…) {}Driver for device 1

…

open(…) {}

read(…) {}

Indirect table

OtherKernelservices

Interrupthandlers

23

Dynamic BindingDownload drivers by users (may require a reboot)

Allocate a piece of kernel memoryPut device driver into the memoryBind device driver with the device

Pros: flexible and support ISVs and IHVsCons: security holes