Distributed Processing Systems (Multithreaded Programming) Modified from Original Slides by Rajkumar Buyya “Concurrent Programming with Threads” (rajkumar/tut/multi-threading.ppt)

Distributed Processing Systems

(Multithreaded Programming)

Modified from Original Slides by Rajkumar Buyya “Concurrent Programming with Threads”

(http://www.dgs.monash.edu.au/~rajkumar/tut/multi-threading.ppt)

오 상 규오 상 규

서강대학교 정보통신 대학원 서강대학교 정보통신 대학원

Email : [email protected] : [email protected]

Page 2

Multi-Threaded Programming

서강대학교 정보통신 대학원

Shared memory

segments, pipes, open

files

Shared memory

segments, pipes, open

files

Basic Process Model Basic Process Model

DATADATA

STACK

TEXTTEXT

DATADATA

STACK

TEXTTEXT

process process

Shared Memory maintained by kernel

System Call

Page 3



Multitasking Systems Multitasking Systems

Hardware

The Kernel

P1 P2 P3 P4

Processes

Each process is completely independent. Too heavy for process creation, context switching.

Context Switch

Page 4



What is Thread ? What is Thread ?

A basic unit of CPU utilization, and consists of a PC, a register set and a stack space (similar to process) - a single stream of control.

An individual thread has at least its own register state, and usually its own stack.

Multiple threads can be created within a same process and share code, address space, and operating resources for the process.

The environment in which a thread executes is called task, and a traditional (heavyweight) process is equal to a task with one thread.

Thread makes creation and switching inexpensive.

Page 5



Threaded Process Model Threaded Process Model

THREAD STACK

THREAD STACK

THREAD DATA

THREAD DATA

THREAD TEXT

THREAD TEXT

SHARED MEMORY

SHARED MEMORY

Threads within a process Independent executables (multiple threads of control). All threads are parts of a process hence communication easier and simpler. Lightweight thread creation and context switching.

Page 6



Single-threaded Process

Single instruction stream Multiple instruction stream

Multiple-threaded ProcessThreads ofExecution

CommonAddress Space

Single and Multithreaded Processes Single and Multithreaded Processes

Page 7



UserCode

GlobalData

The Kernel

Process Structure

(Kernel state and address space are shared)

T1’s SP T3’sPC T1’sPC T2’sPC

T1’s SP

T2’s SP

Scheduling Perspective Scheduling Perspective

Context Switching

Thread switching

Page 8



User-Level Thread vs. Kernel-Level ThreadUser-Level Thread vs. Kernel-Level Thread User-level threads; supported above the kernel, via a set of library calls at

the user level

Kernel-supported threads (Mach, OS/2, NT).

Threads Library

P

User

Space

KernelSpace

- No switch to kernel mode, fast

- Application-specific scheduling

- Portable

* Blocking problem

* Cannot take advantage of

multiprocessor

P

User

Space

KernelSpace

- No thread management in the

user space

- Schedule multiple threads from

the same process on multiple

processors

- No blocking problem

- Can have multithreaded kernel

* Need context switch for thread

operations

* Slow compared to user-level

Page 9



Structured program design

Achieve concurrency and parallelism

Throughput (overlap computation and I/O)

Responsiveness

System resource usage

Distributed objects

Single source across platforms (POSIX)

Single binary for any number of CPUs

The Value of Multi-Thread The Value of Multi-Thread

Page 10



Improve efficiency on uni-processor systems

Use multiprocessor hardware

Improve Throughput

Simple to implement Asynchronous I/O

Leverage special features of the OS

If all operations are CPU intensive do not go far on multithreading

Thread creation is very cheap, it is not free

Thread that has only five lines of code would not be useful

To Thread or Not To Thread To Thread or Not To Thread

Page 11



Thread Mapping and Scheduling Thread Mapping and Scheduling

Parallel Execution due to Concurrency of threads on Virtual Processors Concurrency of threads on Physical Processor

True Parallelism : Single Thread : Single Processor = 1:1

User Level ThreadsUser Level Threads

Virtual ProcessorsVirtual Processors

Physical ProcessorsPhysical Processors

User-Level Schedule (User)

Kernel-Level Schedule (Kernel)LWP in Solaris

Page 12



M : 1 HP-UNIX

1 : 1 DEC, NT, OS/1, AIX. IRIX

M : M 2 - level

Kernel Scheduling Design Options Kernel Scheduling Design Options

Many Threads on One LWP One Thread per LWP

Many Threads on Many LWPs Solaris Two-level Model

Page 13



Proc 1 Proc 2 Proc 3 Proc 4 Proc 5

Traditionalprocess

User

LWPs

Kernelthreads

Kernel

Hardware Processors

SunOS Two-Level Thread ModelSunOS Two-Level Thread Model Many-to-Many model with the ability to specifically request a one-to-one

binding for individual threads.

Page 14



RUNNABLERUNNABLE

SLEEPINGSLEEPINGSTOPPEDSTOPPED

ACTIVEACTIVE

Stop

Continue

Preempt Stop

Stop Sleep

Wakeup

Scheduling States Scheduling States

Simplified View of Thread State Transitions

Preempted vs. Non-preempted Scheduling

Page 15



Few Popular Thread Models Few Popular Thread Models

POSIX, ISO/IEEE standard

Mach C threads, CMU

Sun OS LWP threads, Sun Microsystems

PARAS CORE threads, C-DAC

Java-Threads, Sun Microsystems

Chorus threads, Paris

OS/2 threads, IBM

Windows NT/95 threads, Microsoft

http://www.informatik.hu-berlin.de/~mueller/pthreads/dc_threads.html

Page 16



Process and Threaded Models Process and Threaded Models

thr_exit()exit( )Exit and destroy the

thread

thr_join()wait( )Wait for completion of thread

thr_create() builds the new thread and starts the execution

exec( )Start execution of a

new thread

thr_create( )fork( )Creation of a new

thread

Threads ModelProcess ModelPurpose

Page 17



Code Comparison Code Comparison

Segment ( Process ) main ( )

{

fork( );

fork( );

fork( ); }

Segment ( Process ) main ( )

{

fork( );

fork( );

fork( ); }

Segment ( Thread ) main()

{

thread_create(0,0,func(),0,0);



}

Segment ( Thread ) main()

{




}

Page 18



Printing Thread

Editing Thread

Independent Threads Independent Threads

Page 19



reader()

{

- - - - - - - - - -

lock(buff[i]);

read(src,buff[i]);

unlock(buff[i]);

- - - - - - - - - -

}

writer()

{

- - - - - - - - - -

lock(buff[i]);

write(src,buff[i]);

unlock(buff[i]);

- - - - - - - - - -

}

buff[0]buff[0]

buff[1]buff[1]

Cooperative Parallel Synchronized

Threads

Cooperative Parallel Synchronized

Threads

Cooperative Threads - File Copy Cooperative Threads - File Copy

Page 20



ServerThreads

Message PassingFacility

Server ProcessClient Process

Client Process

User Mode

Kernel Mode

Multithreaded Server Multithreaded Server

Page 21



The Boss/Worker Model

The Peer Model

The Pipeline Model

Thread Programming Models Thread Programming Models

Page 22



taskXtaskX

taskYtaskY

taskZtaskZ

main ( )main ( )

WorkersProgram

Files

Resources

Databases

Disks

SpecialDevices

Boss

Input (Stream)

The Boss/Worker ModelThe Boss/Worker Model

Page 23



main() /* the boss */{ forever {

get a request;

switch ( request ) {

case X: pthread_create(....,taskX);

case Y: pthread_create(....,taskY);

....

}

}

}

taskX() /* worker */{ perform the task, sync if accessing shared resources

}

taskY() /* worker */{ perform the task, sync if accessing shared resources

}

....

Example (Boss/Worker Model) Example (Boss/Worker Model)

Page 24



taskXtaskX

taskYtaskY

WorkersProgram

taskZtaskZ

Input(static)

Input(static)

The Peer ModelThe Peer Model

Files

Resources

Databases

Disks

SpecialDevices

Page 25



main(){ pthread_create(....,task1);

pthread_create(....,task2);

....

signal all workers to start

wait for all workers to finish

do any cleanup

}

task1() /* worker */{ wait for start

perform the task, sync if accessing shared resources

}

task2() /* worker */{ wait for start

perform the task, sync if accessing shared resources

}

Example (Peer Model) Example (Peer Model)

Page 26



Resources Files

Databases

Disks

Special Devices

Files

Databases

Disks

Special Devices

Files

Databases

Disks

Special Devices

Stage 1Stage 1 Stage 2Stage 2 Stage 3Stage 3

Program Filter Threads

Input (Stream)

The Pipeline ModelThe Pipeline Model

Page 27



main()

{

pthread_create(....,stage1);

pthread_create(....,stage2);

...

pthread_create(....,stageN);

...

wait for all pipeline threads to finish

do any cleanup

}

stage1()

{

get next input for the program

do stage 1 processing of the input

pass result to next thread in pipeline

}

Example (Pipeline Model) (1) Example (Pipeline Model) (1)

Page 28



stage2()

{

get input from previous thread in pipeline

do stage 2 processing of the input

pass result to next thread in pipeline

}

...

stageN()

{

get input from previous thread in pipeline

do stage N processing of the input

pass result to program output.

}

Example (Pipeline Model) (2) Example (Pipeline Model) (2)

Page 29



main() main(){ ... { pthread_create( func, arg); thr_create(..func..,arg..); ... ...} }void * func(){ ....}

pthread_exit()

T2

T1

pthread_create(...func...)

POSIX SOLARIS

Thread Create Thread Create

Page 30



main() main(){ . . . { pthread_join(T2); thr_join(T2,&val_ptr); . . . . . .} }void * func(){ . . .}

pthread_exit()

T2

T1

pthread_join()

Waiting for a Thread to Exit Waiting for a Thread to Exit

POSIX SOLARIS

Page 31



The normal C function exit() always causes the process to exit.

That means all of the process -- All the threads.

The thread exit functions:

UI : thr_exit()

POSIX : pthread_exit()

OS/2 : DosExitThread() and _endthread()

NT : ExitThread() and endthread()

all cause only the calling thread to exit, leaving the process intact

and all of the other threads running.

(If no other threads are running, then exit() will be called.)

Thread Exit Thread Exit

Page 32



Cancellation is the means by which a thread can tell another thread

that it should exit.

main() main() main(){... {... {...pthread_cancel (T1); DosKillThread(T1); TerminateThread(T1)} } }

There is no special relation between the killer of a thread and the victim. Requires careful programming.

(pthread exit)

POSIX OS/2 Windows NT

(pthread cancel( ))

T1

T2

Thread Cancellation Thread Cancellation

Page 33



main()

{

... thr_suspend(T1); ... thr_continue(T1); ...}

Continue ( )

T2

T1

Suspend ( )

* POSIX does not support thread suspension

Suspending a Thread Suspending a Thread

SOLARIS

Page 34



On shared memory : shared variables - semaphores

On distributed memory :

Within a task : semaphores, etc. Across the tasks : By passing messages

Unsynchronized Shared Data is a Formula for Disaster

Thread1 Thread2

temp = Your - > BankBalance;

dividend = temp * InterestRate;

newbalance = dividend + temp;

Your->Dividend += dividend; Your->BankBalance+=deposit;

Your->BankBalance = newbalance;

Threads Synchronization Threads Synchronization

Page 35



reader()

{

- - - - - - - - - -

lock(DISK);

...........

...........

...........

unlock(DISK);

- - - - - - - - - -

}

reader()

{

- - - - - - - - - -

lock(DISK);

...........

...........

...........

unlock(DISK);

- - - - - - - - - -

}

writer()

{

- - - - - - - - - -

lock(DISK);

..............

..............

unlock(DISK);

- - - - - - - - - -

}

writer()

{

- - - - - - - - - -

lock(DISK);

..............

..............

unlock(DISK);

- - - - - - - - - -

}

Shared Data

T1 T2

Critical Section (Good Programmer !) Critical Section (Good Programmer !)

Page 36



reader()

{

- - - - - - - - - -

lock(DISK);

...........

...........

...........

unlock(DISK);

- - - - - - - - - -

}

reader()

{

- - - - - - - - - -

lock(DISK);

...........

...........

...........

unlock(DISK);

- - - - - - - - - -

}

writer()

{

- - - - - - - - - -

- - - - - - - - - -

..............

..............

- - - - - - - - - -

- - - - - - - - - -

}

writer()

{

- - - - - - - - - -

- - - - - - - - - -

..............

..............

- - - - - - - - - -

- - - - - - - - - -

}

Shared Data

T1 T2

Critical Section (Bad Programmer !) Critical Section (Bad Programmer !)

Page 37



getc() OLD implementation :

extern int get( FILE * p )

{

/* code to read data */

}

getc() NEW implementation :

extern int get( FILE * p )

{

pthread_mutex_lock(&m);

/* code to read data */

pthread_mutex_unlock(&m);

}

Are Libraries Safe ? (MT-Safe) Are Libraries Safe ? (MT-Safe)

Page 38



In UNIX, the distinguished variable errno is used to hold the error code

for any system calls that fail.

Clearly, should two threads both be issuing system calls around

the same time, it would not be possible to figure out which one set

the value for errno.

Therefore errno is defined in the header file to be a call to

thread-specific data.

This is done only when the flag_REENTRANT (UI) _POSIX_C_SOURCE = 199506L (POSIX) is passed to the compiler,

allowing older, non-MT programs to continue to run.

ERRNO ERRNO

Page 39



Different Thread Specifications (1) Different Thread Specifications (1)

Functionality UI Threads POSIX Threads NT Threads OS/2 Threads

Design PhilosophyBase

PrimitivesNear-Base Primitives

Complex Primitives

Complex Primitives

Scheduling Classes

Local/ Global Local/ Global Global Global

Mutexes Simple Simple Complex Complex

Counting Semaphores

Simple Simple Buildable Buildable

R/W Locks Simple Buildable Buildable Buildable

Condition Variables

Simple Simple Buildable Buildable

Multiple-Object

SynchronizationBuildable Buildable Complex Complex

Page 40



Different Thread Specifications (2) Different Thread Specifications (2)

Functionality UI Threads POSIX Threads NT Threads OS/2 Threads

Thread Suspension YesImpossible

Yes Yes

Cancellation

BuildableYes Yes Yes

Thread-Specific Data Yes Yes Yes Yes

Signal-Handling

PrimitivesYes Yes N/A N/A

Compiler Changes

RequiredNo No Yes No

Vendor Libraries MT-safe?

Most Most All? All?

ISV Libraries MT-safe? Some Some Some Some

Page 41



thread cancellation

scheduling policies

sync attributes

thread attributes

continue

suspend

semaphore vars

concurrency setting

reader/ writer vars

daemon threads

join

exit key creation

priorities sigmask create

thread specific data

mutex vars kill

condition vars

POSIX API Solaris API

POSIX and Solaris API Differences

Page 42



Compiling UI under Solaris

Compiling is no different than for non-MT programs libthread is just another system library in /usr/lib Example:

%cc -o sema sema.c -lthread -D_REENTRANT

All multithreaded programs should be compiled using the _REENTRANT flag

Applies for every module in a new application If omitted, the old definitions for errno, stdio would be used, which yo

u don’t want

All MT-safe libraries should be compiled using the _REENTRANT flag, even though they may be used single in a threaded program.

Page 43



Compiling POSIX under Solaris

Compiling is no different than for non-MT programs libpthread is just another system library in /usr/lib Example :

%cc-o sema sema.c -lpthread -lposix4

- D_POSIX_C_SOURCE=19956L

All multithreaded programs should be compiled using the

_POSIX_C_SOURCE=199506L flag Applies for every module in a new application If omitted, the old definitions for errno, stdio would be used, which yo

u don’t want.

All MT-safe libraries should be compiled using the _POSIX_C_SOURCE =199506L flag, even though they may be used single in a threaded program

Page 44



void main( int argc, char *argv[] )

{

int server_socket, client_socket, clilen;

struct sockaddr_in serv_addr, cli_addr;

int one, port_id;

pthread_t service_thr;

port_id = 4000; /* default port_id */

if ( ( server_socket = socket( AF_INET, SOCK_STREAM, 0 ) ) < 0 ) {

printf("Error: Unable to open socket in parmon server.\n");

exit( 1 );

}

Example: Multithreaded Server (1) Example: Multithreaded Server (1)

Page 45



memset( (char*) &serv_addr, 0, sizeof(serv_addr));

serv_addr.sin_family = AF_INET;

serv_addr.sin_addr.s_addr = htonl(INADDR_ANY);

serv_addr.sin_port = htons(port_id);

setsockopt(server_socket, SOL_SOCKET, SO_REUSEADDR, (char *)&one,

sizeof(one));

if ( bind(server_socket, (struct sockaddr *)&serv_addr, sizeof(serv_addr) ) < 0 ) {

printf( "Error: Unable to bind socket in parmon server->%d\n",errno );

exit(1);

}

listen( server_socket, 5);


Page 46



while ( 1 ) {

clilen = sizeof(cli_addr);

client_socket = accept(server_socket, (struct sockaddr *)&serv_addr,

&clilen);

if ( client_socket < 0 ) {

printf( "connection to client failed in server.\n" );

continue;

}

pthread_create( &service_thr, NULL, service_dispatch, client_socket);

}


Page 47



/* Service Thread */

void *service_dispatch(int client_socket)

{

/* Get user request */

if ( read(client_socket, command, 100) > 0 ) {

/* Identify user request */

/* Do necessary processing, synchronize if necessary */

/* Send results back to the server */

}

/* Close connect and terminate thread */

close(client_socket);

pthread_exit( (void *)0);

}


Page 48



Threads provide a more natural programming paradigm.

Improve efficiency on uni-processor systems.

Allows to take full advantage of multiprocessor hardware.

Improve throughput: simple to implement asynchronous I/O.

Leverage special features of the OS.

Many applications are already multithreaded.

MT is not a silver bullet for all programming problems.

There is already a standard for multithreading--POSIX.

Multithreading support already available in the form of language syntax

(e.g., Java).

Threads allows to model the real world object (e.g.,: in Java).

Summary Summary

Distributed Processing Systems (Multithreaded Programming) Modified from Original Slides by Rajkumar Buyya “Concurrent Programming with Threads” (rajkumar/tut/multi-threading.ppt)

Documents