490dp Synchronous vs. Asynchronous Invocation Robert Grimm.

490dpSynchronous vs. Asynchronous

Invocation

Robert Grimm

Definitions by Merriam-Webster

• Synchronism– chronological arrangement of historical events and

personages so as to indicate coincidence or coexistence; also : a table showing such concurrences

• Asynchrony– the quality or state of being asynchronous :

absence or lack of concurrence in time• Invocation

– the act or process of petitioning for help or support

What’s It All About

• Control flow– Model– Usage– Implementation

Synchronous Invocation

• Model– Structured programming– One process waits for another

to complete a sub-task• Usage

– Procedure / method call• Implementation

– Call instruction, stack

Asynchronous Invocation

• Model– Sending a message to another process

• Usage– Raising an event to be processed

by an event handler• Implementation

– Event loop• Queue of pending events

Adding Concurrency

• Process several tasks at the same time• Synchronous invocation

– Threads• Stack, registers• Locks, condition variables

• Asynchronous invocation– “Just” drop events for different tasks

into event loop

Advantages of Threads

• Sequential programming style– Familiar to programmers

• Mature programming tools– Multi-threaded debuggers

• Scalability with number of processors– One thread per processor

Problems with Threads

• Elimination of race conditions– Michael Burrows– Eraser [Savage et al., TOCS, (15)4:391-411]

• Performance tuning– Lock contention

Advantages of Events

• “Natural” fit to some problem domains– User interfaces

• Robustness under pressure– Better scalability with load

Problems with Events

• Debugging– No standard tools

• Performance– Events do not scale over processors– Event processing may still block

• Page faults, GC

The Best of Both Worlds

• Welsh et al.– Spectrum from threads to pure events

• Components– Queue of pending events– Pool of one or more threads– Taken together

• Task handler or animator

Design Patterns

• Wrap• Pipeline• Combine• Replicate

Wrap

• Wrap a functional unit with an animator• Simplest case

– One thread in pool– No concurrency issues

• More complex– Multiple threads in pool– Higher throughput

Pipeline

• Split functional unit into two or more– Units are cascaded– Each unit has its own animator

• Limit number of threads for low-concurrency operations– I/O operations

• Increase locality– Process less code– Touch less data

Combine

• Merge functional units and their animators• Inverse of pipeline• Limit overall number of threads

Replicate

• Copy functional unit and animator• Improve parallelism

– Increased throughput• Improve reliability

– Added resilience

Summary of Patterns

• Wrap to introduce load conditioning• Pipeline to avoid wasting threads• Pipeline for cache performance• Replicate for fault tolerance• Replicate to scale concurrency• Combine to limit the number of threads

Coding for Spectrum

• Stateless functional units– No concurrency control necessary

• Pass-by-value to avoid sharing– Consistency– Access– Reclamation– Alternative: Release references

• Avoid fate sharing– Improve fault tolerance

• Admission control at front– Avoid internal congestions

Coding for Spectrum

• Stateless functional units– No state shared between units

• Pass-by-value– Crucial!

• Avoid fate sharing– Part of replicate pattern

• Admission control at front– Good practice

Break

The Case for Events

• Scale better than just threads

• Make execution state explicit– Needed for check-pointing and migration

• Provide control over scheduling– Useful for real-time applications

TinyOS Palm OS ChinookWindowsMac OS

Ninja

The Case for a Hybrid Scheme

• Enables better performance conditioning– As discussed by Welsh et al.

• Provides isolation between applications– Separate applications from each other

• Activation, termination• Check-pointing, migration

– Separate core system from applications

Practical Considerations

• Functional units in Ninja– Explicitly include task handlers– Call specific event handlers

• Important goal for one.world– Flexibility, ability to experiment

• Composition of functional units• Break-down amongst animators

Functionality

• Events and event handlers– Basic abstractions

• Components– Building blocks for functional units– Export and import event handlers

• Statically declared• Dynamically linked

– Instantiated in environments• Containers for stored tuples, components,

other environments

Implementation of Event Passing

• Simple implementation• Animator

– Queue and pool of one or more threads– Mechanism– Represented by concurrency domain

• One per environment

Simple Implementation

• Method invocation– Default for event passing within environment

• Advantages– Simple, cheap

• Problems– Breaks for some interactions

• Infinite recursion– Sender needs to wait for long time

• Thread is busy executing event handler

Animators

• Default for event passing across environments– Forcibly linked through animator

• Advantages– Handles all interactions– Sender regains control immediately

• Drawback– More expensive

Concurrency

• Remember: Pool of one or more threads• Declare components concurrency safe

– Protect access to internal state• Least common denominator for environment

– No more than one thread if any component is not concurrency safe

Putting It All Together

• Environment determines– Concurrency domain and animator

• Concurrency safe flag determines– Number of threads

• Forcibly linked flag determines– Simple implementation or animator