Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010.

Testing Concurrent Programs, A 7-Minute Jargon-Free

IntroductionThesis Writing Seminar

Mathias Ricken

Rice University

February 25, 2010

2

Computer Annoyances

• Program bugs • Slow computers

3

Unit Testing

• Test program parts, not the whole program– Test parts individually, then together– Smaller parts larger parts

• Unit tests are programs– Compare expected result to actual result:

assertEquals(12, multiply(3,4));

4

Moore’s Law

Adopted fromSutter 2009

5

Existing Approaches Fail

• Simple Unit Testing Frameworks– Example: JUnit– Assume program is single-threaded– Many errors go undetected

detected

not detected

Main thread:

Other thread:

6

Problem: Non-Determinism

• Synchronization points– Information exchanged between threads– Order of threads reaching sync. points is

non-deterministic

7

Problem: Non-Determinism

• Synchronization points– Information exchanged between threads– Order of threads reaching sync. points is

non-deterministic

– Program could succeed in one order but fail in another

8

Comprehensive Tools: Costly

• Model checking or schedule-based execution– Examples: JPF, ExitBlock– Test all arrangements– Too costly

t = # of threads

s = # of sync. points

N = # of arrangements

t s N2 1 62 2 202 3 703 2 16803 4 7.6E+053 8 2.3E+114 8 2.1E+19

Longer than the universe exists!

(if N=1 1 minute)

9

• Insert random delays before synchronization points to modify the order

• Run tests many times with different delays

Probabilistic Approach

10

Probabilistic Approach

• Insert random delays before synchronization points to modify the order

• Run tests many times with different delays

delay

11

(Proposed) Contributions

• Probabilistic Approach– Random delays at synchronization points

• Call Graph Analysis– Determine which parts of the program may interact– Only insert delays where necessary

• Improved JUnit Framework– Exception handler for all threads– Ensures that all errors in all threads are detected

12

Evaluation

• Tested DrJava with Improved JUnit– Number of unit tests: 900– Previously unknown problems: 20– Slowdown: ~1 percent

• Other parts still under development

13

Conclusion

• Unit testing of concurrent programs will become more important

• Concutest helps…– by improving JUnit’s abilities– by using probabilistic techniques– by using call graph analysis to reduce the

number of synchronization points

14

Image Attribution1. Left image on Computer Annoyances:

Adapted from Microsoft

2. Graph on Moore’s Law:Adapted from Herb Sutter 2009

3. Top image on Multi-Cores and Concurrency:Adapted from Brian Goetz et al. 2006, Addison Wesley

4. Bottom image on Multi-Cores and Concurrency:Caption Fridays

15

Extra Slides

16

Multi-Cores and Concurrency

• Recent trends– Clock speed stagnated– Transistors still increasing– Reason: multi-core CPUs

• Implication– Must use concurrency to

benefit from modern CPUs– Concurrent programming is

difficult

17

• Number of schedules (N)– t: # of threads, s: # of slices per thread

– Note: s is # of slices (between synchronization points)

Number of Arrangements

18

Product of s-combinations

For thread 1: choose s out of ts time slicesFor thread 2: choose s out of ts-s time slices…For thread t-1: choose s out of 2s time slicesFor thread t-1: choose s out of s time slices

Writing s-combinations using factorial

Cancel out terms in denominator and next numerator

Left with (ts)! in numerator and t numerators with s!

Number of Arrangements