The Linux Benchmark Project Randy Appleton Kurt Payne Joe Schmeltzer Carey Stortz benchmark.

The Linux Benchmark Project

Randy Appleton

Kurt Payne

Joe Schmeltzer

Carey Stortz

http://cs.nmu.edu/~benchmark

What We Did

• We are three undergraduates and a professor.

• Measured the source code of the kernel several ways.

• Measured the performance of the kernel several ways.

• This is undergraduate research, yet useful research.

Limitations

• We are kernel level only.

• We can describe, but only guess at why.

Why Linux?

• Linux is important.

• The source code is available.

• The development process is open.– Early versions available.– Lots of versions available.

• Linux is cool!

Which Kernels

• All kernels from 2.0.1 to 2.4.0.

• About 4.5 years of kernels.

• Taken every three months.– Intervals are by time, not version count.

• Ordered by version number.– Could have ordered by date.– Justifiable, but not obvious.

Counting Lines

• Counted– All lines– Lines with semicolons or braces– Bytes

• All the graphs look the same.

• Exponential growth at 26% per year.

• Overall, 319% growth from 2.0.0 to 2.4.0.

Line Count

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

Kernel Version

Lin

es (

Millio

ns)

Comparing Sizesfrom http://www.dwheeler.com/sloc/redhat71-v1/redhat71sloc.html

Redhat 7.1 25,000,000 (about)

Linux 2.4.2 2,437,470

Mozilla 2,065,224

XFree-86-4.0.3 1,837,608

GCC-2.96 984,076

Distribution of Lines Version 2.0.1

Arch Drivers FS Include Init IPC Kernel Lib MM Net12.9% 54.6% 8.7% 11.6% 0.1% 0.3% 0.9% 0.2% 0.9% 10.0%

Drivers

FS

Include

Init

IPC

Kernel

Lib

MM

Net

Distribution of LinesVersion 2.4.0-pre9

Arch

Drivers

FS

Include

Init

IPC

Kernel

Lib

MM

Net

Arch Drivers FS Include Init IPC Kernel Lib MM Net12.2% 62.1% 7.3% 10.5% 0.0% 0.2% 0.4% 0.1% 0.5% 6.8%

Distribution of Lines Over Time

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

2.0.

1

2.1.

1

2.1.

108

2.1.

124

2.1.

14

2.1.

32

2.1.

44

2.1.

58

2.1.

77

2.1.

92

2.2.

0-pr

e3

2.2.

6

2.3.

10

2.3.

20

2.3.

38

2.3.

99-p

re4

2.4.

0

2.4.

0-te

st3

2.4.

0-te

st9

Arch

Drivers

FS

Include

Init

IPC

Kernel

Lib

MM

Net

Measuring Performance

• We used lmbench.

• Ran at least 5 trials.

• Averaged the data– Excluding top and bottom value– Excluding any obvious outliers

Null Call Performance

• All system calls use the same mechanism to begin and end.

• If these mechanisms are slow, then the kernel will always be slow.

• This measures the overhead for all system calls.

• The measures the performance of very simple system calls.

Null Call Graph

Stat Performance

• Stat is the system call that returns file metadata.

• Stat is very common. Almost all file system oriented tasks will use stat.

• Stat speed is determined by the speed of the file system and the directory cache.

• This data is a combination, with lots of caching.

Page Fault Performance

• A Page fault occurs when the system needs to bring a VM page into RAM.

• Hard page faults go to disk, and speed is determined by disk speed.

• Soft page faults recover the page from some other part of RAM, and the speed is determined by the kernel and MMU hardware.

Mmap performance

• Mmap is used to alter the memory map of a process.

• Mmap is fundemental to how programs using shared libraries begin.

• Mmap is also used to establish a shared memory region between communicating processes.

Context Switching Performance

• This test uses rings of processes communicating via read/write.

• Simulates many situations in real life– Processes communicating with the X server.– Commands using pipes.

• This is not a networking test!

Signal handling Performance

• This measures the time needed to switch processes due to software interrupt.

• This is a stand-in for measuring the hardware interrupt switch time.

• Hardware interrupt time is critical for all high speed data throughput.

Signal Handling Analysis

On Thu, 26 Apr 2001, Linus Torvalds wrote:>I'll take a look at what the signal handling thing >is, although I suspect it's just the cleanup for >eventually getting thread-safe signals and> pthreads semantics..

That might be a small part of it, but from initial profiles it appears that the biggest chunk of it by far is just the new floating point format for supporting the full SSE information…

Performance Conclusions

• Linux is generally getting faster.• Extra lines of code are not slowing Linux

down.• Sometimes, algorithm changes cause a

>10x speed improvement.• No one knows for a user-level workload.• There is no easy record of algorithm

changes for Linux.

Being Slashdot-ed

• The research looked interesting.

• I want to maximize the experience for my students.

• I submit to slashdot.http://www.slashdot.org

• They accept!

Slashdot-ed, the day after

• Saw 122,089 hits from 42,645 unique IPs.• 99 different nations, including 39 Belarus

(39) Luxembourg (39), Iceland (76).• The UK had 2,264 hits.• Intro translated into French, Spanish,

German, Russian, Slovak, Italian.• Students absolutely loved it.• Received over 300 emails (not all nice).

Encouragement

• Research is fun!

• Research doesn’t need a national lab.

• Small efforts can also help.