1/19
Secretly Monopolizing the CPU Without Superuser
Privileges
Dan Tsafrir-IBM Watson, NY
Yoav Etsion-Hebrew University, Jerusalem
Dror Feitelson-Hebrew University, Jerusalem
2/19
In a nutshell
• We implement a “cheat” utility:
shell~/> cheat 90% program
(1) hostile aspect(2) concealment aspect
- ‘program’ would get 90% of the CPU cycles…
- …regardless on any background load
- ‘program’ appears as consuming 0%...
- …on some systems, when using tools like top, ps…
3/19
Unlike a rootkit
rootkitcheat
requires superuser access
any non-privileged user can easily do it
hostile-action (breaking in & installing
the rootkit)
concealment-capabilities (being able to secretly
use any computer resource)
concealment-capabilities (being able to secretly
use the CPU)
hostile action(being able to
monopolize the CPU)
4/19
The privileges-conflict axis
Attackers/defenders conflict often revolves around 1) Privileges of using resources, and 2) How long these privileges last
general
limited
cheatsecretly ctrl one resource
(can envision rootless rootkit)
obtain root & install rootkit: secretly ctrl all
resources
e.g. “cosmic rays attack” on JVMs [S&P
2003]privileges-conflict axis
5/19
Concealment: how?
rootkitcheat
requires superuser access
any non-privileged user can easily do it
hostile-action (breaking in & installing
the rootkit)
concealment-capabilities (being able to secretly
use any computer resource)
concealment-capabilities (being able to secretly
use the CPU)
hostile action(being able to
monopolize the CPU)
6/19
• Ticks = periodic hardware interrupts• OSs wake up HZ times a second:
• Since the 1960s (60Hz, like the grid)[Dijkstra; the “THE” system; CACM’68]
Operating System “ticks”
OSLinuxWindowsFreeBSD
2.22.42.6
HZ10010002501001000
every10ms1ms4ms10ms1ms
7/19
The tick-handler
• Tick-handler’s activities: 1) Deliver due alarm signals 2) CPU accounting 3) Initiating preemption (for multitasking) 4) ...
• Kernel’s “main loop”; its way to maintain control
8/19
Exampletick [1ms]
kernelnuclear simulator
movie player [display every 20ms]
n+1billrunsleep
n+2billrunsleep
…
n+19billrunsleep
n+20- bill - deliver alarm - switch
suspended
- display next frame- request alarm @ n+40- yield
switchrunsleepn+21billrunsleepn+22billrunseep
…
9/19
Concealment: how?
time
Run when OS not looking…(Not as easy as it sounds)
tick
n
tick
n
+1
tick
n
+2
tick
n
+3
cheater runs
cheater runs
cheater runs
10/19
Misaccounting: if we pull it off
time
The ‘honest’ process appears consuming 100%
tick
n
tick
n
+1
tick
n
+2
tick
n
+3
cheater runs
cheater runs
cheater runsh
on
est
hon
est
hon
est
hon
est
11/19
• The negative-feedback scheduling-principle - Running reduces priority to run more
• This principle guarantees 1) Fair allocation of CPU
2) Chronic sleepers (like emacs) get high priority
• Latter largely what makes editors responsive - And indeed, our cheater looks like an editor!
Monopolizing: how?
12/19
“Interactivity” makes it worse
• Turns out monopolizing is possible even without concealment capabilities…
• Scheduling for ‘interactivity’ raises ugly head: scheduling:traditionalrecent
workload:editors,
chronic sleepers
multimedia, soft realtime,
movie players, games
CPU demand:
lowhigh
prioritize by: sleep duration
(negative feedback)
sleep frequency(lots of deadlines)
13/19
Vulnerability spectrum
14/19
void cheat_attack( double fraction) {
work = fraction * cycles_per_tick();
nanosleep(0); // sync with ticktick_start = get_cycles();
while( 1 ) {now = get_cycles();
if( now - tick_start ≥ work ) {nanosleep(0); // avoid billtick_start = get_cycles();
}// do some short work here…
}}
The cheat algorithm
15/19
Evaluation
80%-cheater vs. honest-process
tick
tim
e [
ms]
time [sec]
tick
tim
e [
%]
cheaterhonestother
used CPU [ms]
bill
ed
CPU
[m
s]
16/19
‘Top’ snippet
PIDUSERPRSTAT%CPUTIMECOMMAND
5522dants21R99.30:07.79honest
5508dants16R0.30:00.04top
5246dants16S0.00:00.04csh
5259dants16S0.00:00.06csh
5509dants16S0.00:00.03bm-noklog.sh
…
5521dants15S0.00:00.00cheater
17/19
Cycle-accurate billingti
ck t
ime [
ms]
bill
ed
CPU
[m
s]
beforecheater
honest
time [sec] used CPU [ms]
after
tick
tim
e [
ms]
bill
ed
CPU
[m
s]honest
cheater
time [sec] used CPU [ms]
18/19
Curbing interactives preference
time [ms] used CPU [ms]b
illed
CPU
[m
s]
tick
tim
e [
ms]
cheater ( 50.02% )honest ( 49.97% )other ( 0.01% )
all the data zoom in
19/19
• Cheating allows unprivileged users to - seize CPU, often secretly
• To protect against cheating, one must 1) maintain accurate-enough info 2) incorporate info in scheduler (unlike Solaris, XP)
3) do it carefully (unlike Linux2.6, FreeBSD/ULE, XP)
• OS trend: prioritize based on sleep-frequency - plays straight to the hands of cheaters - probably fruitless - alternative: explicitly track relevant devices [NOSSDAV’04, TOMCCAP'06]
Conclusions
“Cheat” attack - Impact
• Linux scheduler:
Ingo Molnar, maintainer of the Linux Scheduler:
[ From the CFS-scheduler announcement people.redhat.com/mingo/cfs-scheduler/sched-design-CFS.txt ]
“Due to its design, the CFS scheduler is not prone to any of the 'attacks' that exist today'attacks' that exist today against the heuristics of the stock scheduler”
“Cheat” attack - Impact
• Amazon’s “Elastic Compute Cloud” (EC2)
– F. Zhou, M. Goel, P. Desnoyers, & R. Sundaram,“Scheduler vulnerabilities & coordinated attacks Scheduler vulnerabilities & coordinated attacks in cloud computingin cloud computing”, Northeastern U., Boston Technical Report, 2010
– Subverting Xen hypervisor with the cheat attack
– “Following the responsible disclosure model, we have reported this vulnerability to Amazon; they reported this vulnerability to Amazon; they have since implemented a fixhave since implemented a fix that we have tested and verified”
22/19
Backup…
23/19
Cheat accuracy
desired cheat rate [%]
ach
ieved
ch
eat
rate
[%
]
24/19
Effect of background load
all are honest one is 80%-cheater
number of competing processes
CPU
[%
]
all the othersone competitor
all the otherson 80% cheater
25/19
Running unmodified programs
Two approaches: 1) Cheat server 2) Binary instrumentation
26/19
The cheat-server protocol
27/19
Stealing a cluster with a P-III
cluster size [nodes; log scale]
CPU
[%
]
sum of 60%-cheaters (one per node)
sum of honest (10 per node)
28/19
Binary instrumentation
instrumentation granularity
slow
dow
n
instruction
blocktrace
function
selected function
29/19
Protecting against cheating: degrees of accuracy
• Two approaches 1) Solaris & Windows way
- Account for every kernel entry/exit (which e.g. slows down syscalls path)
2) BSD & Linux way
- Accounting by tick-handler only (inaccurate)
• Compromise - Accounting only upon a context-switch - Price: context-switch slower by 5%