Knit 1 Knit: Component Composition for Systems Software Alastair Reid Alastair Reid , , Matthew Matthew Flatt, Leigh Stoller, Jay Flatt, Leigh Stoller, Jay Lepreau, Eric Eide Lepreau, Eric Eide University of Utah University of Utah
Dec 16, 2015
Knit 1
Knit: Component Composition for Systems Software
Alastair ReidAlastair Reid, , Matthew Flatt, Leigh Matthew Flatt, Leigh Stoller, Jay Lepreau, Eric EideStoller, Jay Lepreau, Eric Eide
University of UtahUniversity of Utah
Knit 2
Why Components?
Everyone is writing too much codeEveryone is writing too much code Not enough code reuseNot enough code reuse Hard to reconfigureHard to reconfigure Hard to understandHard to understand Hard to test/verifyHard to test/verify
Exceptions: Click, Scout, Ensemble, Fox, Exceptions: Click, Scout, Ensemble, Fox, MMLite, OSKit, …MMLite, OSKit, …
Knit 3
Why Not Components?
OverheadOverhead RuntimeRuntime Programmer timeProgrammer time
Advanced systems don’t work with CAdvanced systems don’t work with C Complex component interdependenciesComplex component interdependencies
Locking restrictionsLocking restrictions Top/bottom-halfTop/bottom-half Bootstrap sequenceBootstrap sequence
Knit 5
Key to Achieving Goal
Static configuration languageStatic configuration language
Enables error detectionEnables error detectionEnables optimizationEnables optimization
Knit 6
Target#1: The Utah OSKit [SOSP’97]
Approximately 500 components:Approximately 500 components:
Device drivers, bootstrap code, TCP/IP Device drivers, bootstrap code, TCP/IP stacks, filesystems, SNMP, etc. stacks, filesystems, SNMP, etc.
Doesn’t impose architectureDoesn’t impose architecture 101066 lines of code from Linux, FreeBSD, lines of code from Linux, FreeBSD,
NetBSD, Mach, Fluke, etc.NetBSD, Mach, Fluke, etc.
Knit 7
Outline
IntroductionIntroduction The Knit component modelThe Knit component model
Atomic unitsAtomic units Compound unitsCompound units Automatic InitializationAutomatic Initialization Detecting Configuration ErrorsDetecting Configuration Errors
Implementation and PerformanceImplementation and Performance Open issuesOpen issues
Knit 8
Atomic Units [PLDI’98]
serve_cgi
serve_web
int serve_web(…) { if (…) serve_cgi(…); else serve_file(…);}
serve_file
-Ioskit-DKERNEL-DHAVE_CONFIG
Knit 10
Initialization
x86
main
pthreads
VM init_x86();init_IDE();init_VM();init_threads();init_filesys();init_main();filesys
IDE
Knit 11
Initialization
x86
main
pthreads
VM init_x86();init_IDE();init_VM();init_threads();init_filesys();init_main();filesys
IDE
X
Knit 12
When Can We Break Cycles?
1.1. Component ‘contains’ subcomponentsComponent ‘contains’ subcomponents
2.2. No dependency between initializersNo dependency between initializers
VM threads locks
f_init f g g_init
Knit 13
Automatic Initialization
Knit generates initialization sequenceKnit generates initialization sequence Cycles are resolved by refining Cycles are resolved by refining
initialization dependencies in unitsinitialization dependencies in units
ExperienceExperience 5% of units need dependencies refined5% of units need dependencies refined Programmers find initialization a big winProgrammers find initialization a big win
Knit 21
Detecting Composition Errors
ethernet
panic
console
filesys
threads context(threads) <= ProcessContext
context(filesys) <= context(threads)
context(console) <= context(filesys)
context(panic) <= context(console)
NoContext <= context(ethernet)
ProcessContext < NoContext
Knit 22
Extensible Constraint System
Constraint system propagates properties Constraint system propagates properties through component interconnectionsthrough component interconnections Knit can detect global errorsKnit can detect global errors
Constraint system is extensibleConstraint system is extensible In context X, don’t do YIn context X, don’t do Y Type system for Modular IP Routers Type system for Modular IP Routers
(e.g., Click)(e.g., Click) ……
Knit 23
Knit
Supports C, assembly and object filesSupports C, assembly and object files Separates interconnections from codeSeparates interconnections from code Automatic initializationAutomatic initialization Extensible constraint systemExtensible constraint system Allows cyclic component dependenciesAllows cyclic component dependencies Allows multiple instances of componentsAllows multiple instances of components Text basedText based
Knit 24
Outline
IntroductionIntroduction The Knit component modelThe Knit component model Implementation and PerformanceImplementation and Performance Open issuesOpen issues
Knit 26
Performance
Component cost should not distort system Component cost should not distort system structurestructure
Reduce overhead by eliminating function Reduce overhead by eliminating function callscalls
Knit 27
Click and Clack
Click modular network router from MIT Click modular network router from MIT [SOSP’99][SOSP’99]
ClackClack Re-implementation of Click using KnitRe-implementation of Click using Knit Similar performance to ClickSimilar performance to Click
Many small componentsMany small components
Knit 28
Performance of Clack
100%
79%
65%
0%
25%
50%
75%
100%
Unoptimized Monolithic Optimized
Tim
e pe
r P
acke
t
Knit 29
Open Issues
Is Knit general purpose?Is Knit general purpose? Need more usersNeed more users Need more applicationsNeed more applications
Is the constraint system extensible enough?Is the constraint system extensible enough? Implicit linking vs. explicit linking?Implicit linking vs. explicit linking?
Knit 30
Conclusions
State of the art component system for CState of the art component system for C Targeted at systems codeTargeted at systems code
Automatic initializationAutomatic initialization Detects local and global errorsDetects local and global errors Low performance overheadLow performance overhead
Available ASAP: http://www.cs.utah.edu/flux/