This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
1. Have you heard Valgrind?2. Have you used Valgrind?3. Do you want to use Valgrind?4. Do you want to understand Valgrind?5. Do you want to fix Valgrind?6. Do you want to maintain Valgrind?7. Do you want to write Valgrind (from scratch)?
..........
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
......
.....
.....
.
Prologue: Audience
Use *BSD?
Write code?
Use (want) Valgrind?
Report problems?
Read Valgrind code?
Writeplatform dependent
code?
Writeplatform independent
code?
Yes
No
User
Yes
Yes
Yes
Yes
Yes
Yes
Advanced user
Valgrind user
Helpful Valgrind user
Luser
Advanced helpful Valgrind user
Valgrind porting developer
Valgrind architect
..........
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
......
.....
.....
.
Prologue: Goals (of this presentation)
1. Involve more people to help development2. Introduce design & explain internal (esp. syscall + execve
+ signal)3. Convince {Open,Net}BSD developers for necessary
changes
..........
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
......
.....
.....
.
Introduction: Authors' words
From ``Valgrind: A Framework for Heavyweight DynamicBinary Instrumentation'':
▶ A dynamic binary instrumentation (DBI) framework▶ Designed for building heavyweight dynamic binary
analysis (DBA) tools▶ ``Shadow values'' (register and memory)
..........
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
......
.....
.....
.
Introduction: Wikipedia
From Wikipedia (https://en.wikipedia.org/wiki/Valgrind)``Overview'':
Valgrind is in essence a virtual machine usingjust-in-time (JIT) compilation techniques, includingdynamic recompilation.
Lines:▶ 499 lines containing ``freebsd'' in *.[cS] files▶ 340 lines containing ``#if .*freebsd'' in *.[cS] files
..........
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
......
.....
.....
.
Development: Code: #ifdef's
OS and architecture conditional macros:▶ VGO_*: OS▶ VGA_*: Architecture▶ VGP_*: Platform (== Architecture + OS)
How many for FreeBSD:▶ VGO_freebsd: 196 lines▶ VGA_amd64: 53 lines▶ VGP_amd64_freebsd: 56 lines
..........
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
......
.....
.....
.
Development: Code: portability
From ``Porting Plans'' inhttp://valgrind.org/info/platforms.html:
...Unlike NetBSD or GCC, we are not interested inhaving Valgrind work on every platform in theknown universe: the maintenance burden is too high....
Portable, but not very portable.From ``Out Of Tree'':
x86/FreeBSD: Doug Rabson and others haved done afairly complete port of Valgrind 3.X. The FreeBSDporting team actively maintains the port. Snapshotsof the work in progress are at this FreeBSD page.
1: /* Even though we can't take a signal until the sigprocmask completes,start the range early.If eip is in the range [1,2), the syscall hasn't been started yet */
/* Set the signal mask which should be current during the syscall. *//* Save and restore all 5 arg regs round the call. This is easier
than figuring out the minimal set to save/restore. */
/* If rip==2, then the syscall was either just aboutto start, or was interrupted and the kernel wasrestarting it. */
2: syscall3: /* In the range [3, 4), the syscall result is in %rax,
but hasn't been committed to RAX. */
..........
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
......
.....
.....
.
/* stack contents: 3 words for syscall above, plus our prologue */setc 0(%rsp) /* stash returned carry flag */
movq -16(%rbp), %r11 /* r11 = VexGuestAMD64State * */movq %rax, OFFSET_amd64_RAX(%r11) /* save back to RAX */movq %rdx, OFFSET_amd64_RDX(%r11) /* save back to RDX */
/* save carry flag to VEX */xorq %rax, %raxmovb 0(%rsp), %almovq %rax, %rdi /* arg1 = new flag */movq %r11, %rsi /* arg2 = vex state */addq $24, %rsp /* remove syscall parameters */call LibVEX_GuestAMD64_put_rflag_c
4: /* Re-block signals. If eip is in [4,5), then the syscallis complete and we needn't worry about it. */
▶ Memory▶ Text, data, bss: ELF Program Header PT_LOAD entries▶ Stack: Arguments, environments, ``aux'' information (for
dynamic linker)▶ Registers
▶ Parameters to start code (crt or ld.so)▶ Per-process resources
▶ Timer, ... (what else?)
..........
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
......
.....
.....
.
Internal: execve(2): On *BSD
▶ Kernel parses ELF Program Header▶ Kernel maps text, data, bss, and stack▶ Kernel prepares stack context▶ Kernel prepares register context (``struct trapframe'')▶ Rest done by userland (start code (``crt'') or dynamic
linker (``ld.so''))
..........
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
......
.....
.....
.
0x400000
text + data + bss(PT_LOAD entries)
Stack
Address Space
Machine State(struct trapframe)
Registers
Flags Co-processor
Per-Process State(struct process/proc)
char **argv
char **env
auxinfo
arg / env strings
argc
RSP
..........
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
......
.....
.....
.
Internal: execve(2): On Valgrind
▶ ``initimg''▶ Basically simulates ``execve(2)''▶ Memory and registers▶ ELF Program Header parsing▶ Address space mapped by Valgrind (not kernel)
..........
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
......
.....
.....
.
VG_(ii_create_image)()
trap vector
Valgrind Text
Valgrind Stack
Kernel Text
syscall
subq $8, $rspandq $~15, $rsp
:
Client Text
Client Stack
ClientMachine State
argc, argv[], env[]auxinfo
argv/env strings
ps_strings
setup_client_stack()
VG_(ii_finalise_image)()
..........
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
......
.....
.....
.
Internal: Signal
▶ From Wikipedia: https://en.wikipedia.org/wiki/Unix_signalSignals are a limited form of inter-processcommunication used in Unix, Unix-like, and otherPOSIX-compliant operating systems.
▶ Signal action: calling registered functions when signaldelivered (kernel calling userland function!)
▶ Signal trampoline: small code fragment to execute signalhandler and/or return to kernel
▶ BSD signal code: Ancient code (no one wants to touch)Signal code is INTERESTING!!!!!
..........
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
......
.....
.....
.
Internal: Signal: Sync signal: On *BSD
Signal Handler
Text
Stack
mov %rcx, (%rax)
Signal Handler
Text
Stack
mov %rcx, (%rax)
Signal Trampoline
Signal Handler
Text
Stack
mov %rcx, (%rax)
Signal Trampoline
InternalSignal Stack Data
Signal Handler
Text
Stack
mov %rcx, (%rax)
Signal Trampoline
Signal Handler
Text
Stack
mov %rcx, (%rax)
Signal Trampoline
Signal Handler
Text
Stack
mov %rcx, (%rax)
Kernel createssignal context
and callssignal trampoline
Kernel restoresoriginal context
Signal trampolinereads stack dataand dispatches
handler
Handler finishesand returns back
to trampoline
Trampoline callssigreturn()
to exitssignal context
InternalSignal Stack Data
InternalSignal Stack Data
InternalSignal Stack Data
User code
Kernel sendsig Kernel sigreturn
User codeUser handlerUser trampoline User trampoline
..........
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
......
.....
.....
.
Internal: Signal: Trampoline: FreeBSD
sys/amd64/amd64/sigtramp.S:
NON_GPROF_ENTRY(sigcode)call *SIGF_HANDLER(%rsp) /* call signal handler */lea SIGF_UC(%rsp),%rdi /* get ucontext_t */pushq $0 /* junk to fake return addr. */movq $SYS_sigreturn,%raxsyscall /* enter kernel with args */
▶ procfs is heavily used▶ To retrieve memory mapping information, opened file
descriptors, ...▶ *BSD is basically moving away from that▶ Need to extend sysctl (KERN_PROC_VMMAP)
..........
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
......
.....
.....
.
Issues: Versioning
▶ Syscall and other ABIs (execve + start code, signal, ...)are hardcoded
▶ Can't handle multiple OS versions
..........
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
......
.....
.....
.
Issues: ioctl
▶ Valgrind needs to know all syscall arguments memoryread/write
▶ ``Broken'' (== impossible to become perfect) by design forcorner cases (user-defined ioctl in kernel modules)
..........
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
......
.....
.....
.
Issues: LD_PRELOAD
▶ OpenBSD prohibits LD_PRELOAD for suid'ed programs
..........
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
......
.....
.....
.
Summary
▶ External project forked from valgrind-freebsd onBitBucket
▶ Basically ported to OpenBSD/amd64 and NetBSD/amd64▶ Supported static and dynamic executables▶ Supported only basic system calls
▶ Need a few changes against base▶ Already found one (potential) bug in OpenBSD's ld.so▶ TODOs
▶ More ground work (execve(2)/fork(2), threading, ...)▶ More system calls (ioctl(2))▶ More architectures (i386, arm, ...)▶ Code clean-ups (esp. around procfs)
..........
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
......
.....
.....
.
ValgrindUser
ValgrindDeveloper
..........
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
......
.....
.....
.
ValgrindUser
ValgrindDeveloper
..........
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
.....
.....
......
.....
......
.....
.....
.
Future
▶ Maitainance is NOT fun (but I will do some)▶ Consider alternatives (clang + ASAN)