SQLite with a Fine-Toothed Comb

SQLitewithaFine-ToothedComb

JohnRegehr

Trust-in-So1/UniversityofUtah

Feasiblestatesforasystemwecareabout

Feasiblestatesforasystemwecareabout

Someexecu<onreachesthisstate

Noexecu<onreachesthisstate

Ini<alstate

Feasiblestates

Figuringoutwhetheranarbitrarystateisfeasibleisvery,veryhard

Feasiblestates

Feasiblestates

Erroneousstates

Feasiblestates

Erroneousstates

BUG!!!

Verifica<on

Verifica<on

Verifica<on

Alarm

Alarm

Alarm

Alarm

Alarm

Alarm

Tes<ng

Tes<ng

Tes<ng

Tes<ng

Tes<ng

AHA!

•  Tes8ngisunsa8sfyingbecauseitgivesnoguarantees–  Inprac8ce,tes8ngalmostinvariablymissescri8calbugs

– Evenmicroprocessorsandrocketsshipwithnastybugs

However,italwaysmakessensetodotes8ngfirst,verifica8onsecond•  Ofcourseweneedtobecon8nuouslytes8ngourso1wareanyway

•  Findingbugsduringverifica8onmakesverifica8onmoredifficult– Wewantverifica8ontobeaboutprovingabsenceofbugs,notaboutfindingbugs

•  8s-interpreterletsusdetectawidevarietyofverysubtleundefinedbehaviors(UBs)inCcodeasasideeffectofnormaltes8ng

AnundefinedbehaviorinCandC++(andotherlanguages)isaprogramerrorthat–  Isnotcaughtbythecompilerorrun8melibrary–  Isassumedtonothappenbythecompiler–  InvalidatesallguaranteesmadebythecompilerBasicallyallnon-trivialCandC++programsexecuteundefinedbehaviors– Thus,accordingtothestandards,almostallCandC++programsaremeaningless

–  Including,forexample,mostoftheSPECCPU2006benchmarks

•  Thisfunc8onexecutesundefinedbehavior: int foo(int x, int y) { return (x + y) >> 32; }

•  Thisfunc8onexecutesundefinedbehavior: int foo(int x, int y) { return (x + y) >> 32; }

LatestversionofLLVMemits:foo: retq

•  Mostsafety-cri8calandsecuritycri8calso1wareiswriZeninCandC++

•  Undefinedbehaviorisahugeproblem– Responsibleforalargefrac8onofmajorsecurityproblemsoverthelast20years

•  Thesolu8onistools– Sta8canalysistofindbugsatcompile8me– Dynamicanalysistofindbugsatrun8me

AllUBs

UBsfoundby<s-interpreter

UBsfoundbyASanorValgrind

UBsfoundbyUBSan

varargsbugs

comparisonsofunrelatedpointers

uses(notdereferences)ofinvalidpointers signedinteger

overflowsOOBarrayaccesses

viola<onsofstrictaliasing

infiniteloopsw/osideeffects

doublefrees,usesaRerfree

unsequencedvariableaccesses

We’vebeenapplying8s-interpretertowidelyused,security-cri8calopensourcelibraries•  Crypto– PolarSSL,OpenSSL,LibreSSL,s2n

•  Fileprocessing–  libjpeg,libpng,libwebp,bzip,zlib

•  Databases– SQLite

Wheredowegettestcases?•  Testsuites•  afl-fuzz

SQLite•  OpensourceembeddedSQLdatabase•  ~113,000linesofC•  MostwidelydeployedSQLdatabase(probablybymul8pleordersofmagnitude)

•  Oneofthemostwidelydeployedso1warepackagesperiod– Mostphones,webbrowserinstances,smartTVs,settopboxescontainatleastoneinstance

•  hZps://www.sqlite.org

SQLiteisextensivelytested•  TestcasesarewriZenbyhand–  100%MC/DCcoverage!–  Everyentryandexitpointisinvoked–  Everydecisiontakeseveryoutcome–  Everycondi8oninadecisiontakeseveryoutcome–  Everycondi8oninadecisionisshowntoindependentlyaffecttheoutcomeofthedecision

•  Testcasesaregeneratedautoma8callybyfuzzers•  hZps://www.sqlite.org/tes8ng.html•  Execu8onsareexaminedbycheckingtoolssuchasValgrind

ArethereproblemsinSQLitele1forustofind?

Libraryfunc8onssuchasmemcpy()andmemset()assumethattheirpointerargumentsarenon-null•  SQLitesome8mescallsthesefunc8onswithnullarguments

void foo(char *p1, char *p2, size_t n) { memcpy(p1, p2, n); if (!p1) error_handler(); }

Libraryfunc8onssuchasmemcpy()andmemset()assumethattheirpointerargumentsarenon-null•  SQLitesome8mescallsthesefunc8onswithnullarguments

void foo(char *p1, char *p2, size_t n) { memcpy(p1, p2, n); if (!p1) error_handler(); }

CodegeneratedbyGCC:foo: jmp memcpy

int sqlite3_config(int op, ...) { … var1 = va_arg(ap, void *); var2 = va_arg(ap, void *); … }

OKtocalllikethis? sqlite3_config(CONFIG_LOG, 0, pLog);

int sqlite3_config(int op, ...) { … var1 = va_arg(ap, void *); var2 = va_arg(ap, void *); … }

Correctcall: sqlite3_config(CONFIG_LOG, (void *)0, pLog);

Howcanthiskindofbuggoundetected?

int sqlite3_config(int op, ...) { … var1 = va_arg(ap, void *); var2 = va_arg(ap, void *); … }

Correctcall: sqlite3_config(CONFIG_LOG, (void *)0, pLog);

Howcanthiskindofbuggoundetected?

Onx86:•  intandpointerarethesamesize•  Integer0andnullpointerhavethesamerepresenta8on

•  Noproblem!Onx86-64:•  inthassize4andpointerhassize8•  Firstsixintegerargumentsarepassedinregisters•  Noproblem!Onotherplanorms,memorycorrup8onispossible

•  Manyoccurrencesofintegerzerovaluesbeingpassedasnullpointers

•  Also,afewotherbugssuchasmoreargumentsbeingpoppedthanpushed

•  Arevarargsbugscommon?– Wedon’tknow– Bugsincallstovariadicstandardlibraryfunc8onsarecaughtbycustomcompilerwarnings

– Bugsinuser-wriZenvariadiccodegetnocheckingwhatsoever

Cdoesnotini8alizefunc8on-scopedvariablesValgrindtracksini8aliza8onatbitlevel,allowingdetec8onofaccessestounini8alizedstorage•  ButValgrindanalyzescompiledcode•  Thecompilercanhideerrors,forexamplebyreusingstackmemorythatwasalreadyini8alized

tis-interpreteralwaysfindsthesebugs–  IncludingseveralinSQLite

int dummy; some sort of loop { ... // we don't care about function()’s // return value (but its other // callers might) dummy += function(); ... } // dummy is not used again

ApointerinCbecomesillegaltouseoncethestoragetowhichitpointsisfreed•  Wefoundmanyloca8onswhereSQLitefreesmemoryandthencon8nuestousetheinvalidpointers

req1_malloc02_alignment(p, z); sqlite3_realloc(z, 0); th3testCheckTrue(p, z!=0);

Crea8ngapointeraheadoformorethanoneelementpasttheendofablockofstorageisillegalinCint a[10]; int *p1 = &a[-1]; // illegal int *p2 = &a[9]; // pointer to last element int *p3 = &a[10]; // OK (one past the end) int *p4 = &a[11]; // illegal

SQLitecomputedillegalpointers…•  Onpurpose:systema8cuseofpointerstoarray[-1]– 1-basedarrayindexingw/owas8ngRAM

•  Accidentally,aspartofinputvalida8on– ThiserrorisseeninalmostallCcode

Resultoftes8ngSQLiteusing8s-interpreter:•  Manybugsfixed•  Developersarenowmoreawareofsubtle8esoftheCstandard– Theyhadbeenwri8ng“1990sCcode”whichignoresmanyundefinedbehaviors

•  TheClanguageisfullofsubtleundefinedbehaviors– Somearedirectlyharmful– OthersmaZerbecausecompilersassumetheywillnothappen

•  8s-interpretermakestes8ngworkbeZerbyusingexis8ngtestcasestofindthesebugs

•  Tes8ngusing8s-interpreterisaveryusefulpreludetoformalverifica8on

•  8s-interpreterisopensource– hZp://trust-in-so1.com/8s-interpreter/