consistency without borders
Peter Alvaro, Peter Bailis, Neil Conway, Joseph M. Hellerstein UC Berkeley
The transaction concept
DEBIT_CREDIT: BEGIN_TRANSACTION; GET MESSAGE; EXTRACT ACCOUT_NUMBER, DELTA, TELLER, BRANCH FROM MESSAGE; FIND ACCOUNT(ACCOUT_NUMBER) IN DATA BASE; IF NOT_FOUND | ACCOUNT_BALANCE + DELTA < 0 THEN PUT NEGATIVE RESPONSE; ELSE DO; ACCOUNT_BALANCE = ACCOUNT_BALANCE + DELTA; POST HISTORY RECORD ON ACCOUNT (DELTA); CASH_DRAWER(TELLER) = CASH_DRAWER(TELLER) + DELTA; BRANCH_BALANCE(BRANCH) = BRANCH_BALANCE(BRANCH) + DELTA; PUT MESSAGE ('NEW BALANCE =' ACCOUNT_BALANCE); END; COMMIT;
The transaction concept
DEBIT_CREDIT: BEGIN_TRANSACTION; GET MESSAGE; EXTRACT ACCOUT_NUMBER, DELTA, TELLER, BRANCH FROM MESSAGE; FIND ACCOUNT(ACCOUT_NUMBER) IN DATA BASE; IF NOT_FOUND | ACCOUNT_BALANCE + DELTA < 0 THEN PUT NEGATIVE RESPONSE; ELSE DO; ACCOUNT_BALANCE = ACCOUNT_BALANCE + DELTA; POST HISTORY RECORD ON ACCOUNT (DELTA); CASH_DRAWER(TELLER) = CASH_DRAWER(TELLER) + DELTA; BRANCH_BALANCE(BRANCH) = BRANCH_BALANCE(BRANCH) + DELTA; PUT MESSAGE ('NEW BALANCE =' ACCOUNT_BALANCE); END; COMMIT;
An application-level contract
Write Read
Application
Opaque store
Transactions
Pervasive distribution
Pervasive distribution
partial failure
Pervasive distribution
partial failure
asynchrony
Pervasive distribution
CAP
partial failure
asynchrony
Research on consistency
Write Read
Application
Opaque store
Consistency models
R1(X=1) R2(X=1) W1(X=2) W2(X=0) W1(X=1) W1(Y=2) R2(Y=2) R2(X=0)
Research on consistency
Write Read
Application
Opaque store
Consistency models
Assert: balance > 0
causal? PRAM? delta? fork/join? red/blue? release? SC?
(translation)
R1(X=1) R2(X=1) W1(X=2) W2(X=0) W1(X=1) W1(Y=2) R2(Y=2) R2(X=0)
Meanwhile, in industry…
Application
Opaque store
Custom solutions
Write Read
Meanwhile, in industry…
Application
Opaque store
Custom solutions Assert: balance > 0
(prayer)
Write Read
Distributed consistency: staying relevant
• Is this an important problem? • Is academia disconnected from reality? • OK, what now?
Goal: help programmers write correct applications. Today: some promising approaches
Case study: a graph
T1T2
T4
T10
T3
T6
T5
T9
T7
T8
T11
T12
T13
T14
Partitioned, for scalability
T1T2
T4
T10
T3
T6
T5
T9
T7
T8
T11
T12
T13
T14
Replicated, for availability
T1T2
T4
T10
T3
T6
T5
T9
T7
T8
T11
T12
T13
T14
T1T2
T4
T10
T3
T6
T5
T9
T7
T8
T11
T12
T13
T14
Problem: deadlock detection
Task: Identify strongly-connected components
Waits-for graph
T1T2
T4
T10
T3
T6
T5
T9
T7
T8
T11
T12
T13
T14
Problem: garbage collection
Task: Identify nodes not reachable from Root.
Root
Refers-to graph
T1T2
T4
T10
T3
T6
T5
T9
T7
T8
T11
T12
T13
T14
T1T2
T4
T10
T3
T6
T5
T9
T7
T8
T11
T12
T13
T14
Correctness
Deadlock detection • Safety: No false positives-
• Liveness: Identify all deadlocks
Garbage collection • Safety: Never GC live memory!
• Liveness: GC all orphaned memory
T1T2
T4
T10
T3
T6
T5
T9
T7
T8
T11
T12
T13
T14
ParLLon
Root
Consistency at the extremes
StorageObjectFlow
LanguageApplication
Linearizable key-value store?
Custom solutions?
Consistency at the extremes
StorageObjectFlow
LanguageApplication
Linearizable key-value store?
Custom solutions?
Efficient Correct
Consistency across the stack
Object-level consistency
Capture semantics of data structures that • allow greater concurrency • maintain guarantees (e.g. convergence)
StorageObjectFlow
LanguageApplication
Insert Read
Convergent data structure (e.g., Set CRDT)
Object-level consistency
Insert Read
Commutativity Associativity Idempotence
Reordering Batching Retry/duplication
Tolerant to
Application
Convergent data structures
Object-level consistency
? ?
GC Assert: No live nodes are reclaimed
Assert: Graph replicas converge
Flow-level consistency
StorageObjectFlow
LanguageApplication
Flow-level consistency
Capture semantics of data in motion • Asynchronous dataflow model • component properties à system-wide guarantees
Graphstore
Transactionmanager
Transitiveclosure
Deadlockdetector
Con!uent Con!uentCon!uent
Flow-level consistency
Order-insensitivity (confluence)
output set = f(input set)
{ }
{ } =
Flow-level consistency
Confluence is compositional
output set = f � g(input set)
Flow-level consistency
Confluence is compositional
output set = f � g(input set)
Graphstore
Memoryallocator
Transitiveclosure
Garbagecollector
Con!uent Not
Con!uent
Con!uent
Graphstore
Transactionmanager
Transitiveclosure
Deadlockdetector
Con!uent Con!uentCon!uent
Graph queries as dataflow
Graphstore
Memoryallocator
Transitiveclosure
Garbagecollector
Con!uent Not
Con!uent
Con!uent
Graphstore
Transactionmanager
Transitiveclosure
Deadlockdetector
Con!uent Con!uentCon!uent
Graph queries as dataflow Confluent
Coordinate here
Language-level consistency
DSLs for distributed programming? • Capture consistency concerns in the
type system
StorageObjectFlow
LanguageApplication
Language-level consistency
CALM Theorem:
Monotonic à confluent
Conservative, syntactic test for confluence
Language-level consistency
Deadlock detector
Garbage collector
Language-level consistency
Deadlock detector
Garbage collector
nonmonotonic
StorageObjectFlow
LanguageApplication
correct reusable
efficient intuitive
Where we’ve been; where we’re headed
StorageObjectFlow
LanguageApplication
correct reusable
efficient intuitive
Where we’ve been; where we’re headed
StorageObjectFlow
LanguageApplication
correct reusable
efficient intuitive
Where we’ve been; where we’re headed
Remember
• Consistency is an application-level property
• Correctness and performance are compatible
• Meet programmers on their home turf
• Build bridges!
Queries?
