Top Banner

Click here to load reader

consistency without borders

Jul 04, 2015

ReportDownload

Technology

palvaro

talk at SoCC'13

  • 1. consistency without borders Peter Alvaro, Peter Bailis, Neil Conway, Joseph M. Hellerstein UC Berkeley

2. The transaction concept DEBIT_CREDIT: BEGIN_TRANSACTION; GETMESSAGE; EXTRACTACCOUT_NUMBER,DELTA,TELLER,BRANCH FROMMESSAGE; FINDACCOUNT(ACCOUT_NUMBER)INDATABASE; IFNOT_FOUND|ACCOUNT_BALANCE+DELTA 0 (prayer) Write Read 13. Distributed consistency: staying relevant Is this an important problem? Is academia disconnected from reality? OK, what now? Goal: help programmers write correct applications. Today: some promising approaches 14. Case study: a graph T1 T2 T4 T10 T3 T6 T5 T9 T7 T8 T11 T12 T13 T14 15. Partitioned, for scalability T1 T2 T4 T10 T3 T6 T5 T9 T7 T8 T11 T12 T13 T14 16. Replicated, for availability T1 T2 T4 T10 T3 T6 T5 T9 T7 T8 T11 T12 T13 T14 T1 T2 T4 T10 T3 T6 T5 T9 T7 T8 T11 T12 T13 T14 17. Problem: deadlock detection Task: Identify strongly-connected components Waits-for graph T1 T2 T4 T10 T3 T6 T5 T9 T7 T8 T11 T12 T13 T14 18. Problem: garbage collection Task: Identify nodes not reachable from Root. Root Refers-to graph T1 T2 T4 T10 T3 T6 T5 T9 T7 T8 T11 T12 T13 T14 19. T1 T2 T4 T10 T3 T6 T5 T9 T7 T8 T11 T12 T13 T14 Correctness Deadlock detection Safety: No false positives- Liveness: Identify all deadlocks Garbage collection Safety: Never GC live memory! Liveness: GC all orphaned memory T1 T2 T4 T10 T3 T6 T5 T9 T7 T8 T11 T12 T13 T14 ParLLon Root 20. Consistency at the extremes Storage Object Flow Language Application Linearizable key-value store? Custom solutions? 21. Consistency at the extremes Storage Object Flow Language Application Linearizable key-value store? Custom solutions? Ecient Correct 22. Consistency across the stack 23. Object-level consistency Capture semantics of data structures that allow greater concurrency maintain guarantees (e.g. convergence) Storage Object Flow Language Application 24. Insert Read Convergent data structure (e.g., Set CRDT) Object-level consistency Insert Read Commutativity Associativity Idempotence Reordering Batching Retry/duplication Tolerant to 25. Application Convergent data structures Object-level consistency ? ? GC Assert: No live nodes are reclaimed Assert: Graph replicas converge 26. Flow-level consistency Storage Object Flow Language Application 27. Flow-level consistency Capture semantics of data in motion Asynchronous dataow model component properties system-wide guarantees Graph store Transaction manager Transitive closure Deadlock detector Confluent ConfluentConfluent 28. Flow-level consistency Order-insensitivity (conuence) outputset=f(inputset) {} {} = 29. Flow-level consistency Conuence is compositional outputset=fg(inputset) 30. Flow-level consistency Conuence is compositional outputset=fg(inputset) 31. Graph store Memory allocator Transitive closure Garbage collector Confluent Not Confluent Confluent Graph store Transaction manager Transitive closure Deadlock detector Confluent ConfluentConfluent Graph queries as dataow 32. Graph store Memory allocator Transitive closure Garbage collector Confluent Not Confluent Confluent Graph store Transaction manager Transitive closure Deadlock detector Confluent ConfluentConfluent Graph queries as dataow Conuent Coordinatehere 33. Language-level consistency DSLs for distributed programming? Capture consistency concerns in the type system Storage Object Flow Language Application 34. Language-level consistency CALM Theorem: Monotonic conuent Conservative, syntactic test for conuence 35. Language-level consistency Deadlock detector Garbage collector 36. Language-level consistency Deadlock detector Garbage collector nonmonotonic 37. Storage Object Flow Language Application correct reusable ecient intuitive Where weve been; where were headed 38. Storage Object Flow Language Application correct reusable ecient intuitive Where weve been; where were headed 39. Storage Object Flow Language Application correct reusable ecient intuitive Where weve been; where were headed 40. Remember Consistency is an application-level property Correctness and performance are compatible Meet programmers on their home turf Build bridges! 41. Queries?

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.