ESOP March 19, 2013 Mihai Budiu, Microsoft Research, Silicon Valley Joel Galenson, UC Berkeley Gordon Plotkin, University of Edinburgh
ESOP
March 19, 2013
Mihai Budiu, Microsoft Research, Silicon Valley
Joel Galenson, UC Berkeley
Gordon Plotkin, University of Edinburgh
• Motivating example
–Declarative parallel programming with LINQ and DryadLINQ
• Divide-and-conquer compilation
–Compilers and Partial Compilers
• Building real compilers – LINQ, DryadLINQ, and matrix computations
2
LINQ Summary
Input
3
.Net objects of type T
LINQ Summary
Where (filter)
Input
4
.Net objects of type T
LINQ Summary
Where (filter)
Select (map)
Input
5
.Net objects of type T
LINQ Summary
Where (filter)
Select (map)
GroupBy
Input
6
.Net objects of type T
LINQ Summary
Where (filter)
Select (map)
GroupBy
OrderBy (sort)
Input
7
.Net objects of type T
LINQ Summary
Where (filter)
Select (map)
GroupBy
OrderBy (sort)
Aggregate (fold)
Input
8
.Net objects of type T
LINQ Summary
Where (filter)
Select (map)
GroupBy
OrderBy (sort)
Aggregate (fold)
Join
Input
9
.Net objects of type T
Distributed Collections
10
Distributed Collections
11
Partition
Distributed Collection
Select (Map)
f( )=
12
Select (Map)
f( )=
Select(x => f(x))
13
Select (Map)
f( )=
Select(x => f(x))
+
14
Select (Map)
f( )=
Select(x => f(x))
+
+
15
Aggregate (Fold)
f( , )=
associative
16
Aggregate (Fold)
f( , )=
Aggregate((x,y) => f(x,y))
associative
17
Aggregate (Fold)
f( , )=
Aggregate((x,y) => f(x,y))
+
associative
18
Aggregate (Fold)
f( , )=
Aggregate((x,y) => f(x,y))
+
associative
+
19
Aggregate (Fold)
f( , )=
Aggregate((x,y) => f(x,y))
+
associative
+
20
GroupBy
GroupBy(x => K(x))
21
GroupBy
GroupBy(x => K(x))
o
22
GroupBy
GroupBy(x => K(x))
o
o
PartitionBy(x => K(x))
23
GroupBy
GroupBy(x => K(x))
o
o
o
PartitionBy(x => K(x))
GroupBy(x => K(x))
Equal under permutation
24
• Motivating example
–Declarative parallel programming with LINQ and DryadLINQ
• Divide-and-conquer compilation
–Compilers and Partial Compilers
• Building real compilers – LINQ, DryadLINQ, and matrix computations
25
Compilers
Compiler
S T
26
𝐶 ∶ source → target
Partial compilers
Child
Compiler
target source
source’ target’
Partial compiler
27
Partial compilers
Child
Compiler
target source
source’ target’
Partial compiler
28
𝑃𝐶 ∶ source → source′ × target′ → target
Partial compilers
Child
Compiler
target source
source’ target’
Partial compiler R
re
du
ce G
gen
erat
e
29
𝑃𝐶 ∶ source → source′ × target′ → target
Compiler Composition target source
source’ target’
PC
C
30
Compiler Composition target source
source’ target’
PC
C
31
PC’
Partial Compiler Composition
PC
32
target source
source’ target’
source’’ target’’
Composition Laws
33
Composition Laws
Id
34
Compilers as First-Class Objects
PC PC’ ⊗
Tensor: 𝑃𝐶 ⊗ 𝑃𝐶′ 35
Multiple Children
SEQ
C1 C2
S T
36
Multiple Children
SEQ
C1 C2
Example: 𝑆 = 𝑓 ∘ 𝑔 𝑇 = 𝐶1 𝑓 ∘ 𝐶2(𝑔)
S T
37
Star
38
C
*
source* target*
Star
39
C
*
source* target*
Star
40
C
*
source* target* … …
Conditional
if gpu
GPU CPU
gpu : source -> bool
41
Case
42
case
Select GroupBy Aggregate
• Motivating example
–Declarative parallel programming with LINQ and DryadLINQ
• Divide-and-conquer compilation
–Compilers and Partial Compilers
• Building real compilers – LINQ, DryadLINQ, and matrix computations
43
A LINQ Compiler
44
Select GroupBy Aggregate
A LINQ Compiler
45
case
Select GroupBy Aggregate
A LINQ Compiler
46
case
Select GroupBy Aggregate
*
A LINQ Compiler
47
case
Select GroupBy Aggregate
*
SEQ
A Multi-Core LINQ Compiler
48
case
Select GroupBy Aggregate
MC
*
SEQ
A Multi-Core LINQ Compiler
49
case
Select GroupBy Aggregate
MC
*
SEQ
MC
A Multi-Core LINQ Compiler
50
case
Select GroupBy Aggregate
MC
*
SEQ
MC
S
T
A Multi-Core LINQ Compiler
51
case
Select GroupBy Aggregate
MC
*
SEQ
MC
S
o
o
T
T T
A Distributed, Multi-Core
LINQ Compiler
52
case
Select GroupBy Aggregate
MC
*
Cluster
SEQ
A Distributed, Multi-Core
LINQ Compiler
53
case
Select GroupBy Aggregate
MC
*
Cluster
SEQ
Cluster
A Distributed, Multi-Core
LINQ Compiler
54
case
Select GroupBy Aggregate
MC
*
Cluster
SEQ
Cluster
S
T
A Distributed, Multi-Core
LINQ Compiler
55
case
Select GroupBy Aggregate
MC
*
Cluster
SEQ
Cluster
S
o
o
T
T T
A Distributed Matrix Compiler
56
*
A Distributed Matrix Compiler
57
Tile Tile *
Cluster
A Distributed Matrix Compiler
58
Matrix
*
SEQ
Tile
Tile Tile *
Cluster
A Distributed Matrix Compiler
59
Matrix
*
SEQ
case
+ * …
Tile Tile *
M1.Times(M2).Plus(M3)
60
M1
M2
M3
*
+
Join. GroupBy.
Select( Aggregate) Concat
GroupBy. Select(
Aggregate)
+
PCTILE
PCTILE
PCTILE
CLINQ
CLINQ
*
PCMATRIX
+
PCMATRIX
PCSEQ
M1.Times(M2).Plus(M3)
61
M1
M2
M3
*
+
Join. GroupBy.
Select( Aggregate) Concat
GroupBy. Select(
Aggregate)
+
PCTILE
PCTILE
PCTILE
CLINQ
CLINQ
*
PCMATRIX
+
PCMATRIX
PCSEQ
M1.Times(M2).Plus(M3)
62
M1
M2
M3
*
+
Join. GroupBy.
Select( Aggregate) Concat
GroupBy. Select(
Aggregate)
+
PCTILE
PCTILE
PCTILE
CLINQ
CLINQ
PCMATRIX
PCMATRIX
PCSEQ
M1.Times(M2).Plus(M3)
63
Partial Compiler Correctness
64
if gpu
GPU CPU
Partial Compiler Correctness
65
𝑔𝑝𝑢 𝑆 𝐶𝐺𝑃𝑈 {¬𝑔𝑝𝑢 𝑆 }𝐶𝐶𝑃𝑈
𝑇 𝐼𝐹 𝑔𝑝𝑢 𝑇𝐻𝐸𝑁 𝐶𝐺𝑃𝑈 𝑒𝑙𝑠𝑒 𝐶𝐶𝑃𝑈 if gpu
GPU CPU
Partial Compiler Correctness
66
𝑔𝑝𝑢 𝑆 𝐶𝐺𝑃𝑈 {¬𝑔𝑝𝑢 𝑆 }𝐶𝐶𝑃𝑈
𝑇 𝐼𝐹 𝑔𝑝𝑢 𝑇𝐻𝐸𝑁 𝐶𝐺𝑃𝑈 𝑒𝑙𝑠𝑒 𝐶𝐶𝑃𝑈
Totally correct compiler from partially correct parts!
if gpu
GPU CPU
Related Work
• Dialectica category
– Inspired partial compilers and their operators.
• Milner’s tactics
– Partial compilers are a typed form of tactics.
• Multistage compilers
– Fit as a composition of unary partial compilers.
• Federated databases, cooperating analyses
– Interesting applications.
67
Thank you
68
Backup slides
69
Homomorphisms
70
ℎ ∶ 𝐴 → 𝐵 ℎ 𝑎1 +𝐴 𝑎2 = ℎ 𝑎1 +𝐵 ℎ(𝑎2)
collections
concatenation
concatenation
Nested Parallelism
71
Machine
Nested Parallelism
72
Machine Core
Correctness Definition
73
⊨ ⊆ 𝑡𝑎𝑟𝑔𝑒𝑡 × 𝑠𝑜𝑢𝑟𝑐𝑒 𝑇 ⊨ 𝑆
T implements the meaning of S
𝐶 ∶ 𝑠𝑜𝑢𝑟𝑐𝑒 → 𝑡𝑎𝑟𝑔𝑒𝑡 𝐶 𝑖𝑠 𝑐𝑜𝑟𝑟𝑒𝑐𝑡 𝑤. 𝑟𝑒𝑠𝑝. 𝑡𝑜 ⊨ iff
∀𝑆 ∈ 𝑠𝑜𝑢𝑟𝑐𝑒. 𝐶 𝑆 ⊨ 𝑆
C
T S
Partial Compiler Correctness
74
⊨ ⊆ 𝑠𝑜𝑢𝑟𝑐𝑒 × 𝑡𝑎𝑟𝑔𝑒𝑡 ⊨′ ⊆ 𝑠𝑜𝑢𝑟𝑐𝑒′ × 𝑡𝑎𝑟𝑔𝑒𝑡′
PC is correct iff ∀𝑆, 𝑇′. 𝑇′ ⊨′ 𝑆′ ⇒ 𝑇 ⊨ 𝑆
T S
T’ S’
Correctness Theorems
75
PC, C correct ⇒ correct
𝑃𝐶 ⊗ 𝑃𝐶′ correct
𝐶∗ correct
etc.
Partial Correctness
76
Correct only for some programs
Correct only for some programs
Generated sub-programs satisfy this predicate
Partial Correctness
77
Correct only for some programs
Correct only for some programs
Generated sub-programs satisfy this predicate
{Precondition} {Postcondition}
Functor
78
functor
CPU
Trans Id
Id : target -> target’ Trans : source -> source’
Iteration
79
iteration
Opt
A Distributed Matrix Compiler
80
Matrix -> sets of tiles
Matrix operations Distributed LINQ
Tile operations LINQ
Staged Compilers
81