Compiler Construction Overview & Lessons Learned from Teaching a class at UW Jim Hogg Program Manager - C++ Compiler Team - Microsoft October 2014 L-1
Dec 23, 2015
Compiler Construction
Overview & Lessons Learned from Teaching a class at UW
Jim HoggProgram Manager - C++ Compiler Team - Microsoft
October 2014
L-1
H-2
What is this Talk About?
• I taught a class in Compilers, earlier this year, at UW
• Part of UW's "Professional Masters Program"
• I'll describe:
• The Course - content, organization• "Inside a Compiler" (at a gallop)• Lessons Learned
H-3
UW's "Professional Master's Program
For "fully-employed professionals"
Part-time (1 evening per week)
Average 2.5 years to reach Masters degree
~ $14k per year tuition fees
Average student: 5 years experience; GRE 83%; GPA 3.5
"The best computer science department you've never heard of" (NY Times)
In recent years, ~50% of attendees employed at Microsoft
H-4
PMP - Classes
1. Compiler Construction2. Principles of Software
Engineering3. Programming Languages4. Concurrency5. Human Computer
Interaction6. Applied Algorithms7. Parallel Computations8. Database Management
Systems9. Transaction Processing10.Data Mining11.Computer Architecture12.Computer Operating
Systems13.Current Trends in Computer
Graphics14.Network Systems
15.Design & Implementation of Digital Systems
16.Applications of Artificial Intelligence
17.Computer Vision18.Distributed Systems19.Computer Animation
Production20.Computational Biology21.Practical Aspects of Modern
Cryptograph22.Complexity Theory23.Alternative Programming
Paradigms24.Software Entrepreneurship25.Business Basics for CS
Professionals
H-6
Course Admin
10 lectures. One each week. Tuesday evenings 6:30-9:30
Broadcast from Microsoft Campus (Building 99)
Recorded at: http://courses.cs.washington.edu/courses/csep501/14sp/
Homework, Tutorials, Project, Final ExamSet of slides from previous years (Hal Perkins)
Teaching Assistant (Nat Mote)
Jim Hogg
Part-time Job: Visiting Lecturer at UW
Day Job: Program Manager in Microsoft’s C++ Compiler Team
Backend – optimizations & codegen
Previous Software Computational Physics; Operations Research; Oil Exploration Operating Systems; Device Drivers Languages & Compilers
Previous Hardware IBM; Cray; Xerox; CDC; DEC-10; VAX; Alpha; PC
Spring 2014 Jim Hogg - UW - CSE - P501 A-9
Who Am I?
Spring 2014 Jim Hogg - UW - CSE - P501 A-10
Execute this:
How? Computers only know 1’s and 0’s
55 8b . . .
int p = 0;int k = 0;while (k < length) {
if (a[k] > 0) p++;k++;
}
Compilers, from 10,000 feet
55 push ebp8b ec mov ebp, esp83 ec 58 sub esp, 88c7 45 f8 00 00 00 00 mov _p$[ebp], 0c7 45 fc 00 00 00 00 mov _k$[ebp], 0$LN3:83 7d fc 14 cmp _k$[ebp], 207d 1e jge $LN48b 45 fc mov eax, _k$[ebp]83 7c 85 a8 00 cmp _a$[ebp+eax*4], 0
. . . . .
Spring 2014 Jim Hogg - UW - CSE - P501 A-11
x86 Target Code
Spring 2014 Jim Hogg - UW - CSE - P501 A-12
Be a better programmer Insight into languages, compilers, and hardware What’s all that stuff in the debugger?
Compiler techniques are everywhere Little languages, verifiers, Lint, query languages, Verilog,
Mathematica Draws from many corners of CS
Finite automata, regex, grammars, graphs Links to Hardware
ISA, pipeline, multi-issue, cache, SIMD, multi-core, memory consistency
Jobs available! http://www.compilerjobs.com/
Why Study Compilers?
Jim Hogg - UW - CSE - P501 L-13
1966 Alan Perlis
1972 Edsger Dijkstra
1974 Donald Knuth
1976 Rabin & Scott
1977 John Backus
1978 Bob Floyd
1979 Ken Iverson
1980 Tony Hoare
1984 Niklaus Wirth
1987 John Cocke
1991 Robin Milner
2001 Dahl & Nygaard
2003 Alan Kay
2005 Peter Naur
2006 Fran Allen
2008 Barbara Liskov
Spring 2014
Compiler-related Turing Awards
Spring 2014 Jim Hogg - UW - CSE - P501 A-14
Structure of a Compiler, approximately Front end: analyze
Read source program; understand its structure and meaning
Specific to the source language used
Back end: synthesize (well, partly) Generate equivalent target language program Mostly unaware of the source language use
Source TargetFront End Back End
Compilers, from 1,000 feet
Spring 2014 Jim Hogg - UW - CSE - P501 A-15
01-Apr Tue Class 1 – Overview, Regex07-Apr Mon Homework 1 – regex08-Apr Tue Class 2 – grammar, LR(1)14-Apr Mon Homework 2 - grammars15-Apr Tue Class 3 – LL(1), ASTs, IR21-Apr Mon Project 1 - Scanner21-Apr Mon Homework 3 - grammars22-Apr Tue Class 4 – Semantics, x8628-Apr Mon Project 2 – Parser, ASTs29-Apr Tue Class 5 – Codeshape06-May Tue Class 6 – Optimizations, Dataflow13-May Tue Class 7 – Loops, SSA19-May Mon Project 3 – Semantics, Symbol Table20-May Tue Class 8 – Instruction Selection, Scheduling, Reg alloc27-May Tue Class 9 – Calling Conventions28-May Wed Exam03-Jun Tue Class 10 – Inlining, Multi-thread, GC09-Jun Mon Project 4 – CodeGen10-Jun Tue Project 5 - Report
Calendar
A-16
Compilers: Principles, Techniques and Tools Aho, Lam, Sethi, Ullman; 2e; 2011; “The Dragon Book” A Classic
Modern Compiler Implementation in Java Appel; 2e; 2013 Where the project comes from; good, but tough, text
Engineering a Compiler Cooper & Torczon; 2e; 2011 Solid; understandable; practical advice from the coal-face
Optimizing Compilers for Modern Architectures Allen & Kennedy; 2001 Good on SIMD & multi-core
Books
Spring 2014 Jim Hogg - UW - CSE - P501 A-17
Advanced Compiler Design & Implementation Muchnick; 1e; 1997 Detailed optimization text; magnum opus
Compiler Construction Wirth; 1996 “A refreshing antidote to heavy theoretical tomes” Now free, from: http://www.ethoberon.ethz.ch/WirthPubl/CBEAll.pdf
Programming Language Processors in Java Watt & Brown; 2000 Pragmatic; lots of code (but no LR parsing)
More Books
1801 - Joseph Marie Jacquard uses punch cards to instruct a loom to weave "hello, world" into a tapestry. Redditers of the time are not impressed due to the lack of tail call recursion, concurrency, or proper capitalization
L-18
1940s - Various "computers" are "programmed" using direct wiring and switches. Engineers do this in order to avoid the tabs vs spaces debate1970 - Niklaus Wirth creates Pascal, a procedural language. Critics immediately denounce Pascal because it uses "x := x + y" syntax instead of the more familiar C-like "x = x + y". This criticism happens in spite of the fact that C has not yet been invented.1983 - Bjarne Stroustrup bolts everything he's ever heard of onto C to create C++. The resulting language is so complex that programs must be sent to the future to be compiled by the Skynet artificial intelligence. Build times suffer.
http://james-iry.blogspot.com/2009/05/brief-incomplete-and-mostly-wrong.html
Best way to learn about compilers is to build one! Course project
MiniJava compiler: classes, objects, inheritance Built using Java Build scanner using JFlex; and parser using CUP Optionally use an IDE (Eclipse, etc) Optionally use a source code repository (SVN, etc) Generate executable x86 code (via assembler) & run it Complete in steps through the quarter
See MiniJava page at: http://www.cambridge.org/resources/052182060X/
Spring 2014 Jim Hogg - UW - CSE - P501 A-19
The Project
Code Flow (for Windows)
Spring 2014M-20
parser.cup
CUP
parser.java
sym.java
scanner.jflex
JFlex scanner.java
javac
mjc.java
ast
visitor
. . .
mjc.class
prog.java
prog.asm
ML
prog.obj
boot.c
CL
boot.obj
link
prog.exe
Spring 2014 Jim Hogg - UW - CSE - P501 H-22
Source TargetFront End Back End
Scan
chars
tokens
AST
IR
‘Middle End’
Optimize
Select Instructions
Parse
Semantics
Allocate Registers
Emit
Machine Code
IR
IR
IR
IR
IR
Convert
AST
AST
AST = Abstract Syntax Tree
IR = Intermediate Representation
Reminder: a token is . . .
Spring 2014 Jim Hogg - UW - CSE P501 A-23
class C { public int fac(int n) { // factorial int nn; if (n < 1) nn = 1; else nn = n * this.fac(n-1); return nn; }}
class∙C∙{◊∙∙public∙int∙fac(int∙n)∙{∙∙//∙factorial◊∙∙∙∙int∙nn;◊∙∙∙∙if(n∙<∙1)◊∙∙∙∙∙∙nn∙=∙1;◊∙∙∙∙else◊∙∙∙∙nn∙=∙n∙*∙(this.fac(n-1));◊∙∙∙∙return∙nn;◊∙∙}◊}
Key for Char Stream:
◊ newline \n∙ space
CLASS ID:C LBRACE PUBLIC INT ID:fac LPAREN INT ID:n RPAREN LBRACE INT ID:nn SEMI IF LPAREN ID:n LT ILIT:1 RPAREN ID:nn EQ ILIT:1 ELSE ID:nn EQ ID:n TIMES LPAREN ID:this DOT ID:fac LPAREN ID:n MINUS ILIT:1 RPAREN RPAREN SEMI RETURN ID:nn SEMI RBRACE RBRACE
Token Spotting
Spring 2014 Jim Hogg - UW - CSE P501 B-24
if(a<=3)++grades[1]; // what are the tokens? (no spaces)
public int fac(int n) { // what are the tokens? (need spaces?)
Counter-example: fixed-format FORTRAN:
DO 50 I = 1,99 // DO loopDO 50 I = 1.2 // assignment: DO50I = 1.2
Spring 2014 Jim Hogg - UW - CSE - P501 H-25
Source TargetFront End Back End
Scan
chars
tokens
AST
IR
‘Middle End’
Optimize
Select Instructions
Parse
Semantics
Allocate Registers
Emit
Machine Code
IR
IR
IR
IR
IR
Convert
AST
AST
AST = Abstract Syntax Tree
IR = Intermediate Representation
Valid Tokens != Valid Program
Spring 2014 Jim Hogg - UW - CSE - P501 C-26
So a MiniJava Scanner would happily accept the following program:
int ; = true { while ( x < true * if { or 123 ) goto count_99
We rely on a MiniJava Parser to reject this kind of gibberish
MiniJava includes the following tokens (among many others):
class int [ ( . true < this ) + * ; while = if id ilit ! / new {
But how do we specify what makes a valid MiniJava program?
Grammar for the Hokum Language
1. Prog Stm ; Prog | Stm2. Stm AsStm | IfStm 3. AsStm Var = Exp4. IfStm if Exp then AsStm5. VorC Var | Const6. Exp VorC | VorC + VorC7. Var [a-z]8. Const [0-9]
Spring 2014 Jim Hogg - UW - CSE - P501 C-27
• Context-Free Grammar ~ CFG ~ Grammar ~ Backus-Naur Form ~ BNF
• Productions, or Rules• Terminals & Non-Terminals; Start (Symbol)• Multiple languages present in the description
Context-Free Grammars (CFG)
Example Hokum Programs
Prog Stm ; Prog | StmStm AsStm | IfStm AsStm Var = ExpIfStm if Exp then AsStmVorC Var | ConstExp VorC | VorC + VorCVar [a-z]Const [0-9]
Spring 2014 Jim Hogg - UW - CSE - P501 C-28
a = 1; b = a + 4z = 1if b + 3 then z = 2
a = x < 20b = a + 4 + 5 ;z = 1 ;if (a == 33) z < 2 ;
Hokum BNF Grammar
But how do we know which programs are legal or illegal, in Hokum?
Legal
Illegal
Derivation
Prog => Stm ; Prog=> AsStm ; Prog=> Var = Exp ; Prog=> a = Exp ; Prog=> a = VorC ; Prog=> a = Const ; Prog=> a = 1 ; Prog=> a = 1 ; Stm=> a = 1 ; IfStm=> a = 1 ; if Exp then AsStm=> a = 1 ; if VorC + VorC then AsStm=> a = 1 ; if Var + VorC then AsStm
Spring 2014 Jim Hogg - UW - CSE - P501 C-29
=> a = 1 ; if a + VorC then AsStm=> a = 1 ; if a + Const then AsStm=> a = 1 ; if a + 1 then AsStm=> a = 1 ; if a + 1 then Var = Exp=> a = 1 ; if a + 1 then b = Exp=> a = 1 ; if a + 1 then b = VorC=> a = 1 ; if a + 1 then b = Const=> a = 1 ; if a + 1 then b = 2
=> versus Leftmost, rightmost, middlemost Sentential Form & Sentence What is a Context-Sensitive
Grammar?
Prog Stm ; Prog | StmStm AsStm | IfStm AsStm Var = ExpIfStm if Exp then AsStmVorC Var | ConstExp VorC | VorC + VorCVar [a-z]Const [0-9]
Parse TreeProg
Stm
1
Spring 2014
; Prog
AsStm
= Exp
Var
a VorC
Const
Stm
IfStm
thenExp
if AsStm
+ VorCVorC
Var
a 1
Const
2
= Exp
Var
b VorC
Const
Prog Stm ; Prog | StmStm AsStm | IfStm AsStm Var = ExpIfStm if Exp then AsStmVorC Var | ConstExp VorC | VorC + VorCVar [a-z]Const [0-9]
Jim Hogg - UW - CSE - P501 C-30
Junk Nodes in the Parse TreeProg
Stm
1
Spring 2014
; Prog
AsStm
= Exp
Var
a VorC
Const
Stm
IfStm
thenExp
if AsStm
+ VorCVorC
Var
a 1
Const
2
= Exp
Var
b VorC
Const
Jim Hogg - UW - CSE - P501 C-31
AST (Abstract Syntax Tree)
Prog
Spring 2014
=
Var:a Const:1
IfStm
+
Var:a Const:1
=
Var:b Const:2
Jim Hogg - UW - CSE - P501 C-32
Spring 2014 Jim Hogg - UW - CSE - P501 H-33
Source TargetFront End Back End
Scan
chars
tokens
AST
IR
‘Middle End’
Optimize
Select Instructions
Parse
Semantics
Allocate Registers
Emit
Machine Code
IR
IR
IR
IR
IR
Convert
AST
AST
AST = Abstract Syntax Tree
IR = Intermediate Representation
Spring 2014 Jim Hogg - UW - CSE - P501 I-34
Beyond Syntax - Semantic Checks
There is a level of correctness not captured by a CFG:
Has a variable been declared before it is used? Are types consistent in an expression? In the assignment x=y, is y assignable to x? Does a method call have right number and types of
parameters? In a selector p.q, is q a method or field of object p? Is variable x guaranteed to be initialized before it is used? Could p be null when p.q is executed? Etc
Spring 2014 Jim Hogg - UW - CSE - P501 I-35
We can specify Java micro-syntax with a few dozen regex Then find a tool (JFlex) to create a scanner from these regex
We can specify Java syntax with a few pages of BNF Then find a tool (CUP) to create a parser from that BNF
What about the huge collection of constraint checks? (760 pages in the Java Language Reference Manual)
Attribute Grammars? Then find a tool (???) to create a semantic checker for that Attribute
Grammar?
It gets progressively harder
Spring 2014 Jim Hogg - UW - CSE - P501 H-36
Source TargetFront End Back End
Scan
chars
tokens
AST
IR
‘Middle End’
Optimize
Select Instructions
Parse
Semantics
Allocate Registers
Emit
Machine Code
IR
IR
IR
IR
IR
Convert
AST
AST
AST = Abstract Syntax Tree
IR = Intermediate Representation
Spring 2014 Jim Hogg - UW - CSE - P501 Q-37
Example Optimization Classic example: Array references in a loop
for (k = 0; k < n; k++) a[k] = 0;
Naive codegen for a[k] = 0 in loop body mov eax, 4 // elemsize = 4 bytes imul eax, [ebp+offsetk] // k * elemsize add eax, [ebp+offseta] // &a[0] + k * elemsize mov [eax], 0 // a[k] = 0
Better! mov eax, [ebp+offseta] // &a[0], once-off
mov [eax], 0 // a[0]=0, a[1]=0, etc add eax, 4 // eax = &a[1],
&a[2], etcNote: pointers allow a user to do this directly in C or C++Eg: for (p = a; p < a + n; ) *p++ = 0;
Any Loops in this Code?
Spring 2014 Jim Hogg - UW - CSE - P501 T-38
i = 0goto L8
L7: i++L8: if (i < N) goto L9
s = 0j = 0goto L5
L4: j++L5: N--
if(j >= N) goto L3if (a[j+1] >= a[j]) goto L2t = a[j+1]a[j+1] = a[j]a[j] = ts = 1
L2: goto L4L3: if(s != ) goto L1 else goto L9L1: goto L7L9: return
Anyone recognize or guess the algorithm?
1 i = 12 j = 13 t1 = 10 * i4 t2 = t1 + j5 t3 = 8 * t26 t4 = t3 - 887 a[t4] = 08 j = j + 19 if j <= 10 goto #3
10 i = i + 111 if i <= 10 goto #212 i = 113 t5 = i - 114 t6 = 88 * t515 a[t6] = 116 i = i + 117 if i <= 10 goto #13
Jim Hogg - UW - CSE - P501 G-39
Typical "tuple stew" - IR generated by traversing an AST
Partition into Basic Blocks:• Sequence of consecutive instructions• No jumps into the middle of a BB• No jumps out of the middles of a BB• "I've started, so I'll finish"• (Ignore exceptions)
Basic Blocks
1 i = 12 j = 13 t1 = 10 * i4 t2 = t1 + j5 t3 = 8 * t26 t4 = t3 - 887 a[t4] = 08 j = j + 19 if j <= 10 goto #3
10 i = i + 111 if i <= 10 goto #212 i = 113 t5 = i - 114 t6 = 88 * t515 a[t6] = 116 i = i + 117 if i <= 10 goto #13
Spring 2014 Jim Hogg - UW - CSE - P501 G-40
Identify Leaders (first instruction in a basic block):• First instruction is a leader• Any target of a branch/jump/goto• Any instruction immediately after a branch/jump/goto
Leaders in red. Why is each leader a leader?
Basic Blocks : Leaders
Jim Hogg - UW - CSE - P501 G-41
i = 1
j = 1
t1 = 10 * it2 = t1 + jt3 = 8 * t2t4 = t3 - 88a[t4] = 0j = j + 1if j <= 10 goto B3
B1
B2
B3
i = i + 1if i <= 10 goto B2
B4
i = 1B5
t5 = i - 1t6 = 88 * t5a[t6] = 1i = i + 1if i <= 10 goto B6
B6
EXIT
ENTRY
Control Flow Graph ("CFG", again!)
• 3 loops total• 2 of the loops are nested
Most of the executions likely spent in loop bodies; that's where to focus efforts at optimization
Basic Blocks : Flowgraph
Loop in a Flowgraph: Intuition
Spring 2014 Jim Hogg - UW - CSE - P501 T-42
Header Node
• Cluster of nodes, such that:
• There's one node called the "header"• I can reach all nodes in the cluster from the header• I can get back to the header from all nodes in the cluster• Only once entrance - via the header• One or more exits
Spring 2014 Jim Hogg - UW - CSE - P501 H-43
Source TargetFront End Back End
Scan
chars
tokens
AST
IR
‘Middle End’
Optimize
Select Instructions
Parse
Semantics
Allocate Registers
Emit
Machine Code
IR
IR
IR
IR
IR
Convert
AST
AST
AST = Abstract Syntax Tree
IR = Intermediate Representation
Spring 2014 Jim Hogg - UW - CSE - P501 J-44
Call Example
; n = sum(1, 2)
push 2 ; push argspush 1call sum ; push L and jump
L:add esp, 8 ; pop argsmov [ebp+offsetn], eax ; store result into n
Spring 2014 Jim Hogg - UW - CSE - P501 J-45
Example Function
int sum(int x, int y) { int a; int b; a = x; b = a + y; return b;}
Simply calculates x+y, but with a few extraneous statements to illustrate the generated code
Spring 2014 Jim Hogg - UW - CSE - P501 J-46
Assembly Language Version
sum PROCpush ebp ; prologmov ebp, esp ; prologsub esp, 8 ;
prolog
mov eax, [ebp+8] ; eax = xmov [ebp-4], eax ; a = eax
mov ecx, [ebp-4] ; ecx = aadd ecx, [ebp+12] ; ecx += ymov [ebp-8], ecx ; b = ecx
mov eax, [ebp-8] ; eax = b
mov esp, ebp ; epilogpop ebp ; epilogret 0
sum ENDP
int sum(int x, int y) { int a; int b; a = x; b = a + y; return b;}
Spring 2014 Jim Hogg - UW - CSE - P501 J-47
Assembly Language Version
sum PROCpush ebp ; prologmov ebp, esp ;
prologsub esp, 8 ; prolog
mov eax, [ebp+8] ; eax = xmov [ebp-4], eax ; a = eax
mov ecx, [ebp-4] ; ecx = aadd ecx, [ebp+12] ; ecx +=
ymov [ebp-8], ecx ; b = ecx
mov eax, [ebp-8] ; eax = b
mov esp, ebp ; epilog
pop ebp ; epilog
retsum ENDP
0000 0001
esp
???? ????
old ebp ebp
???? ????
esp
0000 0002y
x
0000 0001
0000 0002y
x
a
b
stack grows up the page
00A5 F988retaddr
00A5 F988
-
+
Spring 2014 Jim Hogg - UW - CSE - P501 J-48
cdecl - Responsibilities Caller:
if you need the values currently in eax, ecx or edx then save them!
push arguments right-to-left execute call instruction pop arguments from stack after ret restore any of eax, ecx or edx that you saved before call
Callee: allocate space in stack frame for local variables if you will use any of ebx, edi or esi then save them! execute function body move result into eax restore any of ebx, edi or esi that you saved earlier Pop the stack frame so that return-address is top-of-stack execute ret instruction
H-49
Lessons Learned
Example 1st - Theory 2nd
Students are smarter than you think
Check for solutions to the project on the Internet!
You are bound by the syllabus
Thinking on your feet is impossible
Don't try to explain "The Visitor Pattern"
It's much more work than you thought
We reach the exciting stuff, just as the course ends
Dry-run first with your colleagues