CS 153: Concepts of Compiler Design October 29 Class Meeting Department of Computer Science San Jose State University Fall 2014 Instructor: Ron Mak mak.

1

CS 153: Concepts of Compiler DesignOctober 29 Class Meeting

Department of Computer ScienceSan Jose State University

Fall 2014Instructor: Ron Mak

www.cs.sjsu.edu/~mak

http://www.cs.sjsu.edu/~mak

Computer Science Dept.Fall 2014: October 29

CS 153: Concepts of Compiler Design© R. Mak

2

JJDoc

JJDoc produces documentation for your grammar.

Right-click in the .jj edit window.

It generates an HTML file from a .jj grammar file.

Read Chapter 5 of the JavaCC book. Ideal for your project documentation!



3

JJDoc on the Command Line

Example bash script for a Mac:

java –classpath Eclipse_path/plugins/sf.eclipse.javacc_1.5.24/javacc.jar jjdoc .jj file

#!/bin/bash

java -classpath /Applications/Eclipse/plugins/sf.eclipse.javacc_1.5.24/javacc.jar jjdoc $1



4

Assignment #6

Generate a working parser with JavaCC

This assignment is the start of your compiler project. This assignment due Monday, November 10.

By the end of the semester, you must implement enough of your compiler to be able to write nontrivial programs in your chosen source language, compile them, and run them.

Your final compiler project will be due Friday, December 12.



5

Assignment #6, cont’d

Keep the grammar of your source language simple!

Add features incrementally. This assignment represents a snapshot of your

early thinking about language design. You can change or add new features later.

_



6

Assignment #6: Suggested Implementation Order

1. Expressions with numeric constants and scalar variables no type checking yet no arrays and records yet

2. Assignment statements

3. Control statements

4. Variable declarations no type definitions yet

5. Procedure and function declarations

6. Procedure and function calls

7. Type definitions type checking arrays and records

For this assignment,do at least the first 3.

(Some control statements;you can add more later.)



7

Assignment #6: Turn in a Zip File The EBNF production rules for your language.

At least 20 rules for this assignment (you can add more later).

Generate with JJDoc.

A .jj file based on your EBNF rules.

A sample program (or a set of statements) written in your source language. Show off features of your parser as far as you

implemented it for this assignment. Can be error-free (i.e., no syntax error handling yet). Output from compiling your sample program.



8

Assignment #6

Add syntactic actions to print messages when the parser recognizes major source language constructs (such as statements and expressions) in the source program. Doesn’t have to be fancy output.

_



9

JJTree

JJTree builds an Abstract Syntax Tree (AST); i.e., a parse tree.

It works as a preprocessor for JavaCC grammars._



10

JJTree

JJTree augments a JavaCC grammar by inserting tree-building code.

When a parser is generated from the augmented grammar, and The generated parser is executed against a source program … An AST is created.

.jjt JJTree

JavaCC

GeneratedParser

sourceprogram

AST

JavaC

.jj

.java



11

Example: Simple Calculator

void Expression() : {} { {System.out.println("EXPRESSION STARTS");} Operator() {System.out.println("EXPRESSION ENDS");}}

void Operator() : {} { Operand() "+" {System.out.println("Operator: " + tokenImage[PLUS]);} Operand()}

void Operand() : {Token t;} { t=<DIGITS> {System.out.println("Operand: " + t.image);}}

calculator_simple.jj



12

Example: A Simple ASTSimpleNode Expression() : {} { Operator() {return jjtThis;}}

void Operator() : {} { Operand() "+" Operand()}

void Operand() : {} { <DIGITS> }

PARSER_BEGIN(Calculator)import java.io.*;

public class Calculator { public static void main(String[] args) { Reader sr = new StringReader(args[0]); Calculator calc = new Calculator(sr); try { SimpleNode node = calc.Expression(); node.dump(">"); }

catch (ParseException ex) { ex.printStackTrace(); } }}PARSER_END(Calculator)

jjtThis representsthe current AST node.

dump() is theAST print method.

SimpleNodeimplements theJJTree Node class.

calculator_tree.jjt



13

Example: Print AST Node Images

SimpleNode Expression() : {} { Operator() {return jjtThis;}}

void Operator() : {Token t;} { Operand() t="+" {jjtThis.setImage(t.image);} Operand()}

void Operand() : {Token t;} { t=<DIGITS> {jjtThis.setImage(t.image);}}

calculator_tree_image.jjt

Modify the generated SimpleNode class

to add the image fieldand the setImage() method.

You can modify the generated SimpleNode class.



14

Example: Operating CalculatorSimpleNode Expression() : {} { Operator() { SimpleNode operator = (SimpleNode) jjtThis.jjtGetChild(0); SimpleNode first = (SimpleNode) operator.jjtGetChild(0); int firstValue = (Integer) first.jjtGetValue();

SimpleNode second = (SimpleNode) operator.jjtGetChild(1); int secondValue = (Integer) second.jjtGetValue(); if (((String) operator.jjtGetValue()).equals("+")) { jjtThis.jjtSetValue(firstValue + secondValue); } else { System.out.println("Unknown operator"); } return jjtThis; }}

calculator_calculating.jjt

Second child

First child



15

The Visitor Design Pattern

Purpose: Visit the different node types of a tree structure. Perform operations on the nodes

without needing to modify the nodes.

The visitor interface declares a visit() method for each node type that wants to be visited. visit(NodeTypeA node)

visit(NodeTypeB node)

Each tree node has an accept() method to accept a visitor. accept(Visitor v)



16

The Visitor Design Pattern

A tree node’s accept() method calls the visitor’s visit() method that’s appropriate for the node type, passing itself as an argument. Accept a visitor call the visitor’s visit() method “Here’s the house key. Go visit my house now.”

In order to perform its operation, the visitor’s visit() method uses the node argument (the house key) to access the node’s public data and to call the node’s public methods.



17

Calculator and the Visitor Design Pattern The visitor interface

declares an overloaded visit() method for each node type.

Each tree node has an accept() method to accept a visitor.

A tree node’s accept() method calls the visitor’s visit() method that’s appropriate for the node type, passing itself as an argument.

The visitor’s visit() method uses the node argument to access the node’s public data and to call the node’s public methods in order to perform its operation.

JJTree generates theblack classes. You

write the blue classes.



18

Calculator and the Visitor Design Pattern First create a visitor object from

class SumVisitor. There is only one visitor object!

Tell the root node of the expression parse tree to accept the visitor. Call the root node’s accept()

method and pass the visitor object that will visit the root node.

The root node’s accept() method makes the visit happen by calling the visitor’s overloaded visit() method and passing itself (the node = house key) as a parameter. The parameter’s node type

determines which visit() method of the visitor is called.

Reader sr = new StringReader(args[0]);Calculator calc = new Calculator(sr); SimpleNode root = calc.Expression();SumVisitor visitor = new SumVisitor(); root.jjtAccept(visitor, null);System.out.println("Sum is " + visitor.sum);



19

Calculator and the Visitor Design Pattern

public class CalculatorVisitorAdapter implements CalculatorVisitor { public Object visit(SimpleNode node, Object data) { return node.childrenAccept(this, data); }

public Object visit(ASTExpression node, Object data) { return node.childrenAccept(this, data); }

public Object visit(ASTOperator node, Object data) { return node.childrenAccept(this, data); }

public Object visit(ASTOperand node, Object data) { return node.childrenAccept(this, data); }}

The default action when visiting each type of tree node is to tell each of the node’s children to accept the visitor. (In other words, recursively walk down thetree.) This action can be overridden by subclasses.

JJTree generates the CalculatorVisitor interface.

One visit() methodfor each tree node typethat can accept visitors.



20

Calculator and the Visitor Design Pattern, cont’d

SumVisitor’s visit() method is called whenever the tree node is an ASTOperand (i.e., an ASTOperand node is being visited). The SumVisitor object simply adds the value stored

in the operand node to its running sum . Then it performs the default action for any children.

SumVisitor is a subclass of CalculatorVisitorAdaptor. The adaptor implements visit() methods for the other node types.

public class SumVisitor extends CalculatorVisitorAdapter { public int sum = 0;

public Object visit(ASTOperand operand, Object data) { sum += (Integer) operand.jjtGetValue(); return super.visit(operand, data); }}

After summing, do the default action (super.visit) for any children. (This is a preorder traversal.)

Only override the visit method of the ASTOperand node.

calculator_visitor.jjt



21

JJTree: New Node Names

By default, JJTree creates an AST where the nodes are named after the nonterminals in the grammar. This makes the tree

design tightly-coupled to the source language and its grammar.

Use node descriptors to change the name of a node.

SimpleNode Expression() #Expr : {} { Operator() {return jjtThis;}}

void Operator() #AddOp : {} { Operand() "+" Operand()}

void Operand() #Opnd : {} { <DIGITS> }

node_name_change.jjt



22

JJTree: Node Descriptors

You can use node descriptors to combine nodes by reusing node names.

Now both production rules Expression and SimpleExpression will generate Expr nodes.


SimpleNode SimpleExpression() #Expr : {} { Operator() {return jjtThis;}}



23

JJTree: Node Descriptors, cont’d

Use the node descriptor #void to cause a production rule not to generate an AST node.

There are other things you can do with node descriptors.

Read Chapter 4 of the JavaCC book.


void Operator() #void : {} { Operand() "+" Operand()}

void Operand() #Opnd : {} { <DIGITS> }

node_void.jjt



24

JJTree: Tree Shaping

Will our simple calculator grammar accept 1+2+3+4? Change

to

Tree:

void Operator() : {Token t;}{ Operand() "+" Operand()}

void Operator() : {Token t;}{ Operand() ( "+" Operand() )*}

Expression Operator Operand Operand Operand Operand

calculator_tree_shape_1.jjt



25

JJTree: Tree Shaping, cont’d

Solution: Use embedded definite node descriptors:

Tree:

void Operator() #void : {Token t;}{ Operand() ( "+" Operand() #add(2) )*}

Expression add add add Operand Operand Operand Operand calculator_tree_shape_2.jjt



26

What JJTree, JJDoc, and JavaCC Do

You feed JJTree a .jjt grammar file Token specifications using regular expressions Production rules using EBNF

JJTree produces a .jj grammar file

JavaCC generates a scanner, parser, and tree-building routines Code for the visitor design pattern

to walk the parse tree.

JJDoc produces a .html containing the ENBF



27

What JJTree, JJDoc, and JavaCC Do

However, JJTree and JavaCC will not:

Generate code for a symbol table Generate any backend code

You have to providethis code!



28

Pcl

Pcl is a teeny, tiny subset of Pascal. Use JavaCC to generate a Pcl parser

and integrate with our Pascal interpreter’s symbol table components parse tree components

We’ll be able to parse and print the symbol table and the parse tree in our favorite XML format

Sample program test.pcl:

PROGRAM test;VAR i, j, k : integer; x, y, z : real;BEGIN i := 1; j := i + 3; x := i + j; y := 314.15926e-02 + i - j + k; z := x + i*j/k - x/y/zEND.



29

Pcl Challenges

Get the JJTree parse trees to build properly with respect to operator precedence. Use embedded definite node descriptors!

Decorate the parse tree with data type information.

Can be done as the tree is built, or as a separate pass. You can use the visitor pattern to implement the pass.

Hook up to the symbol table and parse tree printing classes from the Pascal interpreter.



30

Pcl, cont’doptions{ JJTREE_OUTPUT_DIRECTORY="src/wci/frontend"; NODE_EXTENDS="wci.intermediate.icodeimpl.ICodeNodeImpl"; ...}

PARSER_BEGIN(PclParser)...public class PclParser{ // Create and initialize the symbol table stack. symTabStack = SymTabFactory.createSymTabStack(); Predefined.initialize(symTabStack); ... // Parse a Pcl program. Reader reader = new FileReader(sourceFilePath); PclParser parser = new PclParser(reader); SimpleNode rootNode = parser.program(); ...



31

Pcl, cont’d ...

// Print the cross-reference table. CrossReferencer crossReferencer = new CrossReferencer(); crossReferencer.print(symTabStack);

// Visit the parse tree nodes to decorate them with type information. TypeSetterVisitor typeVisitor = new TypeSetterVisitor(); rootNode.jjtAccept(typeVisitor, null);

// Create and initialize the ICode wrapper for the parse tree. ICode iCode = ICodeFactory.createICode(); iCode.setRoot(rootNode); programId.setAttribute(ROUTINE_ICODE, iCode); // Print the parse tree. ParseTreePrinter treePrinter = new ParseTreePrinter(System.out); treePrinter.print(symTabStack);}PARSER_END(PclParser)

Demo

CS 153: Concepts of Compiler Design October 29 Class Meeting Department of Computer Science San Jose State University Fall 2014 Instructor: Ron Mak mak.

Documents

concepts of compiler

assignment statements

language design

mak jjdoc

jjdoc jjdoc

final compiler project

ron mak

jj file