CS453 Lecture Lexical Analysis with JLex 1 Plan for Lexical Analysis with JLex Overview of the MeggyJava Assignments Lexer generators, show 15min example Expressing tokens with regular expressions – regular expression syntax for JLex – using JLex with JavaCup How do lexer generators work? – Convert regular expressions to NFA – Converting an NFA to DFA – Implementing the DFA CS453 Lecture Introduction and MiniSVG 2 Structure of the MeggyJava Compiler “sentences” Synthesis Analysis character stream lexical analysis “words” tokens semantic analysis syntactic analysis AST AST and symbol table code gen Atmel assembly code PA2: MeggyJava and Atmel warmup PA3: setPixel compiler PA4: add control flow PA5: add functions PA6: add variables and objects PA7: add arrays CS453 Lecture Lexical Analysis with JLex 3 Specifying Tokens with JLex JLex example input file: package mjparser; import java_cup.runtime.Symbol; %% %line %char %cup %public %eofval{ return new Symbol(sym.EOF, new TokenValue("EOF", yyline, yychar)); %eofval} LETTER=[A-Za-z] DIGIT=[0-9] UNDERSCORE="_" LETT_DIG_UND={LETTER}|{DIGIT}|{UNDERSCORE} ID={LETTER}({LETT_DIG_UND})* %% "&&" {return new Symbol(sym.AND, new TokenValue(yytext(), yyline, yychar)); } "+" {return new Symbol(sym.PLUS, ...); } "if" {return new Symbol(sym.IF,...); } {ID} {return new Symbol(sym.ID, new ... {EOL} { /* reset yychar */ … } {WS} { /* ignore */ } 1 q 2 q 3 q a a a 0 q } {a Alphabet = Nondeterministic Finite Acceptor (NFA)
7
Embed
Nondeterministic Finite Acceptor (NFA)cs453/yr2011/Slides/... · q1q2 q3 a a a q0 Two choices Alphabet = {a} Nondeterministic Finite Accepter (NFA) aa q0 q1q2 q3 a a First Choice
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
CS453 Lecture Lexical Analysis with JLex 1
Plan for Lexical Analysis with JLex
Overview of the MeggyJava Assignments
Lexer generators, show 15min example
Expressing tokens with regular expressions – regular expression syntax for JLex – using JLex with JavaCup
How do lexer generators work? – Convert regular expressions to NFA – Converting an NFA to DFA – Implementing the DFA
CS453 Lecture Introduction and MiniSVG 2
Structure of the MeggyJava Compiler
“sentences”
Synthesis Analysis
character stream
lexical analysis
“words” tokens
semantic analysis
syntactic analysis
AST
AST and symbol table
code gen
Atmel assembly code
PA2: MeggyJava and Atmel warmup PA3: setPixel compiler PA4: add control flow PA5: add functions PA6: add variables and objects PA7: add arrays
CS453 Lecture Lexical Analysis with JLex 3
Specifying Tokens with JLex JLex example input file:
package mjparser; import java_cup.runtime.Symbol;
%% %line %char %cup %public
%eofval{ return new Symbol(sym.EOF, new TokenValue("EOF", yyline, yychar)); %eofval}