4 % 2 3 % # 5 2 )4 0 5 9 · “Dangling Else” Ambiguity One famous ambiguity is “dangling else ...

Comp215: Recursive GrammarsMack Joyner, Dan S. Wallach (Rice University)

Copyright Ⓒ 2016, Mack Joyner, Dan S. Wallach. All rights reserved.

Recall: Regular languagesEasiest way to build a scanner: use regular expressions (“regex”)

First, we need to understand a regular language

Formally:Alphabet : S

Empty language : /0 is a regular language

Singletons : 8a 2 S,{a} is a regular language

Combinations : if A and B are regular languages, then

A[B (union),

A•B (concatenation),

A⇤ (Kleene star), are regular languages

Regular languages / Regular expressionsRegular expressions: convenient syntax for regular languages

Union: (aaa|bbb)Concatenation: aaabbbKleene star: (aaa)*

Noam Chomsky: Grammar Pioneer

Regular GrammarRegular grammar: another way to describe regular languages

There’s a regular grammar for every regular language

A grammar is right-linear (recursive) if all productions are of the form:A aA,A a

A grammar is left-linear (recursive) if all productions are of the form:A Aa,A a,





Production rule





Every non-terminal must appear at least once on LHS

Non-terminals must be rightmost symbol of right-linear grammar





Terminals appear only on RHS





Terminals appear only on RHS

A regular grammar is either right or left recursive

Regular Expression to Regular grammarRegular expression: (a)*(b)*

Strings you’d expect to match: “aaa”, “aaaa”, “aaaabb”, “bbbbbb”,

A BB bBB empty

S AA aA



A BB bBB empty

S AA aA

S is the start symbol



A BB bBB empty

S AA aA


empty string terminal



A BB bBB empty

S AA aA


empty string terminal

Can you construct a regular grammar to accept only (a)n(b)n?

Context-Free GrammarRelax the regular grammar restriction to make it context-free

Describes how sentence of a language may be generatedNon-terminal can be anywhere on the production rule’s right hand side

A BB bBB empty

S AA aA



A BB bBB empty

S AA aA

Can you construct a context-free grammar to accept only (a)n(b)n?




S AA aAbA empty

Non-terminal A can now appear anywhere




S AA aAbA empty


What if I want to call non-terminal “AB” instead of “A”?




S AA aAbA empty


What if I want to call non-terminal “AB” instead of “A”?

Multiple non-terminals on LHS is a context-sensitive grammar

Categories of Grammarsregular

good for identifiers, parameter lists, subscriptsan expanded form of regular expressionsleft-regular or right-regular

context freeLHS of production is single non-terminal

context sensitiveLHS has a non-terminal that can be surrounded by terminals and/or non-terminals

recursively enumerableLHS can be any non-empty sequence of terminals and/or non-terminals

Categories of Grammarsregular

good for identifiers, parameter lists, subscriptsan expanded form of regular expressionsleft-regular or right-regular

context freeLHS of production is single non-terminal

context sensitiveLHS has a non-terminal that can be surrounded by terminals and/or non-terminals

recursively enumerableLHS can be any non-empty sequence of terminals and/or non-terminals

COMP 215

Backus-Naur Form (BNF)Used to describe syntax of PL; first used for Algol-60Nonterminals are enclosed in <...>

<expression>, <identifier>Alternatives indicated by |

<digit> ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9Options (0 or 1 occurrences) indicated by [...]

<stmt> ::= if <cond> then <stmt> [ else <stmt>]Repetition (0 or more occurrences) indicated by {...}

<unsigned> ::= <digit> {<digit>}Derivation

apply the rules, starting with start symbol and ending with a sentence

Backus-Naur Form (BNF)Used to describe syntax of PL; first used for Algol-60Nonterminals are enclosed in <...>

<expression>, <identifier>Alternatives indicated by |

<digit> ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9Options (0 or 1 occurrences) indicated by [...]

<stmt> ::= if <cond> then <stmt> [ else <stmt>]Repetition (0 or more occurrences) indicated by {...}

<unsigned> ::= <digit> {<digit>}Derivation

apply the rules, starting with start symbol and ending with a sentence

How would you do this using regular expressions?

BNF defined using BNF <syntax> ::= <rule> | <rule> <syntax> <rule> ::= <opt-whitespace> "<" <rule-name> ">" <opt-whitespace> "::=" <opt-whitespace> <expression> <line-end> <opt-whitespace> ::= " " <opt-whitespace> | "" <expression> ::= <list> | <list> <opt-whitespace> "|" <opt-whitespace> <expression> <line-end> ::= <opt-whitespace> <EOL> | <line-end> <line-end> <list> ::= <term> | <term> <opt-whitespace> <list> <term> ::= <literal> | "<" <rule-name> ">" <literal> ::= '"' <text1> '"' | "'" <text2> "'" <text1> ::= "" | <character1> <text1> <text2> ::= "" | <character2> <text2> <character> ::= <letter> | <digit> | <symbol> <letter> ::= "A" | "B" | "C" | "D" | "E" | "F" | "G" | "H" | "I" | "J" | "K" | "L" | "M" | "N" | "O" | "P" | "Q" | "R" | "S" | "T" | "U" | "V" | "W" | "X" | "Y" | "Z" | "a" | "b" | "c" | "d" | "e" | "f" | "g" | "h" | "i" | "j" | "k" | "l" | "m" | "n" | "o" | "p" | "q" | "r" | "s" | "t" | "u" | "v" | "w" | "x" | "y" | "z" <digit> ::= "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9" <symbol> ::= | "-" | "!" | "#" | "$" | "%" | "&" | "(" | ")" | "*" | "+" | "," | "-" | "." | "/" | ":" | ";" | "<" | "=" | ">" | "?" | "@" | "[" | "\" | "]" | "^" | "_" | "`" | "{" | "|" | "}" | "~" <character1> ::= <character> | "'" <character2> ::= <character> | ‘"' <rule-name> ::= <letter> | <rule-name> <rule-char>

<rule-char> ::= <letter> | <digit> | "-"

Example Revisited

anbn, n = 0, 1, 2, …

Example Revisited

anbn, n = 0, 1, 2, …

<start> ::= <expr><expr> ::= a <expr> b<expr> ::= empty

Example Grammar and Derivation <program> ::= <stmts> <stmts> ::= <stmt> | <stmt> ; <stmts> <stmt> ::= <var> = <expr> <var> ::= a | b | c | d <expr> ::= <term> + <term> | <term> - <term> <term> ::= <var> | const

<program> => <stmts> => <stmt> => <var> = <expr> => a = <expr> => a = <term> + <term> => a = <var> + <term> => a = b + <term> => a = b + const



Leftmost derivation



Sentence: has only terminal symbols

Leftmost derivation



Sentence: has only terminal symbols

Leftmost derivation

Language: set of all possible sentences derived from grammar

Derivation / Parse Tree

<program>

<stmts>

<stmt>

<var> = <expr>

a <term> + <term>

<var> const

b

A derivation tree is the tree resulting from applying productions to rewrite start symbol

a parse tree is the same tree starting with terminals and building back to the start symbol (goal symbol)


<program>

<stmts>

<stmt>

<var> = <expr>

a <term> + <term>

<var> const

b



derivation tree


<program>

<stmts>

<stmt>

<var> = <expr>

a <term> + <term>

<var> const

b



parse tree

derivation tree

Ambiguity

<expr> <expr>

<expr> <op> <expr> <expr> <op> <expr>

<expr> <op> <expr> <expr> <op> <expr>

const - const / const const - const / const

A grammar is ambiguous iff it generates a sentential form that has two or more distinct parse trees An ambiguous expression grammar:

<expr> ::= <expr> <op> <expr> | const<op> ::= / | -

“Dangling Else” AmbiguityOne famous ambiguity is “dangling else”<stmt> ::= if <cond> then <stmt> [else <stmt>]

This can deriveif X > 9 then if B = 4 then X := 5 else X := 0

“Dangling Else” AmbiguityOne famous ambiguity is “dangling else”<stmt> ::= if <cond> then <stmt> [else <stmt>]

This can deriveif X > 9 then if B = 4 then X := 5 else X := 0

which “if” does this else belong to?

“Dangling Else” AmbiguityCan solve syntactically by adding nonterminals & productions<stmt> ::= <matched> | <unmatched><matched> ::= if <cond> then <matched> else <matched> | <simple><unmatched> ::= if <cond> then <stmt> | if <cond> then <matched> else <unmatched>

Can also solve semantically“elses are associated with immediately preceding unmatched then”

if X > 9 then if B = 4 then X := 5 else X := 0

























Resolving Ambiguity

<expr>

<expr> - <term>

<term> / const <term>

const const

An ambiguous expression grammar: <expr> ::= <expr> <op> <expr> | const<op> ::= / | -

Unambiguous expression grammar: <expr> ::= <expr> - <term> | <term><term> ::= <term> / const | const

Resolving Ambiguity

<expr>

<expr> - <term>

<term> / const <term>

const const

An ambiguous expression grammar: <expr> ::= <expr> <op> <expr> | const<op> ::= / | -

Unambiguous expression grammar: <expr> ::= <expr> - <term> | <term><term> ::= <term> / const | const

Recursion

Left recursive grammars <expr> ::= <expr> + <term> | <term><term> ::= <term> * const | const

Right recursive grammars <expr> ::= <term> + <remain> | <term><term> ::= const * <term> | const<remain> ::= <term> | <term> + <expr>

Associativity

Left associativity: a - b + c = (a - b) + c

Right associativity: a ** b ** c = a ** (b ** c)

Example: Expressions

Consider following unambiguous grammar for expressions: <expr> ::= [<expr> <addop>] <term><term> ::= [<term> <mulop>] <factor><factor> ::= (<expr>) | <digit><addop> ::= + | -<mulop> ::= * | /<digit> ::= 0 | ... | 9

This grammar is left recursive and generates expressions that are left associative

Syntax Graphsequivalent to CFGs put the terminals in circles or ellipses and put the nonterminals in rectangles; Lines and arrows indicate how constructs are built

<expr> ::= <term> [ <addop> < expr>]<term> ::= <factor> [ <mulop> <term>]

term

addop

expr:factor

mulop

term:

JSON Grammar

json.org :

4 % 2 3 % # 5 2 )4 0 5 9 · “Dangling Else” Ambiguity One famous ambiguity is “dangling else ...

Documents