1.Syntax and Semantics The syntax of a programming language is the part of the language definition that says how programs look: their form and structure,

1.1. Syntax and SemanticsSyntax and Semantics

The The syntaxsyntax of a programming language is the part of the language of a programming language is the part of the language definition that says how programs look: their form and structure, such definition that says how programs look: their form and structure, such as its expressions, statements, and so on.as its expressions, statements, and so on.

The The semanticssemantics of a programming language is the part of the language of a programming language is the part of the language definition that says what programs do: the behavior and meaning of definition that says what programs do: the behavior and meaning of those expressions, statements.those expressions, statements.

For example: For example:

The The syntaxsyntax of a C of a C ifif statement is statement is

ifif (<expr>) <statement> (<expr>) <statement>

The The semanticssemantics of this statement form means if the current value of the of this statement form means if the current value of the expression <expr> is expression <expr> is truetrue, the embedded statement <statement> is , the embedded statement <statement> is selected for execution.selected for execution.

Chapter 3 Describing Syntax and Semantics

2.2. A Grammar Example for EnglishA Grammar Example for English

2.12.1 <A> for an article (<A> for an article (a, thea, the) and express our definition:) and express our definition:

<A> <A> a | the a | the

2.2 <N> for a noun (2.2 <N> for a noun (dog, cat, or ratdog, cat, or rat):):

<N> <N> dog | cat | rat dog | cat | rat

2.3 <NP> for a noun phrase (an article followed by a noun):2.3 <NP> for a noun phrase (an article followed by a noun):

<NP> <NP> <A> <N> <A> <N>

2.4 <V> for a verb (2.4 <V> for a verb (loves, hates, or eatsloves, hates, or eats))

<V> <V> loves | hates | eats loves | hates | eats

2.5 <S> for a sentence (a noun phrase, followed by a verb, followed by 2.5 <S> for a sentence (a noun phrase, followed by a verb, followed by another noun phrase: another noun phrase:

<S> <S> <NP> <V> <NP> <NP> <V> <NP>

2.6 Grammar defining a small subset of unpunctuated English2.6 Grammar defining a small subset of unpunctuated English

<S> <S> <NP> <V> <NP> <NP> <V> <NP> <NP> <NP> <A> <N> <A> <N> <V> <V> loves | hates | eats loves | hates | eats <N> <N> dog | cat | rat dog | cat | rat <A> <A> a | the a | the

How does such a grammar define a language? Think of the grammar How does such a grammar define a language? Think of the grammar as a set of rules that say how to build a tree. as a set of rules that say how to build a tree.

<S><S>

<NP><NP> <V><V> <NP><NP>

<A><A> <N><N> <A><A> <N><N>

thethe dogdog

lovesloves

thethe catcat

<S><S>

<NP><NP> <V><V> <NP><NP>

<A><A> <N><N> <A><A> <N><N>

aa catcat

eatseats

thethe ratrat

3 3 The Basic Concepts of Describing SyntaxThe Basic Concepts of Describing Syntax

3.13.1 SentencesSentences (statements) – The strings of a language. The (statements) – The strings of a language. The syntax rules of the language specify which strings of syntax rules of the language specify which strings of characters in the language.characters in the language.

3.2 3.2 lexemes lexemes – The small units (identifier, literals, operators, and – The small units (identifier, literals, operators, and special words). special words).

3.33.3 Token Token – A category of the language lexemes.– A category of the language lexemes. 3.43.4 For example: For example:

StatementStatement index = 2 * count + 17;index = 2 * count + 17;

LexemesLexemes TokensTokensindexindex identifieridentifier== equal_signequal_sign22 int_literalint_literal** mult_opmult_opcountcount identifieridentifier++ plus_opplus_op1717 int_literalint_literal;; semicolonsemicolon

4 4 Formal Methods of Describing SyntaxFormal Methods of Describing Syntax

4.14.1 A Grammar Example for a Programming LanguageA Grammar Example for a Programming Language<exp> <exp> <exp>+<exp> | <exp>*<exp> | ( <exp> ) | a | b | c <exp>+<exp> | <exp>*<exp> | ( <exp> ) | a | b | c

expression: aexpression: aexpression: a + bexpression: a + bexpression: a + b * cexpression: a + b * cexpression: ( ( a + b ) * c )expression: ( ( a + b ) * c )

4.24.2 Parse treesParse trees:: Hierarchical syntactic structures Hierarchical syntactic structures

<exp><exp>

(( <exp><exp> ))

<exp><exp> <exp><exp>**

cc<exp><exp>(( ))

++<exp><exp> <exp><exp>

aa bb

Recursive Recursive GrammarGrammar

4.34.3 Backus-Naur Form and Context-Free GrammarsBackus-Naur Form and Context-Free Grammars

John Backus and Noam Chomsky (The middle to late 1950’s) proposed a John Backus and Noam Chomsky (The middle to late 1950’s) proposed a method (grammar) for describing syntax.method (grammar) for describing syntax.

Start symbolStart symbol <S> <S> <NP> <V> <NP> <NP> <V> <NP>

productionproduction <NP> <NP> <A> <N> <A> <N>

<V> <V> loves | hates | eats loves | hates | eats

Non-terminalNon-terminal <N> <N> dog | cat | rat dog | cat | ratsymbolssymbols

<A> <A> a | the a | the

tokens (Lexemes): terminal symbolstokens (Lexemes): terminal symbols

The grammar has four important parts:The grammar has four important parts:

TheThe Non-terminal symbols Non-terminal symbols are strings enclosed in angle brackets, such as <NP>. are strings enclosed in angle brackets, such as <NP>. TheThe Non-terminal symbols Non-terminal symbols of a grammar often correspond to different kinds of of a grammar often correspond to different kinds of language constructslanguage constructs

The grammar designates one of the non-terminal symbols as the root of the parse The grammar designates one of the non-terminal symbols as the root of the parse tree: the tree: the start symbol.start symbol.

AA production (rule) production (rule) consists of consists of a left-hand side (LHS) a left-hand side (LHS) : a single non-terminal symbol: a single non-terminal symbola arrowa arrow : : a right-hand side (RHS) a right-hand side (RHS) : a sequence of one or more things, each of : a sequence of one or more things, each of

which can be either a token or a non-terminal which can be either a token or a non-terminal symbol. The special symbol symbol. The special symbol || is used to is used to separate the right-hand sides. separate the right-hand sides.

Tokens and LexemesTokens and Lexemes : : Terminal symbols Terminal symbols (the smallest units of syntax).(the smallest units of syntax).

The example: The grammar for a simple language of expressions with three variables.The example: The grammar for a simple language of expressions with three variables.

<exp> <exp> <exp>+<exp> | <exp>*<exp> | ( <exp> ) | a | b | c <exp>+<exp> | <exp>*<exp> | ( <exp> ) | a | b | c

This grammar can be written in a different way, without the This grammar can be written in a different way, without the || notation: notation:

<exp> <exp> <exp>+<exp> <exp>+<exp> <exp> <exp> <exp>*<exp> <exp>*<exp> <exp> <exp> ( <exp> ) ( <exp> ) <exp> <exp> a a <exp> <exp> b b <exp> <exp> c c

The special non-terminal symbol The special non-terminal symbol <empty><empty> : : Generate an empty string – a string of no Generate an empty string – a string of no tokenstokens

if – thenif – then statement with an optional statement with an optional elseelse part might be defined like this: part might be defined like this:

<if-stmt> <if-stmt> if <expr> them <stmt> <else-part> if <expr> them <stmt> <else-part><else-part> <else-part> else <stmt> | <empty> else <stmt> | <empty>

Context-free Grammars: Context-free Grammars: The grammars are context-free grammars, the children The grammars are context-free grammars, the children of a node in the parse tree depend only on that node’s non-terminal symbol; they of a node in the parse tree depend only on that node’s non-terminal symbol; they do not depend on the context of neighboring nodes in the tree.do not depend on the context of neighboring nodes in the tree.

MetalanguageMetalanguage: a language is used to describe another language. BNF is a : a language is used to describe another language. BNF is a metalanguage.metalanguage.

4.4 An example for writing Grammars4.4 An example for writing Grammars::

float a;float a;

boolean boolean (bool)(bool)b, c, d;b, c, d;

int e=1, f, g = 1+2;int e=1, f, g = 1+2;

Step 1: Divide the problem into smaller pieces.Step 1: Divide the problem into smaller pieces.

The major components: The major components: the type name the type name

the list of variablesthe list of variables

the final semicolon the final semicolon

Non-terminal symbolsNon-terminal symbols

<var-dec> <var-dec>

<type-name> <type-name>

<declarator-list><declarator-list>

<var-dec> <var-dec> <type-name> <declarator-list>; <type-name> <declarator-list>;

Step 2: List the primitive typesStep 2: List the primitive types

<type-name> <type-name> boolean ( boolean (boolbool) | byte | short | int | ) | byte | short | int | | long | char | float | double | long | char | float | double

Step 3: Define the declarator listStep 3: Define the declarator list

A declarator list is a list of one or more declarators, A declarator list is a list of one or more declarators, followed by a comma, followed by a (smaller) declarator followed by a comma, followed by a (smaller) declarator list. list.

<declarator-list> <declarator-list> <declartor> <declartor>

| <declarator> , <declarator-list>| <declarator> , <declarator-list>

Step 4: Define a declaratorStep 4: Define a declarator

It is a variable name followed by, optionally, by a It is a variable name followed by, optionally, by a equal sign and an expression.equal sign and an expression.

<declarator> <declarator> <variable-name> <variable-name>

| <variable-name> = <expr> | <variable-name> = <expr>

4.5 Examples:4.5 Examples:

A Grammar for a Small LanguageA Grammar for a Small Language

<<programprogram> > begin < begin <stmt_liststmt_list> end> end

<<stmt_liststmt_list> > < <stmtstmt> | <> | <stmtstmt> ; <> ; <stmt_liststmt_list>>

<<stmtstmt> > < <varvar> = <> = <expressionexpression>>

<<varvar> > AA | | BB | | CC

<<expressionexpression> > < <varvar> + <> + <varvar> | <> | <varvar> - <> - <varvar> | <> | <varvar>>

Remarks: Remarks:

1.1. Only one kind of statement: assignment.Only one kind of statement: assignment.

2.2. A program consists of A program consists of beginbegin and and endend

3.3. A list of assignment statements separated by semicolonsA list of assignment statements separated by semicolons

4.4. An expression: a variable, add two variables, or subtract one variable from An expression: a variable, add two variables, or subtract one variable from another one.another one.

5.5. Only three variable names: A, B, C. Only three variable names: A, B, C.

Leftmost derivations:Leftmost derivations:

<<programprogram> > => begin <=> begin <stmt_liststmt_list> end> end

=> begin <=> begin <stmtstmt> ; <> ; <stmt_liststmt_list> end> end

=> begin <=> begin <varvar> = <> = <expressionexpression>; <>; <stmt_liststmt_list> end> end

=> begin => begin AA = < = <expressionexpression>; <>; <stmt_liststmt_list> end> end

=> begin => begin AA = < = <varvar> + <> + <varvar>; <>; <stmt_liststmt_list> end> end

=> begin => begin AA = = BB + < + <varvar>; <>; <stmt_liststmt_list> end> end

=> begin => begin AA = = BB + + CC; <; <stmt_liststmt_list> end> end

=> begin => begin AA = = BB + + CC; <; <stmtstmt> end> end

=> begin => begin AA = = BB + + CC; <; <varvar> = <> = <expressionexpression> end> end

=> begin => begin AA = = BB + + CC; ; BB = < = <expressionexpression> end> end

=> begin => begin AA = = BB + + CC; ; BB = = CC end end

In this derivation, the replaced non-terminal is always left-most non-In this derivation, the replaced non-terminal is always left-most non-terminal. (right-most non-terminal or neither left-most nor right-most)terminal. (right-most non-terminal or neither left-most nor right-most)

A Grammar for Simple Assignment StatementsA Grammar for Simple Assignment Statements

<<assignassign> > < <idid> > == < <exprexpr> >

<<idid> > AA | | BB | | CC

<<exprexpr> > < <idid> > ++ < <exprexpr>>

| <| <idid> * <> * <exprexpr>>

| ( <| ( <exprexpr> ) > )

| <| <idid>>

A = B * ( A + C )A = B * ( A + C ) is generated by the leftmost derivation:is generated by the leftmost derivation:

<<assignassign> > => <=> <idid> > == < <exprexpr> >

=> => A = A = <<idid> * <> * <exprexpr>>

=> => AA = B= B * < * <exprexpr> >

=> => AA = B= B * ( < * ( <exprexpr> ) > )

=> => AA = B= B * ( < * ( <idid> > ++ < <exprexpr> ) > )

=> => AA = B= B * ( * ( AA ++ < <exprexpr> ) > )

=> => AA = B= B * ( * ( AA ++ < <idid> ) > )

=> => AA = B= B * ( * ( AA ++ CC ) )

A parse tree for the simple statement A = B * (A + C)

55 Ambiguity: A grammar that generates a sentence for which there are two or Ambiguity: A grammar that generates a sentence for which there are two or more distinct parse trees is said to be more distinct parse trees is said to be ambiguous. ambiguous.

An An ambiguousambiguous Grammar for Simple Assignment Statements Grammar for Simple Assignment Statements

<<assignassign> > < <idid> > == < <exprexpr> >

<<idid> > AA | | BB | | CC

<<exprexpr> > < <exprexpr> > ++ < <exprexpr>>

| <| <exprexpr> * <> * <exprexpr>>

| ( <| ( <exprexpr> ) > )

| <| <idid>>

The statement:The statement: A = B + C * A A = B + C * A has two distinct parse trees.has two distinct parse trees.

• By using By using Operator PrecedenceOperator Precedence and and Associativity of OperatorAssociativity of Operator, we solve the , we solve the ambiguous problem. ambiguous problem.

Two distinct parse trees for the same sentence, A = B + C * A

5.15.1 Operator Precedence: An operator in an expression is generated lower in the Operator Precedence: An operator in an expression is generated lower in the parse tree (and therefore must be evaluated first) has precedence over an parse tree (and therefore must be evaluated first) has precedence over an operator produced higher up in the tree.operator produced higher up in the tree.

The left parse tree indicates The left parse tree indicates

The multiplication operator has precedence over the addition The multiplication operator has precedence over the addition operator.operator.

The right parse tree indicatesThe right parse tree indicates

The addition operator has precedence over the multiplication The addition operator has precedence over the multiplication operator.operator.

A grammar can be written to define different operators (the addition and A grammar can be written to define different operators (the addition and multiplication) in a higher to lower ordering.multiplication) in a higher to lower ordering.

An unambiguous Grammar for Simple Assignment StatementsAn unambiguous Grammar for Simple Assignment Statements

<<assignassign> > < <idid> > == < <exprexpr> >

<<idid> > AA | | BB | | CC

<<exprexpr> > < <exprexpr> > ++ < <termterm>>

| <| <termterm> >

<<termterm> > <term> * <factor> <term> * <factor>

| <factor>| <factor>

<factor> <factor> ( < ( <exprexpr> ) > )

| <| <idid>>

Derive the statement A = B + C * A by using the grammar Derive the statement A = B + C * A by using the grammar

The leftmost derivation: The leftmost derivation:

<<assignassign> > => <=> <idid> > == < <exprexpr> >

=> => A = A = <<exprexpr>>

=> => A = A = <<exprexpr> + <t> + <termerm>>

=> => AA = = <<termterm> + <term> > + <term>

=> => AA = <factor> + <term>= <factor> + <term>

=> => AA = = <<idid> > ++ <t <termerm> >

=> => AA = B= B + <t + <termerm> >

=> => AA = B= B + <term> * <factor> + <term> * <factor>

=> => AA = B= B + <factor> * <factor> + <factor> * <factor>

=> => AA = B= B + <id> * <factor> + <id> * <factor>

=> => AA = B= B + C * <factor> + C * <factor>

=> => AA = B= B + C * <id> + C * <id>

=> => AA = B= B + C * A + C * A

The rightmost derivation: The rightmost derivation:

<<assignassign> > => <=> <idid> > == < <exprexpr> >

=> => <id> = <id> = <<exprexpr> + <t> + <termerm>>

=> => <id><id> = = <<exprexpr> + <term> * <factor> > + <term> * <factor>

=> => <id><id> = <expr> + <term> * <id>= <expr> + <term> * <id>

=> => <id><id> = = <<exprexpr> > ++ <term> * <term> * AA

=> => <id><id> = = <<exprexpr> > ++ <factor> * <factor> * AA

=> => <id><id> = = <<exprexpr> > ++ <id> * <id> * AA

=> => <id><id> = = <<exprexpr> > ++ C * C * AA

=> => <id><id> = = <<termterm> > ++ C * C * AA

=> => <id><id> = = <<factorfactor> > ++ C * C * AA

=> => <id><id> = = <<idid> > ++ C * C * AA

=> => <id><id> = = B B ++ C * C * AA

=> A=> A = = B B ++ C * C * AA

A unique parse tree for A = B + C * A using an unambiguous grammar

5.25.2 Associativity of Operators: The parse trees for expressions with two or more Associativity of Operators: The parse trees for expressions with two or more adjacent occurrences of operators with equal precedence have those occurrences adjacent occurrences of operators with equal precedence have those occurrences in proper hierarchical order.in proper hierarchical order.

Example:Example:

Derive this statement A = B + C + A by using the grammar (the Leftmost) Derive this statement A = B + C + A by using the grammar (the Leftmost) and generate its parse tree. This parse tree shows the left addition is lower and generate its parse tree. This parse tree shows the left addition is lower than the right addition (the left associative) than the right addition (the left associative)

A = B + C + AA = B + C + A

The leftmost derivation: The leftmost derivation:

<<assignassign> > => <=> <idid> > == < <exprexpr> >

=> => A = A = <<exprexpr>>

=> => A = A = <<exprexpr> + <t> + <termerm>>

=> => AA = = <<exprexpr>> + <<termterm> + <term> > + <term>

=> => AA = <term> + <term> = <term> + <term> + <term>+ <term>

=> => AA = = <<factorfactor> > ++ <t <termerm> + <term> > + <term>

=> A = <id> + <term> + <term>=> A = <id> + <term> + <term>

=> => AA = B= B + <t + <termerm> + <term> > + <term>

=> => AA = B= B + <factor> + <term> + <factor> + <term>

=> => AA = B= B + <id> + <term> + <id> + <term>

=> => AA = B= B + C + <term> + C + <term>

=> => AA = B= B + C + <factor> + C + <factor>

=> => AA = B= B + C + <id> + C + <id>

=> A = B + C + A => A = B + C + A

A parse tree for A = (B + C) + A illustrating the left associativity of addition

The leftmost derivation:The leftmost derivation:

The rightmost derivation: The rightmost derivation:

<<assignassign> > => <=> <idid> > == < <exprexpr> >

=> => <id> = <id> = <<exprexpr> + <t> + <termerm>>

=> => <id><id> = = <<exprexpr> + <factor> > + <factor>

=> => <id><id> = <expr> + <id>= <expr> + <id>

=> => <id><id> = = <<exprexpr> + > + AA

=> => <id><id> = = <<exprexpr> > ++ <term> + <term> + AA

=> => <id><id> = = <<exprexpr> > ++ <factor> + <factor> + AA

=> => <id><id> = = <<exprexpr> > ++ <id> + <id> + AA

=> => <id><id> = = <<exprexpr> > ++ C + C + AA

=> => <id><id> = = <<termterm> > ++ C + C + AA

=> => <id><id> = = <<factorfactor> > ++ C + C + AA

=> => <id><id> = = <<idid> > ++ C + C + AA

=> => <id><id> = = B B ++ C + C + AA

=> A=> A = = B B ++ C + C + AA

A parse tree for A = (B + C) + A illustrating the left associativity of addition

The rightmost derivation:The rightmost derivation:

• Left recursiveLeft recursive: when a BNF rule has its LHS also appearing at the : when a BNF rule has its LHS also appearing at the beginning of its RHS. beginning of its RHS.

An unambiguous Grammar for Simple Assignment StatementsAn unambiguous Grammar for Simple Assignment Statements

<<assignassign> > < <idid> > == < <exprexpr> >

<<idid> > AA | | BB | | CC

<<exprexpr>> <<exprexpr>> ++ < <termterm> | <> | <termterm> > (Left associative)(Left associative)

<<termterm>> <term><term> * <factor> | <factor> * <factor> | <factor> ((Left associativeLeft associative))

<factor> <factor> ( < ( <exprexpr> ) | <> ) | <idid>>

• Right recursiveRight recursive: when a BNF rule has its LHS also appearing at the end of : when a BNF rule has its LHS also appearing at the end of its RHS. its RHS.

Rules for exponentiation as a right-associative operatorRules for exponentiation as a right-associative operator

<<factorfactor>> < <expexp> > **** <<factorfactor>> | < | <expexp> > (Right associative)(Right associative)

<<expexp> > ( <exp> ) | <id> ( <exp> ) | <id>

5.35.3 An unambiguous grammar for An unambiguous grammar for if-then-elseif-then-else

Rules for Rules for if-then-elseif-then-else

<if_stmt> <if_stmt> if <logic_expr> then <stmt> if <logic_expr> then <stmt>

| if <logic_expr> then <stmt> else <stmt>| if <logic_expr> then <stmt> else <stmt>

Deriving Deriving

if <logic_expr> then if <logic_expr> then <stmt> else if <logic_expr> then if <logic_expr> then <stmt> else <stmt> <stmt> by the rules aboveby the rules above

Two distinct parse trees for the same sentential form

The unambiguous grammar for The unambiguous grammar for if-then-elseif-then-else

Rules for Rules for if-then-elseif-then-else

<stmt> <stmt> <matched> | <unmatched> <matched> | <unmatched>

<matched> <matched> if <logic_expr> then <matched> else <matched> if <logic_expr> then <matched> else <matched>

| | any non-if statementany non-if statement

<unmatched> <unmatched> if <logic_expr> then <stmt> if <logic_expr> then <stmt>

| if <logic_expr> then <matched> else | if <logic_expr> then <matched> else <unmached><unmached>

Deriving Deriving

if <logic_expr> then if <logic_expr> then <stmt> else if <logic_expr> then if <logic_expr> then <stmt> else <stmt> <stmt> by the rules aboveby the rules above