DEFINITION OF PARSING A parser is a compiler or interpreter component that breaks data into smaller elements for easy translation into another language. A parsertakes input in the form of a sequence of tokens or program instructions and usually builds a data structure in the form of a parse tree or an abstract syntax tree. ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 1
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
DEFINITION OF PARSING
A parser is a compiler or interpreter component that breaks data into smaller elements for easy translation into another language.
A parsertakes input in the form of a sequence of tokens or program instructions and usually builds a data structure in the form of a parse tree or an abstract syntax tree.
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 1
ROLE OF PARSER
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 2
• In the compiler model, the parser obtains a string of tokens from the lexical analyzer,
• and verifies that the string can be generated by the grammar for the source language.
• The parser returns any syntax error for the source language.
• It collects sufficient number of tokens and builds a parse tree.
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 3
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 4
• There are basically two types of parser:
• Top-down parser:• starts at the root of derivation tree and fills in
• picks a production and tries to match the input
• may require backtracking
• some grammars are backtrack-free (predictive)
• Bottom-up parser:• starts at the leaves and fills in
• starts in a state valid for legal first tokens
• uses a stack to store both state and sentential forms
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 5
TOP DOWN PARSING
• A top-down parser starts with the root of the parse tree, labeled with the start or goal symbol of the grammar.
• To build a parse, it repeats the following steps until the fringe of the parse tree matches the input string
• STEP1: At a node labeled A, select a production A α and construct the appropriate child for each symbol of α
• STEP2: When a terminal is added to the fringe that doesn’t match the input string, backtrack
• STEP3: Find the next node to be expanded.
• The key is selecting the right production in step 1
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 6
EXAMPLE FOR TOP DOWN PARSING• Supppose the given production rules are as follows:
• S-> aAd|aB
• A-> b|c
• B->ccd
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 7
PROBLEMS WITH TOPDOWN PARSING
1) BACKTRACKING
Backtracking is a technique in which for expansion of non-terminal symbol we choose one alternative and if some mismatch occurs then we try another alternative if any.
If for a non-terminal there are multiple production rules beginning with the same input symbol then to get the correct derivation we need to try all these alternatives.
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 8
EXAMPLE OF BACKTRACKING
• Suppose the given production rules are as follows:
• S->cAd
• A->a|ab
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 9
2) LEFT RECURSION
Left recursion is a case when the left-most non-terminal in a production of a non-terminal is the non-terminal itself( direct left recursion ) or through some other non-terminal definitions, rewrites to the non-terminal again(indirect left recursion). Consider these examples -
(1) A -> Aq (direct)
(2) A -> BqB -> Ar (indirect)
Left recursion has to be removed if the parser performs top-down parsing
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 10
REMOVING LEFT RECURSION
• To eliminate left recursion we need to modify the grammar. Let, G be a grammar having a production rule with left recursion
• A-> Aa
• A->B
• Thus, we eliminate left recursion by rewriting the production rule as:
• A->BA’
• A’->aA’
• A’->c
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 11
3) LEFT FACTORING
Left factoring is removing the common left factor that appears in two productions of the same non-terminal. It is done to avoid back-tracing by the parser. Suppose the parser has a look-ahead ,consider this example-
A -> qB | qCwhere A,B,C are non-terminals and q is a sentence. In this case, the parser will be confused as to which of the two productions to choose and it might have to back-trace. After left factoring, the grammar is converted to-
A -> qD
D -> B | C
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 12
RECURSIVE DESCENT PARSING
• A recursive descent parser is a kind of top-down parser built from a set of mutually recursive procedures (or a non-recursive equivalent) where each such procedure usually implements one of the productions of the grammar.
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 13
EXAMPLE OF RECURSIVE DESCENT PARSING
Suppose the grammar given is as follows:
E->iE’
E’->+iE’
Program:
E()
{
if(l==‘i’)
{
match(‘i’);
E’();
}
} l=getchar();
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 14
E’()
{
if(l==‘+”)
{
match(‘+’);
match(‘i’);
E’();
}
else
return ;
}
Match(char t)
{
if(l==t)
l=getchar();
else
printf(“Error”);
}
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 15
main()
{
E();
If(l==‘$’)
{
printf(“parsing successful”);
}
}
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 16
PREDICTIVE LL(1) PARSING
• The first “L” in LL(1) refers to the fact that the input is processed from left to right.
• The second “L” refers to the fact that LL(1) parsing determines a leftmost derivation for the input string.
• The “1” in parentheses implies that LL(1) parsing uses only one symbol of input to predict the next grammar rule that should be used.
• The data structures used by LL(1) are 1. Input buffer 2. Stack 3. Parsing table
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 17
• The construction of predictive LL(1) parser is based on two very important functions and those are First and Follow.
• For construction of predictive LL(1) parser we have to follow the following steps:• STEP1: computate FIRST and FOLLOW function.
• STEP2: construct predictive parsing table using first and follow function.
• STEP3: parse the input string with the help of predictive parsing table
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 18
FIRST
If X is a terminal then First(X) is just X!
If there is a Production X → ε then add ε to first(X)
If there is a Production X → Y1Y2..Yk then add first(Y1Y2..Yk) to first(X)
First(Y1Y2..Yk) is eitherFirst(Y1) (if First(Y1) doesn't contain ε)
OR (if First(Y1) does contain ε) then First (Y1Y2..Yk) is everything in First(Y1) <except for ε > as well as everything in First(Y2..Yk)
If First(Y1) First(Y2)..First(Yk) all contain ε then add ε to First(Y1Y2..Yk) as well.
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 19
FOLLOW
• First put $ (the end of input marker) in Follow(S) (S is the start symbol)
• If there is a production A → aBb, (where a can be a whole string) then everything in FIRST(b) except for ε is placed in FOLLOW(B).
• If there is a production A → aB, then everything in FOLLOW(A) is in FOLLOW(B)
• If there is a production A → aBb, where FIRST(b) contains ε, then everything in FOLLOW(A) is in FOLLOW(B)
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 20
EXAMPLE OF FIRST AND FOLLOW
The Grammar
E → TE'
E' → +TE'
E' → ε
T → FT'
T' → *FT'
T' → ε
F → (E)
F → id
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 21
PROPERTIES OF LL(1) GRAMMARS
1. No left-recursive grammar is LL(1)
2. No ambiguous grammar is LL(1)
3. Some languages have no LL(1) grammar
4. A ε–free grammar where each alternative expansion for A begins with a distinct terminal is a simple LL(1) grammar.
Example:S aS a
is not LL(1) because FIRST(aS) = FIRST(a) = { a }
S aS´S´ aS ε
accepts the same language and is LL(1)
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 22
PREDICTIVE PARSING TABLE
Method:
1. production A α:a) a FIRST(α), add A α to M[A,a]
b) If ε FIRST(α):
I. b FOLLOW(A), add A α to M[A,b]
II. If $ FOLLOW(A), add A α to M[A,$]
2.Set each undefined entry of M to error
If M[A,a] with multiple entries then G is not LL(1).
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 23
EXAMPLE OF PREDICTIVE PARSING LL(1) TABLE
The given grammar is as followsS E
E TE´
E´ +E —E ε
T FT´
T´ * T / T ε
F num id
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 24
BOTTOM UP PARSING
Bottom-up parsing starts from the leaf nodes of a tree and works in upward direction till it reaches the root node.
we start from a sentence and then apply production rules in reverse manner in order to reach the start symbol.
Here, parser tries to identify R.H.S of production rule and replace it by corresponding L.H.S. This activity is known as reduction.
Also known as LR parser, where L means tokens are read from left to right and R means that it constructs rightmost derivative.
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 25
EXAMPLE OF BOTTOM-UP PARSER
E → T + E | T
T → int * T | int | (E)
Consider the string: int * int + int
int * int + int T → int
int * T + int T → int * T
T + int T → int
T + T E → T
T + T E → T
E
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 26
SHIFT REDUCE PARSING
• Bottom-up parsing uses two kinds of actions: 1.Shift 2.Reduce
• Shift: Move | one place to the right , Shifts a terminal to the left string ABC|xyz ⇒ ABCx|yz
• Reduce: Apply an inverse production at the right end of the left string If A → xy is a production, then Cbxy|ijk ⇒ CbA|ijk
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 27
EXAMPLE OF SHIFT REDUCE PARSING
|int * int + int shift
int | * int + int shift
int * | int + int shift
int * int | + int reduce T → int
int * T | + int reduce T → int * T
T | + int shift
T + | int shift
T + int | reduce T → int
T + T | reduce E → T
T + E | reduce E → T + E
E |
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 28
OPERATOR PRECEDENCE PARSING
Operator grammars have the property that no production right side
is empty or has two adjacent nonterminals.
This property enables the implementation of efficient operator-precedence parsers.
These parser rely on the following three precedence relations:
Relation Meaning
a <· b a yields precedence to b
a =· b a has the same precedence as b
a ·> b a takes precedence over b
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 29
• These operator precedence relations allow to delimit the handles in the right sentential forms: <· marks the left end, =· appears in
• the interior of the handle, and ·> marks the right end.
• . Suppose that $ is the end of the string, Then for all terminals we can write: $ <· b and b ·> $
• If we remove all nonterminals and place the correct precedence relation:<·, =·, ·> between the remaining terminals, there remain strings that can be analyzed by easily developed parser.
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 30
EXAMPLE OF OPERATOR PRECEDENCE PARSING
id + * $
id ·> ·> ·>
+ <· ·> <· ·>
* <· ·> ·> ·>
$ <· <· <· ·>
For example, the following operator precedence relations can
be introduced for simple expressions:
Example: The input string: id1 + id2 * id3
after inserting precedence relations becomes
$ <· id1 ·> + <· id2 ·> * <· id3 ·> $
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 31
UNIT-IIISyntax Directed Translations
Production Semantic Rule
E->E1+T E.code=E1.code||T.code||’+’
• We may alternatively insert the semantic actions inside the grammar
E -> E1+T {print ‘+’}
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 32
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 33
• We can associate information with a language construct by attaching attributes to the grammar symbols.
• A syntax directed definition specifies the values of attributes by associating semantic rules with the grammar productions.
Syntax Directed Definitions1. We associate information with the programming language
constructs by attaching attributes to grammar symbols.
2. Values of these attributes are evaluated by the semantic rulesassociated with the production rules.
3. Evaluation of these semantic rules:• may generate intermediate codes• may put information into the symbol table• may perform type checking, may issue error messages• may perform some other activities• in fact, they may perform almost any activities.
4. An attribute may hold almost any thing.• a string, a number, a memory location, a complex record.
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 34
Syntax-Directed Definitions and Translation Schemes
1. When we associate semantic rules with productions, we use two notations:• Syntax-Directed Definitions• Translation Schemes
A. Syntax-Directed Definitions:• give high-level specifications for translations• hide many implementation details such as order of evaluation of semantic actions.• We associate a production rule with a set of semantic actions, and we do not say
when they will be evaluated.
B. Translation Schemes:• indicate the order of evaluation of semantic actions associated with a production
rule.• In other words, translation schemes give a little bit information about
implementation details.
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 35
Syntax-Directed Translation
• Conceptually with both the syntax directed translation and translation scheme we• Parse the input token stream
• Build the parse tree
• Traverse the tree to evaluate the semantic rules at the parse tree nodes.
Input string parse tree dependency graph evaluation
order for semantic rules
Conceptual view of syntax directed translation
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 36
Syntax-Directed Definitions
1. A syntax-directed definition is a generalization of a context-free grammar in which:• Each grammar symbol is associated with a set of attributes. • This set of attributes for a grammar symbol is partitioned into two subsets called
• synthesized and • inherited attributes of that grammar symbol.
2. The value of an attribute at a parse tree node is defined by the semantic rule associated with a production at that node.
3. The value of a synthesized attribute at a node is computed from the values of attributes at the children in that node of the parse tree.
4. The value of an inherited attribute at a node is computed from the values of attributes at the siblings and parent of that node of the parse tree.
• Most of the programming languages have a common set of statements that define the control flow of a program.
• These control statements are:Assignment statement: It has a single statement assigning some
expression to a variable.if-then-else statement: It has a condition associated with it.
The control flows either to the then-part or to the else-part.
while-do-loopThe control remains within the loop until a specified condition becomes false.
Block of statementsIt is group of statements put within a begin-end block marker.
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 57
Translation of Case Statements
• It is a unique because the structure contains an expression.
• Control jumps to one of the many alternatives.
• Syntax:
switch (E) {case c1: ……..case cn: ……
default : ……
}
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 58
Postfix translation:
• The postfix notation for an expression E can be defined:
1. If E is a variable or constant, then the postfix notation for E is E itself.
2. If E is an expression of the form E1 op E2, where op is any binary operator, then the postfix notation for E is E’1 E’2 op, where E’1 and E’2 are the postfix notations for E1 and E2, respectively.
3. If E is a parenthesized expression of the form (E1), then the postfix notation for E is the same as the postfix notation for E1.
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 59
Postfix notation
• Postfix notation is a linearized representation of a syntax tree.
• It a list of nodes of the tree in which a node appears immediately after its children.
• the postfix notation of below syntax tree is x a –b* a-b*+=
s
b
*
s
b
Assign
+x
*
uminus uminus
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 60
Translation with a top down parser
• which build parse trees from top(root) to bottom(leaves).
Fig. The procedures of a top down parser.
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 61
Recursive Descent Parsing
• Recursive descent is a top-down parsing technique that constructs the parse tree from the top and the input is read from left to right.
• It uses procedures for every terminal and non-terminal entity.
• A form of recursive-descent parsing that does not require any back-tracking is known as predictive parsing.
• This parsing technique is regarded recursive as it uses context-free grammar which is recursive in nature.
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 62
Back-tracking
• Top- down parsers start from the root node (start symbol) and match the input string against the production rules to replace them (if matched).
• To understand this, take the following example of CFG:
S → rXd | rZd
X → oa | ea
Z → ai
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 63
Back tracking
• Now the parser matches all the input letters in an ordered manner.
• The string is accepted.
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 64
Predictive Parser
• Predictive parser is a recursive descent parser.
• It has the capability to predict which production is to be used to replace the input string.
• The predictive parser does not suffer from backtracking.
• The predictive parser puts some constraints on the grammar and accepts only a class of grammar known as LL(k) grammar.
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 65
PREDICTIVE PARSER
• The parser refers to the parsing table to take any decision on the input and stack element combination.
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 66
LL Parser
• An LL Parser accepts LL grammar.
• LL grammar is a subset of context-free grammar but with some restrictions to get the simplified version.
• LL grammar can be implemented by means of both algorithms namely, recursive-descent or table-driven.
• LL parser is denoted as LL(k).
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 67
Array references in arithmetic expressions:
• Array elements can be accessed quickly if they are stored in a block of consecutive locations.
• Elements are numbered 0, 1,…..,n-1, for an array with n elements.
• If the width of each array element is w, then the ith
element of array A begins in location.
base + i * w
where base relative address(storage allocated)
i. e , base is the relative address of A[0].
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 68
Layouts for a 2D Array
A[1, 1]
A[1, 2]
A[1, 3]
A[2, 1]
A[2, 2]
A[2, 3]
First row
Second row
First column
Third Column
Second Column
A[1, 1]
A[2, 1]
A[1, 2]
A[2, 2]
A[1, 3]
A[2, 3]
(a) Row Major (b) Column Major
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 69
Procedures call
• It is imperative for a compiler to generate good code for procedure calls and returns.
• The run-time routines that handle procedure argument passing, calls and returns are part of the run-time support package.
• Let us consider a grammar for a simple procedure call statement:
• (1) S call id ( Elist )
• (2) Elist Elist , E
• (3) Elist E
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 70
Declarations and case statements.
• Declarations
Declarations with lists of names can be handled as follow:
D T id ; D | ε
T B C | record ’{’ D ’}’
B int | float
C ε | [ num ] C
Nonterminal D generates a sequence of declarations. Nonterminal T generates basic, array, or record types. Nonterminal B generates one of the basic types int or float. Nonterminal C, for “ component,” generates string of Zero or more integers, each surrounded by brackets.
ANKUR SRIVASTAVA ASSISTANT PROFESSOR JIT 71
Case Statements
• The “switch” or “case” statement is available in a variety of languages. The switch-statement