Lecture 2

Lecture 2

1

Recurrence Relation• Expresses the value of a function f for an argument n in terms of the

values of f for arguments less than n.• Is an equation or inequality that describes a function in terms of its

values on smaller inputs.• Example: T(n) = 9 if n=1

= 2T(n-2)+2n if n>1• Examples for familiar algorithms:

– Sequential Search T(n)= T(n-1)+1– Selection Sort T(n)= T(n-1)+n– Binary Search T(n)= T(n/2)+1– Recursive Factorial T(n)= T(n-1)+1

Recurrence solution methodsI. The Substitution Method: involves:-

• Guessing the form of the solution.• Using mathematical induction showing that the solution works.

• It is powerful method, but applied only in cases when it is easy to guess the form of the solution.

2

Recurrence Relation…. (continued)

• Example: T(n)=2T(n/2)+n, we guess T(n)= O(n logn), we want to prove that T(n)<= c.n logn for c>0 (By definition of O-notation).

– Assume it holds for n/2. i.e., T(n/2)<= c.n/2 log(n/2).

– Substituting: T(n)<= 2.c.n/2 log (n/2)+n <=c.n.log(n/2)+n= c.n.logn-c.n.log2+n =c.n.logn-cn+n<=c.n.logn for c>=1

• No general way to guess the correct solutions to recurrences.

• Guessing needs experience and occasionally creativity.

• Example: T(n)= 2T(n/2+17)+n, we guess T(n)= O(n logn), because for large n the difference between T(n/2) and T(n/2+17) is not large.

• Example: T(n)= T(n/2)+T(n/2)+1, we guess T(n)=O(n), we want to show that T(n)<= c.n, for c>0.

– Assume it holds for n/2, i.e., T(n/2)<=c.n/2

– Substituting: T(n)<= c(n/2)+c(n/2)+1= cn+1 doesn’t imply T(n)<= c.n.– Solution: Subtract a lower order term from the previous guess. New guess is

T(n)<= c.n-b.• So, T(n)<= (c.n/2-b)+(c.n/2-b)+1= c.n-2b+1<= c.n-b, b>=1.

3


• Changing variables: Example: T(n)= 2T(sqrt(n))+logn (this looks difficult)– Simplifying the recurrence, let m= logn

T(2m)= 2(2m/2)+mLet S(m)= T(2m)S(m)= 2S(m/2)+m, we guess S(m)= O(m logm).

Changing back from S(m) to T(n), T(n)= T(2m)=S(m)=O(m logm)= O(logn log logn).

• Forward Substitution: – Start with initial term(s) for initial conditions.– Generate several terms.– Look for a pattern and express it in closed form.Example: T(n)= 2T(n-1)+1 for n>1, T(1)=1. T(1)= 1

T(2)= 2T(1)+1= 3, T(3)= 2T(2)+1= 7, T(4)= 2T(3)+1= 15, T(5)= 2T(4)+1= 31, --------

So, T(n)= 2n-1 (One less than consecutive powers of two (2, 4, 8, 16, 32, ----)).

4


II. The Iteration Method: • NO need to guess the solution.• Needs more algebra than the substitution method.• Simply iterate (expand) the recurrence and express it as a

summation of terms dependent only on n and the initial conditions.Example: T(n)= T(n-1)+n for n>1, T(0)= 0, T(1)= 1.T(n)= T(n-1)+n = [T(n-2)+n-1]+n= T(n-2)+(n-1)+n = [T(n-3)+n-2]+(n-1)+n= T(n-3)+(n-2)+(n-1)+n after i Substitutions---- = T(n-i)+(n-i+1)+(n-i+2)+------+n ----to the initial condition = T(0)+1+2+3+-------+n= n(n+1)/2Example: T(n)= T(n-1)+2n-1, T(0)= 0. T(n)= T(n-1)+2n-1

= [T(n-2)+2(n-1)-1]+2n-1= T(n-2)+2(n-1)+2n-2= [T(n-3)+2(n-2)-1]+2(n-1)+2n-2= T(n-3)+2(n-2)+2(n-1)+2n-3

Similarly, it continues---------- 5


at ith iteration

=T(n-i)+2(n-i+1)+2(n-i+2)+ ------ + 2n-i

at nth iteration

= T(n-n)+2(n-n+1)+2(n-n+2)+ ------ + 2n-n

= 0+2+4+ ----- +2n-n

= 2n(n+1)/2-n= n2

Recursion Trees: Way to visualize what happens when a recurrence is iterated. It diagrams

the tree of recursive calls, and the amount of work done at each call.

Example: T(n)= T(n/3)+T(2n/3)+n

6


Example: T(n)= 2T(n/2)+n2

III. The Master Method• Another method for solving recurrences of the form T(n)= aT(n/b)+f(n),

where a>=1 and b>1 are constants and f(n) is asymptotically positive function.

• Requires memorization of three cases.• Can solve many recurrences quite easily.

T(n)= aT(n/b)+f(n) means:• The algorithm divides a problem of size n into a sub problems.• Each sub problem has size of n/b, a and b are positive constants. • The a sub problems are solved recursively each in time T(n/b).• f(n) describes the cost of dividing the problem and combining the results

of the sub problems.

7


Master Theorem: For Master method:• Let a>=1 and b>1 be constants, let f(n) be a function, and let T(n) be

defined on the non-negative integers by the recurrence T(n)= aT(n/b)+f(n), then T(n) can be bounded asymptotically as follows:

1. If f(n)= O(n logba-€) for some constant €>0, then T(n)= (nlog

ba).

2. If f(n)= (nlogba), then T(n)= (nlog

ba log n).

3. If f(n)= (n logba+€) for some constant €>0 and if af(n/b)<=c.f(n) for some

constant c<1 and all large n, then T(n)= (f(n)).

Example 1: T(n)= 9T(n/3)+n. We have a=9, b=3, f(n)= n, thus nlogba= nlog

39=

(n2).

Since f(n)= O(n log39-€), where €=1, we can apply case 1, So, T(n)= (n2).

Example 2: T(n)= T(2n/3)+1, we have a=1, b=3/2, f(n)=1, thus nlogba=

nlog(3/2)

1=no=1.

f(n)= (nlogba)= (1), so, T(n)= (log n), by case 2 of the master theorem.

Example 3: T(n)= 3T(n/4)+n logn, we have a=3, b=4, f(n)=n logn and nlogba=

nlog43=O(n0.793).

f(n)= (n log43+€) , where € is approximately =0.2, case 3 applies.

For large n af(n/b)=3(n/4) log(n/4)<= (3/4)n log n= c.f(n) for c=3/4. So, T(n)= (n logn). 8

Linked Lists• Non-contiguous list (as versus contiguous array structure). Eleme

nts are linked by pointers. • But it is a linear/sequential structure elements are organized in 1

dimension.

• Linked lists have desirable properties: – Dynamic size - allocate enough "nodes" as needed – Ease of insertion/deletion - new nodes can be inserted in by simple

pointer reassignment (O(1)), and no need for expensive (O(n)) shifting elements operation, thus efficient.

– There are several variations of linked lists. But the most basic ones are: • Singly-linked list• Doubly-linked list • Circular Linked list

9

Linked List … (continued)

1. Singly-linked ListsClass Node (for singly-linked lists)• Each node in a list consists of at least two parts:

– data – pointer to the next node

• Example: class Node whose element type (for data) is int struct Node { //data members int data; Node *next; };Typical Operations• Building a linked list (using dynamically allocating Node objects)

int main() { Node *p1 = new Node(3); Node *p2 = new Node(1); Node *p3 = new Node(5);

p1->next = p2; // p1's next points to where p2 points to p2->next = p3; // p2's next points to where p3 points to

Node *head = p1; // the beginning of the list } 10


• Traversal (for printing) for (Node *ptr = head; ptr != NULL; ptr = ptr->next) cout << ptr->data << ' ';

• Insertion, Deletion operations: Read your Data Structure and Algorithms course materials.

2. Doubly-linked Lists• In a doubly-linked list, every node is connected with its next and previou

s nodes. • With doubly-linked lists, we can traverse the list backwards (as well as fo

rwards) easily. • A doubly-linked list is often implemented with header and tail nodes.Class Node (for doubly-linked lists)• The node needs two pointers: next and prev. • Example: Revised class Node whose element type (for data) is int

struct Node { // data members Node *prev, *next; int data; };

11

Typical Operations• List Initialization

head = new Node; tail = new Node;

head->next = tail; tail->prev = head; Traversal (forward, for printing) Node* ptr; for (ptr = head->next; // start from the node after head

ptr != tail; // when ptr points to tail, terminate.

ptr = ptr->next) //advance ptr to the next node cout << ptr->data <<

' '; • Insertion, Deletion operations: Read your Data Structure and

Algorithms course materials.3. Circular Linked Lists The last node in a list keeps a link back to the first. Can also be done with doubly linked lists (The first node’s previous link

is to the last node), (The last node’s next link is to the first node).


12

• There are many times where it is useful to use a list in a way where we always add to the end, and also always remove from the end.

• The last element put into the list will be the first element we take out of the list

– Last-In, First-Out ("LIFO")• stack: a more restricted List with the following constraints:

– Elements are stored by order of insertion from "bottom" to "top“.

– Items are added to the top.– Only the last element added onto the stack (the top

element) can be accessed or removed.• Goal: every operation on a stack should be O(1).

– Stacks are straightforward to implement in several different ways, and are very useful.

• Operations on a stack– push: add an element to the top.

– pop: remove and return the element at the top.

Abstract data type: Stack

13

– peek: return (but not remove) top element;

pop or peek on an empty stack causes an exception.

– Other operations: isEmpty, size

push(a)

push(b)

pop()

Stack … (continued)

14

Stack features:• ORDERING: maintains order elements were added

(new elements are added to the end by default).• OPERATIONS:

– Add element to end of list ('push')– Remove element from end of list ('pop')– Examine element at end of list ('peek')– Clear all elements– is empty, get size– All of these operations are efficient! O(1).

Stacks in computer science• the lowly stack is one of the most important data structures in all of

computer science– Function/method calls are placed onto a stack.– Compilers use stacks to evaluate expressions.– Stacks are great for reversing things, matching up related pairs of

things, and backtracking algorithms.

Stack … (continued)

15

Stack and Queue

– Stack programming problems: Reverse letters in a string, reverse words in a line, or reverse a

list of numbers. Find out whether a string is a palindrome. Examine a file to see if its braces { } and other operators match.

Abstract data type: Queue• Many times, we use a list in a way where we always add to the end, and

always remove from the front.• The first element put into the list will be the first element we take out of

the list:– First-In, First-Out ("FIFO")

• Queue: a more restricted List with the following constraints:– Elements are stored by order of insertion from front to back.– Items can only be added to the back of the queue.– Only the front element can be accessed or removed.

• Goal: every operation on a queue should be O(1).

16

Queue … (continued)

Operations on a queue

• Offer or enqueue: add an element to the back.

• Remove or dequeue: remove and return the element at the front.

• peek: return (but not remove) front element:

– peek on an empty queue returns null.

• Other operations: isEmpty, size.

Queue features• ORDERING: maintains order elements were added (new elements

are added to the end by default).

17

Queue and Graph … (continued)• OPERATIONS:

– Add element to end of list ('offer' or 'enqueue').– Remove element from beginning of list ('remove' or 'dequeue')

examine element at beginning of list ('peek').– Clear all elements.– is empty, get size.– All of these operations are efficient! O(1).

• Revise your Data Structure and Algorithms course materials for the implementations of Queue and Stack.

Graph• A graph is a mathematical structure consisting of a set of vertices

and a set of edges connecting the vertices.Formally: G = (V,E), where V is a set and E is subset of V X V .

• G = (V,E) undirected if for all v,w € V :(v,w) € E < == > (w, v) € E.

Otherwise directed.A directed graph: G = (V,E) with vertex set V = {0,1,2,3,4,5, 6} and

edge set. 18

Graph …(continued)

• E = {(0, 2), (0, 4), (0, 5), (1, 0), (2, 1), (2, 5), (3, 1), (3, 6), (4, 0), (4, 5), (6, 3), (6, 5)}

19


An undirected graph

20


Examples of graphs in “real life"

• Road Maps.Edges represent streets and vertices represent crossings.

21


Examples:

• Computer Networks, Vertices represent computers and edges represent network connections (cables) between them.

– For example, we might be using a graph to represent a computer network (such as the Internet), and we might be interested in finding the fastest way to route a data packet between two computers.

• The World Wide Web. Vertices represent web pages, and edges represent hyperlinks.

• Flowcharts. Vertices represent boxes and edges represent arrows.

22


Example:

The adjacency list data structure

• Array with one entry for each vertex v, which is a list of all vertices adjacent to v.

• Example:

23


Sparse and dense graphs

24


• General strategy:

1. Let v be an arbitrary vertex.

2. Visit all vertices reachable from v.

3. If there are vertices that have not been visited, let v be such a vertex and go back to (2).

Weighted Graph• A weighted graph is a graph that has a numerical level associated with each

edge e, called the weight of edge e.

• Edge weights can be integers, rational numbers, or real numbers, which represent a concept such as distance, connection costs, or affinity.

• Example: Figure in the next slide.

Unweighted Graph

• An unweighted graph is a graph that has not a numerical level associated with each edge e.

25


26

Tree• In computer science, a tree is a widely-used data structure that emulates a

hierarchical tree structure with a set of linked nodes. • An acyclic connected graph where each node has zero or more children nod

es and at most one parent node.

• Furthermore, the children of each node have a specific order.

27

Tree …(continued)

• A node is a structure which may contain a value, a condition, or represent a separate data structure (which could be a tree of its own).

• Each node in a tree has zero or more child nodes.• A node that has a child is called the child's parent node (or ancestor node,

or superior). A node has at most one parent. • Nodes that do not have any children are called leaf nodes (terminal

nodes).

• The height of a node is the length of the longest downward path to a leaf

from that node. The height of the root is the height of the tree. • The depth of a node is the length of the path to its root (i.e., its root path).

• All other nodes can be reached from the root node by following edges or lin

ks. • An internal node or inner node is any node of a tree that has child nodes

and is thus not a leaf node.

• A subtree of a tree T is a tree consisting of a node in T and all of its descendants in T.

28

Tree …(continued)

• The subtree corresponding to the root node is the entire tree.

• The sub tree corresponding to any other node is called a proper subtree.

Binary Tree • Each of the nodes in a binary tree contains two pointers, typically called left

and right.

• In addition to these pointers, of course, the nodes can contain other types of data.

struct TreeNode {

int item; //The data in this node.

TreeNode *left; // Pointer to the left subtree.

TreeNode *right; // Pointer to the right subtree.

} • The left and right pointers in a TreeNode can be NULL or can point t

o other objects of type TreeNode. 29

Binary Tree …(continued)

• Here is a function that prints all the items in a Binary Tree on one line of out

put:

30

Binary Tree …(continued)

void preorderPrint( TreeNode *root ) {

// Print all the items in the tree to which root points.

// The item in the root is printed first, followed by the

// items in the left subtree and then the items in the

// right subtree.

if ( root != NULL ) { // (Otherwise, there's nothing to print.)

cout << root->item << " "; // Print the root item.

preorderPrint( root->left ); // Print items in left subtree.

preorderPrint( root->right ); // Print items in right subtree.

}

} // end preorderPrint() • Exercise: Write two functions inorderPrint(TreeNode *root) and

postorderPrint(TreeNode *root) that print all the items in a Binary Tree on on

e line of output. 31

Tree …(continued)

Binary Search Tree (BST)• An important special kind of binary tree is the binary search tree (BST).

• In a BST, each node stores some information including a unique key value,

and perhaps some associated data. • A binary tree is a BST if and olny if, for every node n in the tree:

– All keys in n's left subtree are less than the key in n, and – All keys in n's right subtree are greater than the key in n.

• Here are some BSTs in which each node just stores an integer key:

32

BST Tree …(continued)

• These are not BSTs:

• In the left one 5 is not greater than 6. In the right one 6 is not greater than 7.

• Note that more than one BST can be used to store the same set of key values. For example, both of the following are BSTs that store the same set of integer keys:

33

Tree …(continued)

• The reason binary-search trees are important is that the following operations can be implemented efficiently using a BST: – Insert a key value.– Determine whether a key value is in the tree. – Remove a key value from the tree.

– Print all of the key values in sorted order. AVL Trees (Adelson-Velskii and Landis)• An AVL tree (also called an "admissible tree") is a tree in which the height of the left a

nd right subtrees of every node differ by at most one - referred to as "height-balanced".

• Example AVL trees: • AVL Tree is BST.• The simplest idea is to require

that the left and right sub trees have the same height (approximately).

34

AVL Trees …(continued)

• Example non-AVL Trees:

• In order to indicate the differences between the heights of the right and left subtrees of a given (root) node, a balance factor is defined for that node of the subtree.

• We define the balance factor, BF:

BF = (height of right subtree - height of left subtree). So, BF = -1, 0 or +1 for an AVL tree.

• Balance factors for example AVL trees (node key values not shown):

35


• Balance factors for an example non-AVL tree (node key values not shown):

• When the AVL property is lost we can rebalance the tree via one of four rotations:

a) Single Right Rotation (SRR):

• A is the node that the rotation is performed on. 36


• This rotation is performed when A is unbalanced to the left (the left subtree is 2 higher than the right subtree) and B is left-heavy (the left subtree of B is 1 higher than the right subtree of B). T1, T2 and T3 represent subtrees (a node was added

to T1 which made B leftheavy and unbalanced A).

b) Single Left Rotation (SLR): A is the node that the rotation is performed on. This rotation is performed when A is unbalanced to the right (the right subtree is 2 higher than the left subtree) and B is right-heavy (the right subtree of B is 1 higher than the left subtree of B). T1, T2 and T3 represent subtrees (a node was added to T3 which made B right-heavy and unbalanced A).

37


c) Double Left Rotation (DLR): C is the node that the rotation is performed on. This rotation is performed when C is unbalanced to the left (the left subtree is 2 higher than the right subtree), A is right-heavy (the right subtree of A is 1 higher than the left subtree of A) and B is left-heavy. T1, T2, T3, and T4 represent subtrees (a node was added to T2 which made B left-heavy, made A right-heavy and unbalanced C). This consists of a single left rotation at node A, followed by a single right rotation at node C.

38


d) Double Right Rotation (DRR): A is the node that the rotation is performed on. This rotation is performed when A is unbalanced to the right (the right subtree is 2 higher than the left subtree), C is leftheavy (the left subtree of C is 1 higher than the right subtree of C) and B is right-heavy. T1, T2, T3, and T4 represent subtrees (a node was added to T3 which made B right-heavy, made C left-heavy and unbalanced A). This consists of a single right rotation at node C, followed by a single left rotation at node A.

39


Insertion in a AVL-tree• An AVL tree may become out of balance in two basic situations:

– After inserting a node in the right sub tree of the right child. – After inserting a node in the left sub tree of the right child.

• Insertion of a node in the right sub tree of the right child: This involves a SLR about node P:

40


Example:

41


• Insertion of a node in the left sub tree of the right child:

This involves a DRR:

• In each case the tree is rebalanced locally (since there is no need to modify the balance factors higher in the tree).

• Since rebalancing is local, insertion requires one single or one double rotation, i.e., O(constant) for rebalancing.

• Experiments have shown that 53% of insertions do not bring the tree out of balance.

42


Example:

43


AVL Tree Examples

1) Consider inserting 46 into the following AVL Tree:

44


• Initially, using the standard binary search tree insert, 46 would go to the right of 44. Now, let's trace through the rebalancing process from this place.

• First, we check to see if the node is balanced. (This simply look up the heights of the left and right sub trees, and decide if the difference is more than 1.) In this case, the node is balanced, so we march up to the parent node, that stores 44.

• Similarly, we decide that the nodes storing 44, 40 and 48 are balanced as well. Finally, when we reach the root node storing 32, we realize that our tree is imbalanced.

• Thus, the restructuring occurs on the nodes containing the 32, 48 and 40. After restructuring, the tree looks as follows:

45


2) Example, consider inserting 61 into the following AVL Tree:

46


• Tracing through the tree, we find the first place an imbalance occurs tracing up the ancestry of the node storing 61 is at the node storing 56.

• Using our re-structuring algorithm, we rearrange the tree as follows:

47


3) Example, we will delete the node storing 8 from the AVL tree below:

48


• Tracing through the tree, we find that we must rebalance the parent of the deleted node, which stores 16. This node needs rebalancing and gets restructured as follows:

49


• Next, we march up to the parent of the node storing 24, the node storing 32.

• This node is imbalanced. The reason for this is that the restructuring of the node with a 16 reduced the height of that sub tree. By doing so, there was in INCREASE in the difference of height between the sub trees of the old parent of the node storing 16. This increase could propagate an imbalance in the AVL tree.

• When we restructure at the node storing 32, we get the final tree as follows:

50


4). Example, we will delete the node storing 4 from the AVL tree below:

• We do NOT find an imbalance at an ancestral node until we get to the root node of the tree. Accordingly, we restructure the tree as follows:

51

Lecture 2

Documents

right subtree of b

right subtree of c

left subtree of b

left subtree of c

ns right subtree

right subtrees

b rightheavy

node n