Amortized Analysis

CS333 / Cutler Amortized Analysis

1

Amortized AnalysisThe average cost of a sequence of n operations on a given Data

Structure.

Aggregate Analysis

Accounting Method

Potential Method


2

Amortized Analysis

• Amortized analysis computes the average time required to perform a sequence of n operations on a data structure

• Often worst case analysis is not tight and the amortized cost of an operation is less that its worst case.


3

Applications of amortized analysis

• Vectors/ tables

• Disjoint sets

• Priority queues

• Heaps, Binomial heaps, Fibonacci heaps

• Hashing


4

Difference between amortized and average cost

• To do averages we need to use probability

• For amortized analysis no such assumptions are needed

• We compute the average cost per operation for any mix of n operations


5

Operations on Data Structures

• A data structure has a set of operations associated with it.

• Example: A stack with– push(), pop() and MultiPop(k).

• Often some operations may be slow while others are fast.

• push() and pop() are fast.• MultiPop(k) may be slow. • Sometimes the time of a single operation

can vary


6

Methods

• Aggregate analysis- the total amount of time needed for the n operations is computed and divided by n

• Accounting - operations are assigned an amortized cost. Objects of the data structure are assigned a credit

• Potential – The prepaid work (money in the “bank”) is represented as “potential” energy that can be released to pay for future operations


7

Aggregate analysis

• n operations take T(n) time

• Amortized cost of an operation is T(n)/n


8

Stack - aggregate analysis

• A stack with operations

• Push, Pop and Multipop.

Multipop(S, k)

while not empty(S) and k>0 do

Pop(S);

k:=k-1

end while


9


• Push and Pop are O(1) (to move 1 data element)

• Multipop is O(min(s, k)) where

s is the size of the stack and

k the number of elements to pop.

• Assume a sequence of n Push, Pop and Multipop operations


10


• Each object can be popped only once for each time it is pushed

• So the total number of times Pop can be called ( directly or from Multipop) is bound by the number of Pushes <=n.


11


• A sequence of n Push and Pop operations is therefore O(n) and the amortized cost of each is O(n)/n=O(1). All operations have same amortized cost.


12

Stack Example: Op/Moves <=2

Start

push a a

push b ab

push c abc

Multipop(3)ab

pop b a

pop a

Operation Stack

a

ab

abc

Stack

pop c

6 Operation: 6 Moves 4 Operation: 6 Moves


13

Accounting Method

• Charge each operation an (invented) amortized cost. – Often different from actual run time cost. Some

operations may have an amortized cost larger than runtime, others may have less.

– Unlike businesses we do not want to make a profit– We want to cover the actual cost

• Amount charged but not used in performing an operation is stored with objects of the data structure


14

Accounting method

• Later operations can use stored amount to pay for their actual cost

• Credit balance must not go negative (always enough to pay for performance of future operations)


15

Stack - amortized analysis

• We assign the amortized costs:

$2 for Push

$0 for both Pop and Multipop

• For a sequence of n Push and Pop operations the total amortized cost is at most 2n or O(n)


16

Stack - amortized analysis

• Each time we do a Push we pay $1 for the actual cost of the Push and the element has a credit of $1.

• Each time an element is popped we take the $1 credit to pay for it

• Thus the balance is always nonnegative


17

k bit binary counter

Increment (A) i = 0 while i < length[A] and

A[i] = 1 A[i] = 0 i ++ if i < length[A] A[i] = 1

• Initially the counter contains 0

• Eventually it becomes 2k -1

• Next it is reset to 0

1 0k -1

A


18

Aggregate analysis

• Count number of times a bit is flipped.

• Let number increments n = 2k (if n < 2k analysis similar)

• A[0] flipped n times• A[1] flipped n/21 times• …• A[k - 1] flipped n/2 k-1 times

Bit 2 1 0

D No.0 0 0 01 0 0 12 0 1 0 3 0 1 1

4 1 0 0 5 1 0 1 6 1 1 0 7 1 1 1Flips 2 4 8

3 bit counterk=3

nnn

ii

k

ii

22

1

2 0

1

0


19

Accounting method

• Charge amortized cost of $2 to set a bit to 1• When a bit is set to 1, pay $1 for actual cost

and store $1 with bit• Note: at all times a bit with value 1 has $1• When a bit is reset to 0 use $1 to pay for

actual cost• $2 per Increment operation


20

Accounting method

• Let the value stored in the counter be:

• After increment and a payment of

$2 = $1 +$1:

1 1 0 0 1 0 0 1 1 1 $1 $1 $0 $0 $1 $0 $0 $1 $1 $1

$1 $1 $0 $0 $1 $0 $1 $0 $0 $0 1 1 0 0 1 0 1 0 0 0


21

Dynamic table (object table, hash table, vector, etc)

• The table is dynamic

• We can’t predict its maximum size

• We would like to avoid allocating a lot of unused space (reasonable load balance)

• May not be able to avoid table overflow.

• Overflow should not cause run time failure

Amortized Analysis

Documents

amortized costs

invented amortized cost

amortized cost larger

operations push

sequence of n operations

set of operations

naccounting operations

actual run time cost