Approximation Algorithms Part One: Constant factor approximations

Advanced Algorithms – COMS31900

Approximation algorithms part one

Constant factor approximations

Benjamin Sach

NP-completeness recap

A problemA is NP-complete if

NP is the class of decision problems we cancheck the answer to in polynomial time

A is in NP

EveryB in NP has a polynomial time reduction toA

(this second part is the definition of NP-hard)




A is in NP



‘yes/no’ problems




A is in NP






A is in NP



we can solveBusingA




A is in NP






A is in NP


If we could solveA quickly we could solve every problem in NP quickly





A is in NP



They are the ‘hardest’ problems in NP





A is in NP





that you can’t solve them in polynomial time (i.e. that P 6= NP)Most computer scientists (I’ve met) believe




A is in NP





A polynomial time algorithm for an

NP-complete problem is worth

(a lot more than) a million dollars




A is in NP








A is in NP




So if a problem is NP-complete, we give up right?


Bin packing

1

1

4/8 4/87/8

3/82/82/8

Bin packing

Bins

1

1

4/8 4/87/8

3/82/82/8

Bin packing

Bins

1

Items

1

4/8 4/87/8

3/82/82/8

Bin packing

Bins

1

Items

1

4/8 4/87/8

3/82/82/8

0 < |Item| 6 1

Bin packing

Bins

1

Items

1

4/8 4/87/8

3/82/82/8

0 < |Item| 6 1

I is the sum of all item sizes

Bin packing

Bins

1

Items

1

4/8 4/87/8

3/82/82/8

0 < |Item| 6 1

|Bin| = 1 and there is an unlimited number of bins. . .

I is the sum of all item sizes

Bin packing

1

1

4/8 4/87/8

3/82/82/8

Problem pack all items into the fewest possible bins

Bin packing

1

1

4/8 4/87/8

3/82/82/8


This is an example of an optimisation problem

Bin packing

1

1


4/8 4/87/8

2/82/8

3/8


Bin packing

1

1

4/8 4/87/8

3/82/82/8



Bin packing

1

1


4/8

4/8

7/8

2/8

2/8

3/8


Bin packing

1

1


4/8

4/8

7/8

2/8

2/8

3/8

Bin packing

1

1


4/8

4/8

7/8

2/8

2/8

3/8

The BINPACKING problem is known to be NP-hard

Bin packing

1

1


4/8

4/8

7/8

2/8

2/8

3/8


and the decision version. . . “Can you pack the items into at most k bins?”

is NP-complete

Bin packing

1

1


4/8

4/8

7/8

2/8

2/8

3/8


and the decision version. . . “Can you pack the items into at most k bins?”

is NP-complete

In the decision version,k is part of the input

Bin packing

1

1


4/8

4/8

7/8

2/8

2/8

3/8


Bin packing

1

1


4/8

4/8

7/8

2/8

2/8

3/8

The BINPACKING problem is known to be NP-hardbut fortunately we can approximate

Next fit

1

1

4/8 4/87/8

3/82/82/8

If item i fits into bin j: pack it, i++; else j++;

Next fit

1

1

4/8 4/87/8

3/82/82/8


Next fit

1

1

4/87/8

3/82/82/8


4/8

Next fit

1

1

4/87/8

3/82/82/8


4/8

Next fit

1

1

4/87/8

3/82/8


4/8

2/8

Next fit

1

1

4/87/8

3/82/8


4/8

2/8

Next fit

1

1

4/87/8

3/82/8


4/8

2/8

Next fit

1

1 7/83/82/8


4/8

2/8

4/8

Next fit

1

1 7/83/82/8


4/8

2/8

4/8

Next fit

1

1 7/83/82/8


4/8

2/8

4/8

Next fit

1

1

3/82/8


4/8

2/8

4/87/8

Next fit

1

1

3/82/8


4/8

2/8

4/87/8

Next fit

1

1

3/82/8


4/8

2/8

4/87/8

Next fit

1

1

3/8


4/8

2/8

4/87/8

2/8

Next fit

1

1

3/8


4/8

2/8

4/87/8

2/8

Next fit

1

1


4/8

2/8

4/87/8

2/8

3/8

Next fit

1

1

4/8

2/8

4/87/8

2/8

3/8

Next fit

1

1

4/8

2/8

4/87/8

2/8

3/8

Next fit runs inO(n) time but how good is it?

Next fit

1

1

4/8

2/8

4/87/8

2/8

3/8


where n is the number of items

Next fit

1

4/8

2/8

4/87/8

2/8

3/8


Next fit

1

4/8

2/8

4/87/8

2/8

3/8


Let fill(i) be the sum of item sizes in bin i

and s be the number of non-empty bins (using Next fit)

Next fit

1

4/8

2/8

4/87/8

2/8

3/8



Observe that fill(2i− 1) + fill(2i) > 1 (for 1 6 2i 6 s)


Next fit

1

4/8

2/8

4/87/8

2/8

3/8





so bs/2c <∑

162i6s

fill(2i− 1) + fill(2i)

Next fit

1

4/8

2/8

4/87/8

2/8

3/8





so bs/2c <∑

162i6s

fill(2i− 1) + fill(2i) 6 I

Next fit

1

4/8

2/8

4/87/8

2/8

3/8





so bs/2c <∑

162i6s

fill(2i− 1) + fill(2i) 6 I

the sum of theitem weights

Next fit

1

4/8

2/8

4/87/8

2/8

3/8





so bs/2c <∑

162i6s

fill(2i− 1) + fill(2i) 6 I 6 Opt


Next fit

1

4/8

2/8

4/87/8

2/8

3/8





so bs/2c <∑

162i6s



the optimal number of bins

Next fit

1

4/8

2/8

4/87/8

2/8

3/8





so bs/2c <∑

162i6s


Next fit

1

4/8

2/8

4/87/8

2/8

3/8





so bs/2c <∑

162i6s


therefore s 6 2 ·Opt

Next fit

1

4/8

2/8

4/87/8

2/8

3/8





so bs/2c <∑

162i6s


therefore s 6 2 ·Opt in other words the Next Fit is never worse than twice the optimal

Next fit

1

4/8

2/8

4/87/8

2/8

3/8





so bs/2c <∑

162i6s


therefore s 6 2 ·Opt

Approximation Algorithms

An algorithmA is an α-approximation algorithm for problem P if,

◦A runs in polynomial time

◦A always outputs a solution with value s

within an α factor of Opt





Here P is an optimisation problem with optimal solution of value Opt







• If P is a maximisation problem, Optα 6 s 6 Opt









• If P is a minimisation problem (like BINPACKING), Opt 6 s 6 α ·Opt









We have seen a 2-approximation algorithm for BINPACKING










the number of bins used, s is always between Opt and 2 ·Opt










the number of bins used, s is always between Opt and 2 ·Opt

In the examples we consider, α will be a constant but it could depend on n (the input size)

1

4/8

2/8

4/87/8

2/8

3/8

We have seen that Next fit is a 2-approximation algorithm for Bin packingwhich runs inO(n) time

can we do better?

First fit decreasing (FFD)

1

1

4/8 4/87/8

3/82/82/8


1

1

Step 1: Sort the items into non-increasing order

4/8 4/87/8

3/82/82/8


1

1

4/8 4/87/8

3/8 2/8 2/8

Step 1: Sort the items into non-increasing order


1

1

4/8 4/87/8

3/8 2/8 2/8

Step 2: Put each item in the first (left-most) bin it fits in


1

1

4/8 4/8 3/8 2/8 2/8


7/8


1

1

4/8 3/8 2/8 2/8


7/84/8


1

1

3/8 2/8 2/8


7/84/8

4/8


1

1

2/8 2/8


7/84/8

4/8

3/8


1

1

2/8


7/84/8

4/8

3/8

2/8


1

1


7/84/8

4/8

3/8

2/8

2/8


1

1


7/84/8

4/8

3/8

2/8

2/8

this will be important

for the proof


1

1


7/84/8

4/8

3/8

2/8

2/8


1

1

7/84/8

4/8

3/8

2/8

2/8


1

1

7/84/8

4/8

3/8

2/8

2/8

FFD runs inO(n2) time but how good is it?


1 7/84/8

4/8

3/8

2/8

2/8

FFD runs inO(n2) time but how good is it?


Consider bin j =⌈2s3

⌉(s is the number of bins FFD uses on this input)

ss




j

ss




j

ss




Case 1: Bin j contains an item of size> 1/2

j

ss





j

ss





Every bin j′ 6 j contains an item of size> 1/2

j

ss






because we packed big things first and each thing was

packed in the lowest numbered bin

j

ss






because we packed big things first and each thing was

packed in the lowest numbered bin

j

ss






j

ss






each of these items has to be in a different bin (even in Opt)

j

ss







So Opt uses at least 2s3 bins

j

ss







So Opt uses at least 2s3 bins

or. . .s 63Opt

2

j

ss




Case 2: Bin j contains only items of size 6 1/2

j

ss





when FFD packed the first item into bin j,

j

ss






j

ss






j

ss






j

1. all bins j, (j + 1), . . . , (s− 2), (s− 1) were empty

ss






j


ss

2. and all unpacked items had size 6 1/2

(because we pack in non-increasing order)





so Bins j, (j + 1), . . . , (s− 2), (s− 1) each contain at least two items


j


ss









j


ss



2+ 2+ 2+ 2+







j


ss



2+ 2+ 2+ 2+

(we only use a new bin when the item won’t fit in any previous bin)







j


ss



2+ 2+ 2+ 2+





so Bins j, (j + 1), . . . , (s− 2), (s− 1) each contain at least two itemsand bin s contains at least one item


j


ss



2+ 2+ 2+ 2+







j


ss



2+ 2+ 2+ 2+ 1+






j

ss

2+ 2+ 2+ 2+ 1+





j

ss

2+ 2+ 2+ 2+ 1+





j

ss

2+ 2+ 2+ 2+ 1+






This gives a total of 2(s− j) + 1 items, none of which fits into bins 1, 2, 3, . . . , (j − 1)

j

ss

2+ 2+ 2+ 2+ 1+







otherwise we would have packed them there

j

ss

2+ 2+ 2+ 2+ 1+







j

ss

2+ 2+ 2+ 2+ 1+







so I > min{j − 1, 2(s− j) + 1}

j

ss

2+ 2+ 2+ 2+ 1+







so I > min{j − 1, 2(s− j) + 1}recall I is the total weight of all items

j

ss

2+ 2+ 2+ 2+ 1+








pairing these with theseconsiderconsider

j

ss








pairing these with theseconsiderconsider

j

ss


this + this> 1







j

ss







so I > min{j − 1, 2(s− j) + 1}

j

ss







so I > min{j − 1, 2(s− j) + 1}> d2s/3e − 1

j

ss







so I > min{j − 1, 2(s− j) + 1}> d2s/3e − 1

by plugging in j = d2s/3e

j

ss






As d2s/3e − 1 < I

j

ss





As d2s/3e − 1 < I and I 6 Opt

j

ss





As d2s/3e − 1 < I

we have that d2s/3e − 1 < Opt

and I 6 Opt

j

ss





As d2s/3e − 1 < I

we have that d2s/3e − 1 < Opt

and I 6 Opt

. . . but both sides are integers. . .

so d2s/3e 6 Opt

finally . . .2s/3 6 d2s/3e 6 Opt

or s 6 (3/2)Opt

j

ss




j

ss






in both cases. . .s 63Opt

2

j

ss




So FFD is a 3/2-approximation algorithm for BINPACKING



in both cases. . .s 63Opt

2

j

ss

Approximation Algorithms Summary





If P is a maximisation problem, Optα 6 s 6 Opt


If P is a minimisation problem (like BINPACKING), Opt 6 s 6 α ·Opt

We have seen Next Fit which is a 2-approximation algorithm for BINPACKING

which runs inO(n) time

and First Fit Decreasing which is a 3/2-approximation algorithm for BINPACKING

which runs inO(n2) time

Bin Packing is NP-hard so solving it exactly in polynomial time would prove that P = NP

Approximation Algorithms Part One: Constant factor approximations

Education