Approximation Algorithms Part Two: More Constant factor approximations

Advanced Algorithms – COMS31900

Approximation algorithms part two

more constant factor approximations

Benjamin Sach

Approximation Algorithms Recap

An algorithmA is an α-approximation algorithm for problem P if,

◦A runs in polynomial time

◦A always outputs a solution with value s

Here P is an optimisation problem with optimal solution of value Opt

• If P is a maximisation problem, Optα 6 s 6 Opt

within an α factor of Opt

• If P is a minimisation problem, Opt 6 s 6 α ·Opt

We have seen

a 3/2-approximation algorithm for Bin Packing(and a faster 2-approximation)

Scheduling Jobs on Parallel Machines

1

2

3

4

5

m identical

machines

n jobs

time taken

Goal: minimise the (wall-clock) time

taken to process all jobs


1

2

3

4

5

m identical

machines

n jobs

time taken



(it’s NP-hard)


1

2

3

4

5



wall-clock time (also called makespan)


1

2

3

4

5




• Job j takes tj time units


1

2

3

4

5




• We say that j ∈ J(i) iff job j is assigned to machine i



1

2

3

4

5





• The load of machine i is Li =∑j∈J(i) tj



1

2

3

4

5







L2

L3

L1


1

2

3

4

5







• So the wall-clock time is maxi Li (which we want to minimise)

L2

L3

L1


1

2

3

4

5







• So the wall-clock time is maxi Li (which we want to minimise)

L2

L3

L1L1


1

2

3

4

5


1

2

3

4

5

Algorithm: Put job j on the machine i with smallest (current) load


1

2

3

4

5



1

2

3

4

5



1

2

3

4

5



1

2

3

4

5



1

2

3

4

5



1

2

3

4

5



1

2

3

4

5



1

2

3

4

5



1

2

3

4

5



1

2

3

4

5



1

2

3

4

5



1

2

3

4

5



1

2

3

4

5



1

2

3

4

5



1

2

3

4

5



1

2

3

4

5


How long does it take to compute this schedule?


1

2

3

4

5



m machines

n jobs


1

2

3

4

5



O(nm) time naively,O(n logm) time using a priority queue

m machines

n jobs


1

2

3

4

5




(it’s also an online solution)

m machines

n jobs


1

2

3

4

5




How good is it?

(it’s also an online solution)

m machines

n jobs

The greedy approximation

Theorem The greedy algorithm given is a 2-approximation algorithm

Let Opt denote the time taken by the optimal scheduling of jobs

Let Tg denote the time taken by the greedy schedule

Job j takes tjtime units

Li is the load of

machine i

m machines

n jobs


• Before we prove this, we prove two useful facts,





Li is the load of

machine i

m machines

n jobs



Fact Opt > maxj tj





Li is the load of

machine i

m machines

n jobs



Fact Opt > maxj tj

◦ Some machine must process the largest job





Li is the load of

machine i

m machines

n jobs



Fact Opt > maxj tj

Fact Opt >

∑j tjm






Li is the load of

machine i

m machines

n jobs



Fact Opt > maxj tj

Fact Opt >

∑j tjm


◦ There is a total of∑j tj time units of work to be done





Li is the load of

machine i

m machines

n jobs



Fact Opt > maxj tj

Fact Opt >

∑j tjm



◦ Some machine i must have load Li at least

∑j tjm





Li is the load of

machine i

m machines

n jobs



Fact Opt > maxj tj

Fact Opt >

∑j tjm



◦ Some machine i must have load Li at least

∑j tjm

(them machines can’t all have below average load)





Li is the load of

machine i

m machines

n jobs





Li is the load of

machine i


m machines

n jobs



Proof Consider the machine i with largest load Tg = Li



Li is the load of

machine i


m machines

n jobs






Li is the load of

machine i


1

2

3

4

5

Li = Tg

m machines

n jobs



• Let j denote the last job machine i completes




Li is the load of

machine i


1

2

3

4

5

Li = Tg

m machines

n jobs







Li is the load of

machine i


1

2

3

4

5

Li = Tg

m machines

n jobs

job j





• When job j was assigned, machine i had the smallest load, Li − tj



Li is the load of

machine i


1

2

3

4

5

Li = Tg

m machines

n jobs

job j








Li is the load of

machine i


1

2

3

4

5

Li = Tg

m machines

n jobs








Li is the load of

machine i


1

2

3

4

5

Li = Tg

m machines

n jobs

job j






So. . .



Li is the load of

machine i


1

2

3

4

5

Li = Tg

m machines

n jobs

job j






Li − tj 6 Lk for all 1 6 k 6 m,

So. . .



Li is the load of

machine i


1

2

3

4

5

Li = Tg

m machines

n jobs

job j







So. . .

• If we then sum over all k,



Li is the load of

machine i


1

2

3

4

5

Li = Tg

m machines

n jobs

job j







So. . .


m(Li − tj) 6m∑k=1

Lk



Li is the load of

machine i


1

2

3

4

5

Li = Tg

m machines

n jobs

job j







So. . .



Lk

so (Li − tj) 6∑m

k=1 Lk

m 6 Opt (by the second fact)



Li is the load of

machine i


1

2

3

4

5

Li = Tg

m machines

n jobs

job j







So. . .



Lk


k=1 Lk




Li is the load of

machine i


1

2

3

4

5

Li = Tg

m machines

n jobs

job j

Fact Opt >

∑j tjm







So. . .



Lk


k=1 Lk




Li is the load of

machine i


1

2

3

4

5

Li = Tg

m machines

n jobs

job j







So. . .



Lk


k=1 Lk


also tj 6 Opt (by the first fact)



Li is the load of

machine i


1

2

3

4

5

Li = Tg

m machines

n jobs

job j







So. . .



Lk


k=1 Lk





Li is the load of

machine i


1

2

3

4

5

Li = Tg

m machines

n jobs

job j

Fact Opt > maxj tj







So. . .



Lk


k=1 Lk





Li is the load of

machine i


1

2

3

4

5

Li = Tg

m machines

n jobs

job j







So. . .



Lk


k=1 Lk



Therefore, Tg = Li = (Li − tj) + tj 6 Opt + Opt = 2Opt



Li is the load of

machine i


1

2

3

4

5

Li = Tg

m machines

n jobs

job j

Longest Processing Time (LPT)

1

2

3

4

5

Step 1: Sort the jobs into non-increasing order (job 1 is now largest)


1

2

3

4

5



1

2

3

4

5


Step 2: Put job j on the machine i with smallest (current) load


1

2

3

4

5




1

2

3

4

5




1

2

3

4

5




1

2

3

4

5




1

2

3

4

5




1

2

3

4

5




1

2

3

4

5




1

2

3

4

5




1

2

3

4

5




1

2

3

4

5




1

2

3

4

5




1

2

3

4

5




1

2

3

4

5




1

2

3

4

5




1

2

3

4

5




1

2

3

4

5



greedy

solution


1

2

3

4

5



greedy

solution



1

2

3

4

5



greedy

solution


O(n logn) time (to sort the jobs)


1

2

3

4

5



greedy

solution


O(n logn) time (to sort the jobs)

How good is it?

The LPT approximation

Theorem The LPT algorithm is a 3/2-approximation algorithm

• Let Tl denote the time taken by the LPT scheduleLi is the load of

machine i

m machines

n jobs




• Before we prove this, we prove another useful fact and a Lemma


machine i

m machines

n jobs





Fact If there are at mostm jobs (n 6 m) then LPT is optimal


machine i

m machines

n jobs






LPT gives each job its own machine so maxi Li 6 maxj tj 6 Opt


machine i

m machines

n jobs


If there are at mostm jobs then





Lemma If n > m then Opt > 2t(m+1) (after sorting)



machine i

m machines

n jobs









• Let Tl denote the time taken by the LPT schedule

Proof

Li is the load of

machine i

m machines

n jobs









◦ Note that t1 > t2 > t3 > . . . tm > t(m+1)


Proof

Li is the load of

machine i

m machines

n jobs










◦ One of them machines must be assigned


Proof

(at least) two of thesem+ 1 jobs under any schedule

Li is the load of

machine i

m machines

n jobs












Proof


◦ So we have that any schedule takes at least 2t(m+1) time

Li is the load of

machine i

m machines

n jobs












Proof


◦ So we have that any schedule takes at least 2t(m+1) time

Li is the load of

machine i

m machines

n jobs

in particular Opt > 2t(m+1)





Li is the load of

machine i

m machines

n jobs




Proof Consider the machine i with largest load Tl = Li

Li is the load of

machine i

m machines

n jobs






Li is the load of

machine i

m machines

n jobs






Li is the load of

machine i

m machines

n jobs


job j

12345





• Using the same argument as before, we have that,

Li is the load of

machine i

m machines

n jobs


job j

12345






(Li − tj) 6 Opt

Li is the load of

machine i

m machines

n jobs


job j

12345






(Li − tj) 6 Opt

• If n 6 m then we are done so assume n > m

Li is the load of

machine i

m machines

n jobs


job j

12345






(Li − tj) 6 Opt


Li is the load of

machine i

m machines

n jobs


job j

12345

because LPT is optimal in this case






(Li − tj) 6 Opt


Li is the load of

machine i

m machines

n jobs


job j

12345






(Li − tj) 6 Opt


• Further if (Li − tj) = 0 then Tl = Li = tj 6 Opt

Li is the load of

machine i

m machines

n jobs


job j

12345






(Li − tj) 6 Opt



Li is the load of

machine i

m machines

n jobs


job j

12345

Fact Opt > maxj tj






(Li − tj) 6 Opt



Li is the load of

machine i

m machines

n jobs


job j

12345






(Li − tj) 6 Opt



so assume that (Li − tj) > 0

Li is the load of

machine i

m machines

n jobs


job j

12345






(Li − tj) 6 Opt




• Therefore machine i was assigned at least two jobs

Li is the load of

machine i

m machines

n jobs


job j

12345






(Li − tj) 6 Opt





By the algorithm description, we have that j > m+ 1

Li is the load of

machine i

m machines

n jobs


job j

12345






(Li − tj) 6 Opt






Li is the load of

machine i

m machines

n jobs


job j

12345

it doesn’t assign a second job to any machine untilevery machine has at least one job






(Li − tj) 6 Opt






Li is the load of

machine i

m machines

n jobs


job j

12345






(Li − tj) 6 Opt






tj 6 tm+1 6 Opt/2 (by the Lemma)

Li is the load of

machine i

m machines

n jobs


job j

12345






(Li − tj) 6 Opt







Li is the load of

machine i

m machines

n jobs


job j

12345







(Li − tj) 6 Opt







Li is the load of

machine i

m machines

n jobs


job j

12345






(Li − tj) 6 Opt

Therefore, Tl = Li = (Li − tj) + tj 6 Opt + Opt/2 = (3/2) ·Opt







Li is the load of

machine i

m machines

n jobs


job j

12345

Scheduling conclusions


which runs inO(n logn) time

Theorem The greedy algorithm is a 2-approximation algorithm

which runs inO(n logm) time and it’s online

In fact, LPT is a 4/3-approximation algorithm (using better analysis)

m machines

n jobs

k-centers

Goal Minimise the largest distance from any site to the closest center

k-centers


k-centers

n points (sites) in 2D space


k-centers


The distance between points si, sj is√

(xi − xj)2 + (yi − yj)2


k-centers



(xi − xj)2 + (yi − yj)2


(i.e. ‘normal’ euclidean distance)

k-centers



(xi − xj)2 + (yi − yj)2


Select k sites to be

centers


k-centers



(xi − xj)2 + (yi − yj)2



centers


k-centers



(xi − xj)2 + (yi − yj)2



centers

k-centers



(xi − xj)2 + (yi − yj)2




centers

k-centers



(xi − xj)2 + (yi − yj)2




centers

k-centers



(xi − xj)2 + (yi − yj)2




centers

r

r

k-centers



(xi − xj)2 + (yi − yj)2




centers

k-centers



(xi − xj)2 + (yi − yj)2




centers

k-centers



(xi − xj)2 + (yi − yj)2




centers

r

k-centers



(xi − xj)2 + (yi − yj)2



(in general it’s NP-hard)


centers

r

A Greedy approximation

Start by picking any point to be a center





Repeatedly pick the site which is furthest from any existing center












































r



This takesO(nk) time




This takesO(nk) time


but is it any good?

The Greedy approximation

Theorem The Greedy algorithm for k-center is a 2-approximation algorithm

Let Cg (resp. COpt) denote the set of centers selected by Greedy (resp. Optimal)

Proof

Let rg (resp. Opt) denote largest site-center distance using Greedy (resp. Optimal)




Proof


Case 1: No si, si′ ∈ Cg are closest to the same sj ∈ COpt




Proof



Optimal centers




Proof



Optimal centers

Opt




Proof



Optimal centers

Opt

Sites




Proof



Optimal centers

Opt

Sitespurposes only

Disclaimer: for illustrative




Proof



Optimal centers

Opt

Sitespurposes only


Greedy centers




Proof



Optimal centers

Opt

Sitespurposes only


Greedy centers




Proof



Optimal centers

Opt

Sitespurposes only


Greedy centers

Distance at most 2Opt




Proof



Optimal centers

Opt

Sitespurposes only


Greedy centers

Distance at most 2Opt

so rg 6 2Opt




Proof


Case 2: Some si, si′ ∈ Cg are closest to the same sj ∈ COpt




Proof



si

si′




Proof



Assume wlog. that Greedy made si a center after si′

si

si′




Proof




si

si′

si was added as a center because it was




Proof




si

si′


the furthest from any existing Greedy center




Proof




si

si′



However, si is at most 2Opt away from si′




Proof




si

si′







Proof




si

si′




So even before adding si as a center, all sites

were 6 2Opt away from a Greedy center




Proof




si

si′




So even before adding si as a center, all sites

were 6 2Opt away from a Greedy center

Therefore, rg 6 2Opt

k-center Conclusions


which runs inO(nk) time




• The approximation works for any (metric) distance function,





d(si, sj) = L1 or L∞ for example






d(x, y) = d(y, x), d(x, y) > 0

• Distance function d is a metric iff

(d(x, y) = 0 iff x = y) and d(x, z) 6 d(x, y) + d(y, z)






• For a general (metric) d, the problem is not α-approximable with α < 2

d(x, y) = d(y, x), d(x, y) > 0









d(x, y) = d(y, x), d(x, y) > 0

• For d = L2 , the problem is not α-approximable with α <√3 ≈ 1.73









d(x, y) = d(y, x), d(x, y) > 0

• For d = L2 , the problem is not α-approximable with α <√3 ≈ 1.73

• For d = L1 or d = L∞ , the problem is not α-approximable with α < 2



Approximation Algorithms Part Two: More Constant factor approximations

Education