0-1 Knapsack Problem - sites.cs.ucsb.eduteo/cs230.f14/kp.pdf · CS 230A KP-0 0-1 Knapsack Problem Define object oi with profit pi > 0 and weight wi > 0, for 1 ≤ i ≤ n. Given

CS 230A KP-0✬

✫

✩

✪

0-1 Knapsack Problem

Define object oi with profit pi > 0 and weight

wi > 0, for 1 ≤ i ≤ n.

Given n objects and a knapsack capacity C > 0,

the problem is to select a subset of objects

with largest total profit and with total weight

at most C.

In other words,

Maximize∑n

i=1pixi

Subject to∑n

i=1wixi ≤ C.

xi ∈ {0, 1} ∀i

UCSB TG

CS 230A KP-1✬

✫

✩

✪

Example

Object Profit Weight

textbook 10 1

computer 50 10

ipod 8 3

iphone 22 4

pen 5 1

beer ?? 3

C = 15

UCSB TG

CS 230A KP-2✬

✫

✩

✪

Greedy Criterion

GC1: From the remaining objects, select one

with largest profit that fits into the knapsack.

Counter-example with n = 3.

C = 105 o1 o2 o3

Weight 100 10 10

Profit 20 15 15

GC2: From the remaining objects, select one

with smallest weight that fits into the

knapsack.

Counter-example with n = 3.

C = 25 o1 o2 o3

Weight 10 20 10

Profit 5 100 5

UCSB TG

CS 230A KP-3✬

✫

✩

✪

GC3: From the remaining objects, select one

with largest pi/wi that fits into the knapsack.

Counter-example with n = 3.

C = 30 o1 o2 o3

Weight 20 15 15

Profit 40 25 25

pi/wi 2 < 2 < 2

UCSB TG

CS 230A KP-4✬

✫

✩

✪

Approximations Via Rounding

Rounding Factor: δ(n, ǫ), where n is the problem

size and ǫ is the error.

δ(n, ǫ) is also called δ.

Let α be a positive value

Round up α means ... ⌈α

δ⌉

Round down α means ... ⌊α

δ⌋

Randomized Rounding means

Round up with prob. [αδ]

Round down with prob. 1− [αδ],

where [αδ] is fractional part of α

δ.

e.g., α = 7 and δ = 4

[ 74] → [ 3

4] so

Round to 2 with prob. .75, and

Round to 1 with probability .25.

UCSB TG

CS 230A KP-5✬

✫

✩

✪

Knapsack Instance (pi = wi ∀i)

C = 27 o1 o2 o3 o4 o5

pi and wi 1 2 4 8 16

Optimal xi 1 1 0 1 1

I.e., x1 = x2 = x4 = x5 = 1 and x3 = 0

Optimal solution can be found (next slides) in

O(min{2n, nF̃ , nC}) time, where F̃ is the

profit in an optimal solution.

UCSB TG

CS 230A KP-6✬

✫

✩

✪

Profit-Weight Pairs

Profit-Weight pairs are used to represent feasible

solutions over a set of objects.

For example (P1,W1) and (P2,W2), where

P1 and P2 are the profits for solutions 1 and 2.

W1 and W2 are the weights for solutions 1 and 2.

I.e., (20, 15) and (50, 35).

Dominating Pairs

(P1,W1) dominates (P2,W2)

iff

either P1 ≥ P2 and W1 < W2 or

P1 > P2 and W1 = W2

UCSB TG

CS 230A KP-7✬

✫

✩

✪

Profit-Weight Pair List

Si := List of non-dominated profit-weight pairs

for all possible feasible solutions chosen from

the first i items.

For Example

C = 45 o1 o2 o3

pi 5 3 9

wi 10 20 15

S0 = {(0, 0)}

S1 = {(0, 0), (5, 10)}

S2 = {(0, 0), (5, 10), (3,20), (8, 30)}

S3 =

{(0, 0), (5, 10), (8,30), (9, 15), (14, 25), (17, 45)}

UCSB TG

CS 230A KP-8✬

✫

✩

✪

Construction of Si Lists

• Start with Si = {(0, 0)}

• Construct iterativly S1, S2 . . ., Sn as follows

• Si = Si−1 ⊕ {(a+ pi, b+ wi)|

(a, b) ∈ Si−1 and b+ wi ≤ C},

– where ⊕ is the Union operation with the

side effect of eliminating dominated pairs.

Note that we only keep one pair of

duplicate pairs.

UCSB TG

CS 230A KP-9✬

✫

✩

✪

Example With pi = wi

C = 27 o1 o2 o3 o4 05

pi = wi 1 2 4 8 16

Profit-weight pair (P,W ) can be represented by

just P in this case.

S0 = {0}

S1 = {0} ⊕ {1} = {0, 1}

S2 = {0, 1} ⊕ {2, 3} = {0, 1, 2, 3}

S3 = {0, 1, 2, 3} ⊕ {4, 5, 6, 7} = {0, 1, ..., 7}

S4 = {0, 1, ..., 7} ⊕ {8, 9, ..., 15} = {0, 1, ..., 15}

S5 = {0, 1, ..., 15} ⊕ {16, 17, ..., 27} =

{0, 1, ..., 27}

UCSB TG

CS 230A KP-10✬

✫

✩

✪

Example With pi = wi

C = 17 o1 o2 o3 o4

pi = wi 1 1 8 8

Profit-weight pair (P,W ) can be represented by

just P in this case.

S0 = {0}

S1 = {0} ⊕ {1} = {0, 1}

S2 = {0, 1} ⊕ {1, 2} = {0, 1, 2}

S3 = {0, 1, 2} ⊕ {8, 9, 10} = {0, 1, 2, 8, 9, 10}

S4 = {0, 1, 2, 8, 9, 10} ⊕ {8, 9, 10, 16, 17} =

{0, 1, 2, 8, 9, 10, 16, 17}

UCSB TG

CS 230A KP-11✬

✫

✩

✪

Time Complexity

• List Si is sorted in ascending order of the

profit P .

• Because of the Dominance Rule the list is also

sorted in ascending order of the weight W .

• Since wi and pi are positive integers

|Si| ≤ min{F̃ , C}+ 1

|Si| ≤ 2i

• This implies that the Lists Si can be

generated in O(min{2n, nF̃ , nC}) time.

• Last pair in Sn is (represents) an optimal

solution value.

UCSB TG

CS 230A KP-12✬

✫

✩

✪

Determining the xi values

• How do we determine the xi values in an

optimal solution from the profit-weight pair?

• Let (P,W ) be the pair with maximum profit

in Sn.

for (i = n; i > 0; i--)

if (P,W) is not in S_{i-1}

then {x_i = 1;

P = P - p_i

W = W - w_i

}

else { x_i = 0

}

endfor

UCSB TG

CS 230A KP-13✬

✫

✩

✪

Approximation Via Rounding

• Original Instance (pi, wi, C)

• Reduced Instance (p′i, w′

i, C ′), where

• p′i= ⌊pi/δ⌋,

• w′

i= wi

• C ′ = C

• δ to be specified later on.

• F̃ ′: Optimal solution for the reduced instance.

• Optimal solution for the reduced instance can

be constructed in O(nF̃ ′) time by the above

procedure.

• When δ > 1 a feasible solution has a smaller

profit in the reduced instance than in the

original instance.

UCSB TG

CS 230A KP-14✬

✫

✩

✪

Example With pi = wi and δ = 4

C = 17 o1 o2 o3 o4

pi = wi 1 1 8 8

C ′ = 17 o1 o2 o3 o4

p′i

0 0 2 2

w′

i1 1 8 8

• Optimal solution for the reduced problem has

profit 4 (x′

1 = x′

2 = 0 and x′

3 = x′

4 = 1).

• In the original problem instance the same

solution has profit 16

UCSB TG

CS 230A KP-15✬

✫

✩

✪

Many knapsack problem insatances have the same

reduced instance. For example.

C = 17 o1 o2 o3 o4

pi 2 1 6 7

wi 1 1 8 8

With δ = 3 the reduced instance has the same p′i

and w′

ias in the example above.

UCSB TG

CS 230A KP-16✬

✫

✩

✪

For any feasible solution x1, x2, . . . , xn and since

p′iδ ≤ pi < (p′

i+ 1)δ, then

δ∑

p′ixi ≤

∑pixi < δ

∑(p′

i+ 1)xi ... eq(1)

We claim,

δF̃ ′ ≤ F̃ < δ(F̃ ′ + n) ... eq(2)

In the next slides we prove that

δF̃ ′ ≤ F̃

F̃ < δ(F̃ ′ + n)

UCSB TG

CS 230A KP-17✬

✫

✩

✪

Prove that δF̃ ′ ≤ F̃

Let X ′ be an optimal solution to the reduced

problem.

From (first part of) eq(1) we know that

δF̃ ′ = δ∑

p′ix′

i≤

∑pix

′

i

Since X ′ is just a feasible solution to the original

problem intance we know that∑

pix′

i≤ F̃

Therefore, δF̃ ′ ≤ F̃ .

UCSB TG

CS 230A KP-18✬

✫

✩

✪

Prove that F̃ < δ(F̃ ′ + n)

Let X∗ be an optimal solution to the original

problem instance.

From (second part of) eq(1) we know that

F̃ =∑

pix∗

i< δ

∑(p′

i+ 1)x∗

i

Since X∗ is just a feasible solution to the

reduced problem intance we know that∑

p′ix∗

i≤ F̃ ′

Therefore, F̃ < δ(F̃ ′ + n)

UCSB TG

CS 230A KP-19✬

✫

✩

✪

Guarantee ǫ Approximation

• To guarantee an ǫ-approximation δ has to be

selected carefully.

• F̂ is the objective function value, in the

original instance, for the optimal solution for

the reduced instance. I.e. obj. func. value of

solution we generate.

• From (first part of) eq(1) we know F̂ ≥ δF̃ ′.

• From (second part of) eq(2) we know

δF̃ ′ > F̃ − nδ

• Implies F̃ − F̂ < nδ and F̃−F̂

F̃< nδ

F̃

• We just need to set δ such that nδ

F̃≤ ǫ, i.e.,

δ ≤ ǫF̃

n.

• So the only rhs unknown is F̃ !!!!

UCSB TG

CS 230A KP-20✬

✫

✩

✪

Figuring Out δ

• Let l be a lower bound for F̃ , i.e., F̃ ≥ l

• Then, δ ≤ ǫl

nimplies δ ≤ ǫF̃

n

• Set l = Pmax = maxi{pi}.

• So set δ to ǫPmax

n

• Since F̃ ≤ nPmax. Substituting in δF̃ ′ ≤ F̃ ,

we know F̃ ′ ≤ nPmax

δ= n

2

ǫ.

• So the time complexity is O(nF̃ ′) which is

O(n3

ǫ)

UCSB TG

CS 230A KP-21✬

✫

✩

✪

Time Complexity

Time complexity can be improved to O(n2/ǫ) and

O(n/ǫ) (by using an algorithm that finds the kth

smallest element of n elements in O(n) time).

You also need to find a better lower bound for the

value of the optimal solution.

Variations of Rounding

• Interval Partition

• Separation

UCSB TG