Maximizing Submodular Function over the Integer Lattice
Post on 21-Feb-2017
255 Views
Preview:
Transcript
Maximizing Submodular Functionover the Integer Lattice
Tasuku Soma(Univ. Tokyo)
Joint work with:Yuichi Yoshida (NII, Tokyo)
1 / 22
1 Monotone Submodular Function Maximization on ZS+
2 Algorithms
3 DR-Submodular Cover
4 Summary
2 / 22
1 Monotone Submodular Function Maximization on ZS+
2 Algorithms
3 DR-Submodular Cover
4 Summary
3 / 22
Monotone Submodular Func Maximization
E: finite set, f : 2E → R ... monotone submodular
maximize f (X ) subject to X ∈ F
|X | ≤ k,∑
i∈X w (i) ≤ 1, etc
• NP-hard in general• O(1)-appriximable for various constraints (cardinality,knapsack, matroid, etc) and efficient algorithms
• Powerful model for machine learning
4 / 22
Monotone Submodular Func Maximization
E: finite set, f : 2E → R ... monotone submodular
maximize f (X ) subject to X ∈ F
|X | ≤ k,∑
i∈X w (i) ≤ 1, etc
• NP-hard in general• O(1)-appriximable for various constraints (cardinality,knapsack, matroid, etc) and efficient algorithms
• Powerful model for machine learning
4 / 22
Limitation of Set Function
Some real scenarios cannot be captured by a set function.
Budget Allocation [Alon–Gamzu–Tennenholtz ’13,S.–Kakimura–Inaba–Kawarabayashi ’14]:We want to decide how much budget set aside for eachad source.
Generalized Sensor Placement:We can put more than one sensors in each spot.
Can we generalize these set functionmodels?
5 / 22
Limitation of Set Function
Some real scenarios cannot be captured by a set function.
Budget Allocation [Alon–Gamzu–Tennenholtz ’13,S.–Kakimura–Inaba–Kawarabayashi ’14]:We want to decide how much budget set aside for eachad source.
Generalized Sensor Placement:We can put more than one sensors in each spot.
Can we generalize these set functionmodels?
5 / 22
Limitation of Set Function
Some real scenarios cannot be captured by a set function.
Budget Allocation [Alon–Gamzu–Tennenholtz ’13,S.–Kakimura–Inaba–Kawarabayashi ’14]:We want to decide how much budget set aside for eachad source.
Generalized Sensor Placement:We can put more than one sensors in each spot.
Can we generalize these set functionmodels?
5 / 22
Limitation of Set Function
Some real scenarios cannot be captured by a set function.
Budget Allocation [Alon–Gamzu–Tennenholtz ’13,S.–Kakimura–Inaba–Kawarabayashi ’14]:We want to decide how much budget set aside for eachad source.
Generalized Sensor Placement:We can put more than one sensors in each spot.
Can we generalize these set functionmodels?
5 / 22
Definitions of Submodularity on {0, 1}E
E: finite set
f : {0, 1}E → R is submodularf (X ) + f (Y ) ≥ f (X ∪ Y ) + f (X ∩ Y ) (∀X,Y ⊆ E)
m
Diminishing Returnf (X ∪ e) − f (X ) ≥ f (Y ∪ e) − f (Y )
(X ⊆ Y ⊆ E, e ∈ E \ Y )
6 / 22
Definitions of Submodularity on {0, 1}E
E: finite set
f : {0, 1}E → R is submodularf (X ) + f (Y ) ≥ f (X ∪ Y ) + f (X ∩ Y ) (∀X,Y ⊆ E)
m
Diminishing Returnf (X ∪ e) − f (X ) ≥ f (Y ∪ e) − f (Y )
(X ⊆ Y ⊆ E, e ∈ E \ Y )
6 / 22
Definitions of Submodularity on ZE
f : ZE → R is lattice submodular:f (x) + f (y) ≥ f (x ∨ y) + f (x ∧ y) (∀x, y ∈ ZE)
coord-wise max coord-wise min
⇑ 6⇓
f is diminishing return submodular (DR-submodular):f (x + ei) − f (x) ≥ f (y + ei) − f (y)
(x ≤ y ∈ ZE, i ∈ E)
7 / 22
Definitions of Submodularity on ZE
f : ZE → R is lattice submodular:f (x) + f (y) ≥ f (x ∨ y) + f (x ∧ y) (∀x, y ∈ ZE)
coord-wise max coord-wise min
⇑ 6⇓
f is diminishing return submodular (DR-submodular):f (x + ei) − f (x) ≥ f (y + ei) − f (y)
(x ≤ y ∈ ZE, i ∈ E)
7 / 22
Definitions of Submodularity on ZE
f : ZE → R is lattice submodular:f (x) + f (y) ≥ f (x ∨ y) + f (x ∧ y) (∀x, y ∈ ZE)
coord-wise max coord-wise min
⇑ 6⇓
f is diminishing return submodular (DR-submodular):f (x + ei) − f (x) ≥ f (y + ei) − f (y)
(x ≤ y ∈ ZE, i ∈ E)
7 / 22
Monotone Submod Func Maximization on ZE+
f : ZE+ → R+ ... monotone lattice/DR-submodular func
(with f (0) = 0), r ∈ Z+
Maximize f (x)
subject to 0 ≤ x ≤ r1, x ∈ ZE+ ∩ F
• F = {x : x(E) ≤ k} (cardinality)• F = P(+) (ρ) (polymatroid)• F = {x : w>x ≤ 1} (knapsack)
8 / 22
Can We Reduce it to Set Function?YES, if f is DR-submodular.
E
r copies
←→
240
Drawback:• The new ground set has r |E | size, pseudopoly• Does not work for lattice-submodular function
9 / 22
Can We Reduce it to Set Function?YES, if f is DR-submodular.
E
r copies
←→
240
Drawback:• The new ground set has r |E | size, pseudopoly• Does not work for lattice-submodular function
9 / 22
Can We Reduce it to Set Function?YES, if f is DR-submodular.
E
r copies
←→
240
Drawback:• The new ground set has r |E | size, pseudopoly• Does not work for lattice-submodular function
9 / 22
Can We Reduce it to Set Function?YES, if f is DR-submodular.
E
r copies
←→
240
Drawback:• The new ground set has r |E | size, pseudopoly• Does not work for lattice-submodular function
9 / 22
Our Results
Theorem (S. and Yoshida ’15)For any ε > 0, (1 − 1/e − ε )-appriximate polytimealgorithms for various constaints.
DR-submodular lattice submodular
cardinality X(deterministic) X(deterministic)
polymatroid X(random) open
knapsack X(random) only pseudopoly
10 / 22
1 Monotone Submodular Function Maximization on ZS+
2 Algorithms
3 DR-Submodular Cover
4 Summary
11 / 22
Algorithms
Naive Approach:Choosing the best coordinate and step size in everyiteration?
k∗, i∗ ∈ argmaxk,i
f (kei | x)k
,
where f (kei | x) = f (kei + x) − f (x).
Idea:
• Use “Decreasing Threshold Greedy”[Badanidiyuru–Vondrák ’14] to determine step size
12 / 22
Algorithms
Naive Approach:Choosing the best coordinate and step size in everyiteration?
k∗, i∗ ∈ argmaxk,i
f (kei | x)k
,
where f (kei | x) = f (kei + x) − f (x).
Idea:
• Use “Decreasing Threshold Greedy”[Badanidiyuru–Vondrák ’14] to determine step size
12 / 22
Cardinality/DR-Submodular
DecresingThresholdGreedy1: x := 0, d := maxi∈E f (ei), θ := d2: while θ ≥ ε d
r :3: for each i ∈ E :4: Find the largest k s.t.
f (kei | x)k
≥ θ and x + kei
feasible.5: x := x + kei6: θ := (1 − ε )θ7: return x
13 / 22
Cardinality/DR-Submodular
f is concave along each coordinate.
i
f (· | x)
step size k
slopeθ
Such k can be found in O(log r) time with binary search.
14 / 22
Cardinality/Lattice-SubmodularIdea: Devide the range into polynomially many regions.
i
fmax
(1 − ε )fmax
(1 − ε )2fmax k
k′
LemmaIf there exists k with f (kei | x) ≥ kθ, we can find k′ withf (k′ei | x) ≥ (1 − ε )k′θ.
15 / 22
Cardinality/Lattice-SubmodularIdea: Devide the range into polynomially many regions.
i
fmax
(1 − ε )fmax
(1 − ε )2fmax
k
k′
LemmaIf there exists k with f (kei | x) ≥ kθ, we can find k′ withf (k′ei | x) ≥ (1 − ε )k′θ.
15 / 22
Cardinality/Lattice-SubmodularIdea: Devide the range into polynomially many regions.
i
fmax
(1 − ε )fmax
(1 − ε )2fmax k
k′
LemmaIf there exists k with f (kei | x) ≥ kθ, we can find k′ withf (k′ei | x) ≥ (1 − ε )k′θ.
15 / 22
Polymatroid/DR-Submodular
Idea: Mimic Continuous Greedy Algorithm
Multilinear Extension of f : 2E → R
F (x) =∑X⊆E
f (X )∏i∈X
x(i)∏i<X
(1 − x(i)) (x ∈ [0, 1]E)
Key Facts• F is monotone if f is monotone• F is concave along positive direction if f is submodular
16 / 22
Polymatroid/DR-Submodular
Idea: Gluing the multilinear extensions on eachhypercube.
fill by the multilinear ext off̃ (X ) = f (1 + 1X )
The resulting extension shares the same property?→ YES, if f is DR-submodular.
17 / 22
Polymatroid/DR-Submodular
Idea: Gluing the multilinear extensions on eachhypercube.
fill by the multilinear ext off̃ (X ) = f (1 + 1X )
The resulting extension shares the same property?→ YES, if f is DR-submodular.
17 / 22
Polymatroid/DR-Submodular
Idea: Gluing the multilinear extensions on eachhypercube.
fill by the multilinear ext off̃ (X ) = f (1 + 1X )
The resulting extension shares the same property?→ YES, if f is DR-submodular.
17 / 22
1 Monotone Submodular Function Maximization on ZS+
2 Algorithms
3 DR-Submodular Cover
4 Summary
18 / 22
Submodular Cover [Wolsey ’82]
Somewhat “dual” problem of maximization
f, c : 2S → R+ monotone submodular, α > 0
minimize c(X ) subject to f (X ) ≥ α
c : cost, f : quality, α : worst guarantee
19 / 22
Submodular Cover [Wolsey ’82]
Somewhat “dual” problem of maximization
f, c : 2S → R+ monotone submodular, α > 0
minimize c(X ) subject to f (X ) ≥ α
c : cost, f : quality, α : worst guarantee
Examples in ML:• Efficient Sensor Placement
[Krause&Guestrin ’05, Krause&Leskovec ’08]
• Text Summarization [Lin & Bilmes ’10]
• Object Finding [Song et al. ’14, Chen et al. ’14]
19 / 22
Submodular Cover [Wolsey ’82]
Somewhat “dual” problem of maximization
f, c : 2S → R+ monotone submodular, α > 0
minimize c(X ) subject to f (X ) ≥ α
c : cost, f : quality, α : worst guarantee
Algorithmic Results:• For c(X ) = |X |, O(log d/β)-approx [Wolsey ’82]
• For integral f, c, O(ρ log d)-approx [Wan et al. ’09]
d = maxs f (s), ρ: curvature of c,β := min{f (s | X ) : s ∈ S,X ⊆ S, f (s | X ) > 0}
19 / 22
DR-Submodular Cover
f, c : ZS → R+ monotone DR-submodular, α > 0, r ∈ Z+
minimize c(x)subject to f (x) ≥ α
0 ≤ x ≤ r1
Theorem (S.–Yoshida, to appear in NIPS ’15)An algorithm for finding a (nearly) feasible solution ofO(ρ log d/β) approx in O (n log nr log r) time.
20 / 22
DR-Submodular Cover
f, c : ZS → R+ monotone DR-submodular, α > 0, r ∈ Z+
minimize c(x)subject to f (x) ≥ α
0 ≤ x ≤ r1
Theorem (S.–Yoshida, to appear in NIPS ’15)An algorithm for finding a (nearly) feasible solution ofO(ρ log d/β) approx in O (n log nr log r) time.
20 / 22
1 Monotone Submodular Function Maximization on ZS+
2 Algorithms
3 DR-Submodular Cover
4 Summary
21 / 22
Summary
Our Results• Useful genealizations of monotone submodular funcmaximization and submodular cover
• Various polytime approximation algorithms
Recent Work• Online monotone submodular func maximization onZE+ [Avigdor-Elgrabli et al. ’15]
• Nonmonotone submodular func maximization on ZE+
[Gottschalk–Peis ’15]
22 / 22
top related