Fast Deterministic Algorithms for Matrix Completion Problems Tasuku Soma Research Institute for Mathematical Sciences, Kyoto Univ. Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 1 / 29
May 31, 2015
Fast Deterministic Algorithmsfor Matrix Completion Problems
Tasuku Soma
Research Institute for Mathematical Sciences,Kyoto Univ.
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 1 / 29
1 Introduction
2 Matrix Completion by Rank-One Matrices
3 Application to Network Coding
4 Mixed Skew-Symmetric Matrix Completion
5 Skew-Symmetric Matrix Completion by Rank-Two Skew-SymmetricMatrices
6 Conclusion
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 2 / 29
1 Introduction
2 Matrix Completion by Rank-One Matrices
3 Application to Network Coding
4 Mixed Skew-Symmetric Matrix Completion
5 Skew-Symmetric Matrix Completion by Rank-Two Skew-SymmetricMatrices
6 Conclusion
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 3 / 29
Matrix Completion
Matrix CompletionF: Field
Input Matrix A(x1, . . . , xn) over F(x1, . . . , xn) with indeterminatesx1, . . . , xn
Find α1, . . . , αn ∈ F maximizing rank A(α1, . . . , αn).
ExampleF = Q,
A =
[1 + x1 2 + x2
x3 x4
]−→ A ′ =
[2 21 0
](x1 := 1, x2 := 0, x3 := 1, x4 := 0)
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 4 / 29
Matrix Completion
Matrix CompletionF: Field
Input Matrix A(x1, . . . , xn) over F(x1, . . . , xn) with indeterminatesx1, . . . , xn
Find α1, . . . , αn ∈ F maximizing rank A(α1, . . . , αn).
ExampleF = Q,
A =
[1 + x1 2 + x2
x3 x4
]−→ A ′ =
[2 21 0
](x1 := 1, x2 := 0, x3 := 1, x4 := 0)
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 4 / 29
Backgrounds
A variety of combinatorial optimization problems can be formulated bymatrices with indeterminates:
Maximum matching,
Structural rigidity,
Network coding, etc.
Previous WorksMatrix completion for general matrices is solvable in polynomial timeby a randomized algorithm if the field is sufficiently large.
Deterministic algorithms are known only for special matrices(cf. polynomial identity testing)
NP hard over a general field.
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 5 / 29
Backgrounds
A variety of combinatorial optimization problems can be formulated bymatrices with indeterminates:
Maximum matching,
Structural rigidity,
Network coding, etc.
Previous WorksMatrix completion for general matrices is solvable in polynomial timeby a randomized algorithm if the field is sufficiently large.
Deterministic algorithms are known only for special matrices(cf. polynomial identity testing)
NP hard over a general field.
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 5 / 29
Our Results
Our ResultsDeterministic polynomial time algorithms for the following matrixcompletion problems:
Matrix completion by rank-one matrices— a faster algorithm than the previous one
Mixed skew-symmetric matrix completion— the first deterministic polynomial time algorithm
Skew-symmetric matrix completion byrank-two skew-symmetric matrices
— the first deterministic polynomial time algorithm
They are working over an arbitrary field!
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 6 / 29
Our Results
Our ResultsDeterministic polynomial time algorithms for the following matrixcompletion problems:
Matrix completion by rank-one matrices— a faster algorithm than the previous one
Mixed skew-symmetric matrix completion— the first deterministic polynomial time algorithm
Skew-symmetric matrix completion byrank-two skew-symmetric matrices
— the first deterministic polynomial time algorithm
They are working over an arbitrary field!
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 6 / 29
1 Introduction
2 Matrix Completion by Rank-One Matrices
3 Application to Network Coding
4 Mixed Skew-Symmetric Matrix Completion
5 Skew-Symmetric Matrix Completion by Rank-Two Skew-SymmetricMatrices
6 Conclusion
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 7 / 29
Problem Definition
Matrix Completion by Rank-One MatricesMatrix completion for A = B0 + x1B1 + · · ·+ xnBn, where B1, . . . ,Bn are ofrank one.
Example
B0 =
[1 00 0
], B1 =
[1 10 0
], B2 =
[2 01 0
]A =
[1 + x1 + 2x2 x1
x2 0
]
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 8 / 29
Problem Definition
Matrix Completion by Rank-One MatricesMatrix completion for A = B0 + x1B1 + · · ·+ xnBn, where B1, . . . ,Bn are ofrank one.
Example
B0 =
[1 00 0
], B1 =
[1 10 0
], B2 =
[2 01 0
]A =
[1 + x1 + 2x2 x1
x2 0
]
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 8 / 29
Previous Works
In the case of B0 = 0:
Lovasz ’89This can be reduced to linear matroid intersection.
solvable in O(mn1.62) time using the algorithm of Gabow & Xu ’96
For the general case:
Ivanyos, Karpinski & Saxena ’10
An optimal solution can be found in O(m4.37n) time.
Our ResultAn optimal solution can be found in O((m + n)2.77) time.
m: the larger of row and column sizes, n: # of indeterminates
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 9 / 29
Previous Works
In the case of B0 = 0:
Lovasz ’89This can be reduced to linear matroid intersection.
solvable in O(mn1.62) time using the algorithm of Gabow & Xu ’96
For the general case:
Ivanyos, Karpinski & Saxena ’10
An optimal solution can be found in O(m4.37n) time.
Our ResultAn optimal solution can be found in O((m + n)2.77) time.
m: the larger of row and column sizes, n: # of indeterminates
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 9 / 29
Previous Works
In the case of B0 = 0:
Lovasz ’89This can be reduced to linear matroid intersection.
solvable in O(mn1.62) time using the algorithm of Gabow & Xu ’96
For the general case:
Ivanyos, Karpinski & Saxena ’10
An optimal solution can be found in O(m4.37n) time.
Our ResultAn optimal solution can be found in O((m + n)2.77) time.
m: the larger of row and column sizes, n: # of indeterminates
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 9 / 29
Idea
For A = B0 + x1B1 + · · ·+ xnBn (Bi = uiv>i (i = 1, . . . , n))
A :=
1. . .
10
v>1...
v>nx1
. . .
xn
1. . .
10
0 u1 · · · un B0
.
Lemma
rank A = 2n + rank A
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 10 / 29
Idea
For A = B0 + x1B1 + · · ·+ xnBn (Bi = uiv>i (i = 1, . . . , n))
A :=
1. . .
10
v>1...
v>nx1
. . .
xn
1. . .
10
0 u1 · · · un B0
.
Lemma
rank A = 2n + rank A
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 10 / 29
Algorithm
Each indeterminate appears only once in A ! (A is a mixed matrix)
Harvey, Karger & Murota ’05
Matrix completion for a mixed matrix can be done in O(m2.77) time.
↓ Apply to A
TheoremMatrix completion by rank-one matrices can be done in O((m + n)2.77)time.
m: the larger of row and column sizes, n: # of indeterminates
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 11 / 29
Algorithm
Each indeterminate appears only once in A ! (A is a mixed matrix)
Harvey, Karger & Murota ’05
Matrix completion for a mixed matrix can be done in O(m2.77) time.
↓ Apply to A
TheoremMatrix completion by rank-one matrices can be done in O((m + n)2.77)time.
m: the larger of row and column sizes, n: # of indeterminates
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 11 / 29
Algorithm
Each indeterminate appears only once in A ! (A is a mixed matrix)
Harvey, Karger & Murota ’05
Matrix completion for a mixed matrix can be done in O(m2.77) time.
↓ Apply to A
TheoremMatrix completion by rank-one matrices can be done in O((m + n)2.77)time.
m: the larger of row and column sizes, n: # of indeterminates
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 11 / 29
Min-Max Theorem
TheoremFor A = B0 + x1B1 + · · ·+ xnBn,
max{rank A : x1, . . . , xn}
=min{
rank[
0 [vj : j < J]>
[uj : j ∈ J] B0
]: J ⊆ {1, . . . , n}
}.
Corollary (Lovasz ’89)If B0 = 0, then
max{rank A : x1, . . . , xn}
=min{dim〈uj : j ∈ J〉+ dim〈vj : j < J〉 : J ⊆ {1, . . . , n}}
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 12 / 29
Min-Max Theorem
TheoremFor A = B0 + x1B1 + · · ·+ xnBn,
max{rank A : x1, . . . , xn}
=min{
rank[
0 [vj : j < J]>
[uj : j ∈ J] B0
]: J ⊆ {1, . . . , n}
}.
Corollary (Lovasz ’89)If B0 = 0, then
max{rank A : x1, . . . , xn}
=min{dim〈uj : j ∈ J〉+ dim〈vj : j < J〉 : J ⊆ {1, . . . , n}}
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 12 / 29
Simultaneous Matrix Completion by Rank-One Matrices
Simultaneous Matrix Completion by Rank-One MatricesF: Field
Input CollectionA of matrices in the form of B0 + x1B1 + · · ·+ xnBn
Find Value assignments αi ∈ F for each indeterminate xi
maximizing the rank of every matrix in A
TheoremA solution of simultaneous matrix completion by rank-one matrices can befound in polynomial time, if |F| > |A|.
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 13 / 29
Simultaneous Matrix Completion by Rank-One Matrices
Simultaneous Matrix Completion by Rank-One MatricesF: Field
Input CollectionA of matrices in the form of B0 + x1B1 + · · ·+ xnBn
Find Value assignments αi ∈ F for each indeterminate xi
maximizing the rank of every matrix in A
TheoremA solution of simultaneous matrix completion by rank-one matrices can befound in polynomial time, if |F| > |A|.
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 13 / 29
1 Introduction
2 Matrix Completion by Rank-One Matrices
3 Application to Network Coding
4 Mixed Skew-Symmetric Matrix Completion
5 Skew-Symmetric Matrix Completion by Rank-Two Skew-SymmetricMatrices
6 Conclusion
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 14 / 29
Network Coding
Network communication model s.t. intermediate nodes can perform coding
Classical model Network coding
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 15 / 29
Multicast Problem with Linearly Correlated Sources
Messages in source nodes are linearlycorrelated
Each sink node demands the originalmessages x1 & x2
TheoremA solution of this multicast can be found in polynomial time.
Approach: simultaneous matrix completion by rank-one matrices.
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 16 / 29
1 Introduction
2 Matrix Completion by Rank-One Matrices
3 Application to Network Coding
4 Mixed Skew-Symmetric Matrix Completion
5 Skew-Symmetric Matrix Completion by Rank-Two Skew-SymmetricMatrices
6 Conclusion
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 17 / 29
Problem Definition
Mixed Skew-Symmetric Matrix CompletionMatrix completion for a skew-symmetric matrix s.t. each indeterminateappears twice (mixed skew-symmetric matrix).
Example
A =
0 −1 11 0 0−1 0 0
+ 0 x 0−x 0 y0 −y 0
= 0 −1 + x 11 − x 0 y−1 −y 0
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 18 / 29
Our Result
There were no algorithms for this problem, but we can compute the rank.
Murota ’03 (←Geelen, Iwata & Murota ’03)The rank of an m ×m mixed skew-symmetric matrix can be computed inO(m4) time.
Our ResultMatrix completion for an m ×m mixed skew-symmetric matrix can be donein O(m4) time.
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 19 / 29
Our Result
There were no algorithms for this problem, but we can compute the rank.
Murota ’03 (←Geelen, Iwata & Murota ’03)The rank of an m ×m mixed skew-symmetric matrix can be computed inO(m4) time.
Our ResultMatrix completion for an m ×m mixed skew-symmetric matrix can be donein O(m4) time.
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 19 / 29
Rank of Mixed Skew-Symmetric Matrix
Lemma (Murota ’03)For an m ×m mixed skew-symmetric matrix A = Q + T(Q : constant part, T : indeterminates part),
rank A = max{|FQ 4 FT | : both Q[FQ ],T [FT ] are nonsingular
}RHS is linear delta-covering.
Optimal FQ and FT can be found in O(m4)time (Geelen, Iwata & Murota ’03).
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 20 / 29
Support Graph and Pfaffian
Support graph:
A =
0 −2 1 12 0 0 3−1 0 0 21 −3 −2 0
Pfaffian:
pf A :=∑
M:perfect matching in G
±∏ij∈M
Aij
= A12A34 − A13A24
Lemmadet A = (pf A)2
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 21 / 29
Support Graph and Pfaffian
Support graph:
A =
0 −2 1 12 0 0 3−1 0 0 21 −3 −2 0
Pfaffian:
pf A :=∑
M:perfect matching in G
±∏ij∈M
Aij
= A12A34 − A13A24
Lemmadet A = (pf A)2
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 21 / 29
Sketch of Algorithm
Algorithm1: Find an optimal solution FQ and FT for linear delta-covering.2: Find a perfect matching M in the support graph of T [FT ].
3: for each ij ∈ M do4: Substitute α to Tij so that Q[FQ ∪ {i, j}] will be nonsingular after
substitution.5: FQ := FQ ∪ {i, j}6: end for7: Substitute 0 to the rest of indeterminates8: return the resulting matrix
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 22 / 29
Sketch of Algorithm
Algorithm1: Find an optimal solution FQ and FT for linear delta-covering.2: Find a perfect matching M in the support graph of T [FT ].3: for each ij ∈ M do4: Substitute α to Tij so that Q[FQ ∪ {i, j}] will be nonsingular after
substitution.5: FQ := FQ ∪ {i, j}6: end for
7: Substitute 0 to the rest of indeterminates8: return the resulting matrix
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 22 / 29
Sketch of Algorithm
Algorithm1: Find an optimal solution FQ and FT for linear delta-covering.2: Find a perfect matching M in the support graph of T [FT ].3: for each ij ∈ M do4: Substitute α to Tij so that Q[FQ ∪ {i, j}] will be nonsingular after
substitution.5: FQ := FQ ∪ {i, j}6: end for7: Substitute 0 to the rest of indeterminates8: return the resulting matrix
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 22 / 29
Sketch of Algorithm
How can we find α s.t. Q[FQ ∪ {i, j}] will be nonsingular?
A = Q + T
A ′ = Q ′ + T ′
LemmaQ ′: modified matrix of Q as Q ′ij := Qij + α, Q ′ji := Qji − α
pf Q ′[FQ ∪ {i, j}] = pf Q[FQ ∪ {i, j}] ± α · pf Q[FQ ]
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 23 / 29
Sketch of Algorithm
How can we find α s.t. Q[FQ ∪ {i, j}] will be nonsingular?
A = Q + T A ′ = Q ′ + T ′
LemmaQ ′: modified matrix of Q as Q ′ij := Qij + α, Q ′ji := Qji − α
pf Q ′[FQ ∪ {i, j}] = pf Q[FQ ∪ {i, j}] ± α · pf Q[FQ ]
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 23 / 29
Sketch of Algorithm
How can we find α s.t. Q[FQ ∪ {i, j}] will be nonsingular?
A = Q + T A ′ = Q ′ + T ′
LemmaQ ′: modified matrix of Q as Q ′ij := Qij + α, Q ′ji := Qji − α
pf Q ′[FQ ∪ {i, j}] = pf Q[FQ ∪ {i, j}] ± α · pf Q[FQ ]
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 23 / 29
Sketch of Algorithm
Finally, we obtain Q ′ s.t. rank Q ′ = rank A .
TheoremMatrix completion for an m ×m mixed skew-symmetric matrix can be donein O(m4) time.
Using delta-covering algortihm of Geelen, Iwata & Murota ’03
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 24 / 29
1 Introduction
2 Matrix Completion by Rank-One Matrices
3 Application to Network Coding
4 Mixed Skew-Symmetric Matrix Completion
5 Skew-Symmetric Matrix Completion by Rank-Two Skew-SymmetricMatrices
6 Conclusion
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 25 / 29
Problem Definition
Skew-Symmetric Matrix Completion by Rank-Two Skew-SymmetricMatricesMatrix completion for A = B0 + x1B1 + · · ·+ xnBn,where B0 is skew-symmetric and B1, . . . ,Bn are rank-two skew-symmteric
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 26 / 29
Our Result
In the case of B0 = 0:
Lovasz ’89This can be reduced to linear matroid parity.
solvable in O(m3n) time using the algorithm of Gabow & Stallman ’86.
For the general case:
Our ResultAn optimal solution can be found in O((m + n)4) time.
Idea: Reduction to mixed skew-symmetric matrix completion(similar to matrix completion by rank-one matrices)
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 27 / 29
Our Result
In the case of B0 = 0:
Lovasz ’89This can be reduced to linear matroid parity.
solvable in O(m3n) time using the algorithm of Gabow & Stallman ’86.
For the general case:
Our ResultAn optimal solution can be found in O((m + n)4) time.
Idea: Reduction to mixed skew-symmetric matrix completion(similar to matrix completion by rank-one matrices)
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 27 / 29
1 Introduction
2 Matrix Completion by Rank-One Matrices
3 Application to Network Coding
4 Mixed Skew-Symmetric Matrix Completion
5 Skew-Symmetric Matrix Completion by Rank-Two Skew-SymmetricMatrices
6 Conclusion
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 28 / 29
Conclusion
Our ResultsFaster algorithm and Min-Max theorem for matrix completion byrank-one matrices.
Application for multicast problem with linearly correlated sources.
First deterministic polynomial time algorithm for mixedskew-symmetric matrix completion.
First deterministic polynomial time algorithm for skew-symmetricmatrix completion by rank-two skew-symmetric matrices.
Future WorksApplication of skew-symmetric matrix completion
Matrix completion for other types of matrices
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 29 / 29
Conclusion
Our ResultsFaster algorithm and Min-Max theorem for matrix completion byrank-one matrices.
Application for multicast problem with linearly correlated sources.
First deterministic polynomial time algorithm for mixedskew-symmetric matrix completion.
First deterministic polynomial time algorithm for skew-symmetricmatrix completion by rank-two skew-symmetric matrices.
Future WorksApplication of skew-symmetric matrix completion
Matrix completion for other types of matrices
Tasuku Soma (Kyoto Univ.) Fast Matrix Completion Algorithms 29 / 29