Preliminaries Properties of tensor rank Open problems The role of tensor rank in the complexity analysis of bilinear forms Dario A. Bini Dipartimento di Matematica, Universit` a di Pisa www.dm.unipi.it/ bini ICIAM07, Z¨ urich, 16-20 July 2007 Dario A. Bini The role of tensor rank
46
Embed
The role of tensor rank in the complexity analysis of ...lekheng/meetings/multilinear/bini.pdf · Dario A. Bini The role of tensor rank. Preliminaries Properties of tensor rank Open
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
PreliminariesProperties of tensor rank
Open problems
The role of tensor rank in the complexity analysisof bilinear forms
Dario A. Bini
Dipartimento di Matematica, Universita di Pisawww.dm.unipi.it/˜bini
ICIAM07, Zurich, 16-20 July 2007
Dario A. Bini The role of tensor rank
PreliminariesProperties of tensor rank
Open problems
1 PreliminariesTensorsTensor rankBilinear forms
2 Properties of tensor rankBorder rankLower bounds
3 Open problemsThe direct sum conjectureSome tensors of unknown rank
Dario A. Bini The role of tensor rank
PreliminariesProperties of tensor rank
Open problems
TensorsTensor rankBilinear forms
Tensors
Let F be a number field, say, R, C; tensors of the kind
A = (ai1,i2,...,ih) ∈ Fn1×n2×···×nh ,
that is, h-way arrays, are encountered in many problems of verydifferent nature [Comon 2001], [Comon, Golub, Lim, Mourrain,2006], [De Silva, Lim 2006]
Blind source separation
High order factor analysis
Independent component analysis
Candecomp/Parafac model
Complexity analysis
Psycometric, Chemometric, Economy,...
Dario A. Bini The role of tensor rank
PreliminariesProperties of tensor rank
Open problems
TensorsTensor rankBilinear forms
Tensors
It is surprising that little interplay occurred among thesedifferent research areas
Some properties have been rediscovered in different contexts
Apparently, results obtained in one field have not migrated tothe other fields
In this talk I wish to provide an overview of the main resultsconcerning tensors obtained in the research field of computationalcomplexity (starting from 1969) with the aim of
creating a synergic exchange of information between theseresearch areas
presenting problems which might be solved with the morerecent tools
presenting old results that might be adapted and extended tothe new needs
Dario A. Bini The role of tensor rank
PreliminariesProperties of tensor rank
Open problems
TensorsTensor rankBilinear forms
Tensors
It is surprising that little interplay occurred among thesedifferent research areas
Some properties have been rediscovered in different contexts
Apparently, results obtained in one field have not migrated tothe other fields
In this talk I wish to provide an overview of the main resultsconcerning tensors obtained in the research field of computationalcomplexity (starting from 1969) with the aim of
creating a synergic exchange of information between theseresearch areas
presenting problems which might be solved with the morerecent tools
presenting old results that might be adapted and extended tothe new needs
Dario A. Bini The role of tensor rank
PreliminariesProperties of tensor rank
Open problems
TensorsTensor rankBilinear forms
Tensor rank
Definition (Hitchcock 1927)
A tensor T = (ti1,...,ih) has rank 1 if there exist vectors
u(k) = (u(k)i ) ∈ Fnk , k = 1 : h such that ti1,...,ih = u
(1)i1
u(2)i2· · · u(h)
ih,
T = u(1) ◦ u(2) ◦ · · · ◦ u(h)
Definition (Hitchcock 1927)
The tensor rank rk(A) of A = (ai1,...,ih) is the minimum number rof rank-1 tensors Ti ∈ Fn1×···×nh such that
A = T1 + T2 + · · ·+ Tr canonical decomposition
For matrices, the tensor rank coincides with the customary rank
For simplicity of notation we restrict ourselves to the case h = 3,i.e., to three-way arrays.
U = (ui ,j) ∈ Fm×r , V = (vi ,j) ∈ Fn×r , W = (wi ,j) ∈ Fp×r ,
whose columns are the vectors u(i), v(i), w(i), i = 1 : r ,respectively.
Dario A. Bini The role of tensor rank
PreliminariesProperties of tensor rank
Open problems
TensorsTensor rankBilinear forms
Some remarks
A tensor A = (ai ,j ,k) ∈ Fm×n×p, can be represented by means ofthe set of the 3-slabs Ak = (ai ,j ,k)i ,j ∈ Fm×n of A or by a singlematrix of variables
A =
p∑k=1
skAk
A =
[[1 00 −1
];
[0 11 0
]]↔
[s1 s2s2 −s1
]
This suggests a different point of view for tensor rank:
rk(A) is the minimum set of rank one matrices which span the linearspace generated by the 3-slabs A1, . . . ,Ap [Gastinel 71, Fiduccia 72].
Dario A. Bini The role of tensor rank
PreliminariesProperties of tensor rank
Open problems
TensorsTensor rankBilinear forms
Some remarks
A tensor A = (ai ,j ,k) ∈ Fm×n×p, can be represented by means ofthe set of the 3-slabs Ak = (ai ,j ,k)i ,j ∈ Fm×n of A or by a singlematrix of variables
A =
p∑k=1
skAk
A =
[[1 00 −1
];
[0 11 0
]]↔
[s1 s2s2 −s1
]This suggests a different point of view for tensor rank:
rk(A) is the minimum set of rank one matrices which span the linearspace generated by the 3-slabs A1, . . . ,Ap [Gastinel 71, Fiduccia 72].
Dario A. Bini The role of tensor rank
PreliminariesProperties of tensor rank
Open problems
TensorsTensor rankBilinear forms
Bilinear forms
Problem: Given matrices Ak = (ai ,j ,k)i=1:m,j=1:n, k = 1 : pcompute the set of bilinear forms
fk(x, y) = xTAky, k = 1 : p
with the minimum number of nonscalar multiplications with no useof commutativity (noncommutative bilinear complexity)
A nonscalar multiplication is a multiplication of the kind
s = (m∑
i=1
αixi )(n∑
j=1
βjyj), αi , βj ∈ F
Remark: The set of bilinear forms is uniquely determined by thetensor A = (ai ,j ,k).
Dario A. Bini The role of tensor rank
PreliminariesProperties of tensor rank
Open problems
TensorsTensor rankBilinear forms
Bilinear forms
A canonical decomposition of the tensor A associated with the setof bilinear forms provides an algorithm of complexity r . In fact
A =r∑
`=1
u(`) ◦ v(`) ◦w(`) → Ak =r∑
`=1
wk,`u(`) ◦ v(`)
→ xTAky =r∑
`=1
wk,`(xTu(`))(v(`)Ty)
Theorem (Strassen 1975)
The noncommutative bilinear complexity of a set of bilinear formsfk(x, y) = xTAky, k = 1 : p, Ak = (ai ,j ,k), is given by the tensorrank of the associated tensor A = (ai ,j ,k).
brk(A) is the minimum number of nonscalar multiplicationssufficient to approximate the set of bilinear forms associatedwith A with arbitrarily small nonzero error
Dario A. Bini The role of tensor rank
PreliminariesProperties of tensor rank
Open problems
Border rankLower bounds
More on rank and border rank
Let Aε = A+ Eε be such that
rk(Aε) = brk(A)the entries of Eε are polynomials of degree d
Then
rk(A) ≤ (d + 1)brk(A)
Proof:
Write d + 1 copies of the canonical decomposition of lengthrk(Aε) obtained with (d + 1) pairwise different values of εTake linear combinations of these decompositions withcoefficients γj , j = 1 : d + 1 in order to kill the terms in εi ,i = 1 : d and to have
∑γj = 1
Obtain a decomposition of length (d + 1)brk(A) with no error
Dario A. Bini The role of tensor rank
PreliminariesProperties of tensor rank
Open problems
Border rankLower bounds
More on rank and border rank
Let Aε = A+ Eε be such that
rk(Aε) = brk(A)the entries of Eε are polynomials of degree d
Then
rk(A) ≤ (d + 1)brk(A)
Proof:
Write d + 1 copies of the canonical decomposition of lengthrk(Aε) obtained with (d + 1) pairwise different values of ε
Take linear combinations of these decompositions withcoefficients γj , j = 1 : d + 1 in order to kill the terms in εi ,i = 1 : d and to have
∑γj = 1
Obtain a decomposition of length (d + 1)brk(A) with no error
Dario A. Bini The role of tensor rank
PreliminariesProperties of tensor rank
Open problems
Border rankLower bounds
More on rank and border rank
Let Aε = A+ Eε be such that
rk(Aε) = brk(A)the entries of Eε are polynomials of degree d
Then
rk(A) ≤ (d + 1)brk(A)
Proof:
Write d + 1 copies of the canonical decomposition of lengthrk(Aε) obtained with (d + 1) pairwise different values of εTake linear combinations of these decompositions withcoefficients γj , j = 1 : d + 1 in order to kill the terms in εi ,i = 1 : d and to have
∑γj = 1
Obtain a decomposition of length (d + 1)brk(A) with no error
Dario A. Bini The role of tensor rank
PreliminariesProperties of tensor rank
Open problems
Border rankLower bounds
More on rank and border rank
Let Aε = A+ Eε be such that
rk(Aε) = brk(A)the entries of Eε are polynomials of degree d
Then
rk(A) ≤ (d + 1)brk(A)
Proof:
Write d + 1 copies of the canonical decomposition of lengthrk(Aε) obtained with (d + 1) pairwise different values of εTake linear combinations of these decompositions withcoefficients γj , j = 1 : d + 1 in order to kill the terms in εi ,i = 1 : d and to have
∑γj = 1
Obtain a decomposition of length (d + 1)brk(A) with no error
Dario A. Bini The role of tensor rank
PreliminariesProperties of tensor rank
Open problems
Border rankLower bounds
Lower bounds and linear algebra
Simple criteria for providing lower bounds on tensor rank andborder rank can be givenTrivial bounds
brk(A) ≥ dim(span(A1, . . . ,Ap))
Similar inequalities are valid w.r.t. the other coordinates
Assume for simplicity p = dim(span(A1, . . . ,Ap))
Dario A. Bini The role of tensor rank
PreliminariesProperties of tensor rank
Open problems
Border rankLower bounds
Lower bounds and linear algebra
Let U,V ,W be the matrices defining a canonical factorization ofA of length rk(A)
A =
rk(A)∑`=1
u(`) ◦ v(`) ◦w(`)
Assume w.l.o.g. that the first p columns of W are linearlyindependent, that is
W =[
W1 W2
], W1 ∈ Fp×p, det W1 6= 0
Let A′k =
∑pj=1 w
(−1)k,j Aj , define
A′ = [A′1,A
′2, . . . ,A
′p] = A •3 W−1
1 ,
i.e., choose a different basis to represent the space spanned by theslabs A1, . . . ,Ap
Dario A. Bini The role of tensor rank
PreliminariesProperties of tensor rank
Open problems
Border rankLower bounds
Lower bounds and linear algebra
Let U,V ,W be the matrices defining a canonical factorization ofA of length rk(A)
A =
rk(A)∑`=1
u(`) ◦ v(`) ◦w(`)
Assume w.l.o.g. that the first p columns of W are linearlyindependent, that is
W =[
W1 W2
], W1 ∈ Fp×p, det W1 6= 0
Let A′k =
∑pj=1 w
(−1)k,j Aj , define
A′ = [A′1,A
′2, . . . ,A
′p] = A •3 W−1
1 ,
i.e., choose a different basis to represent the space spanned by theslabs A1, . . . ,Ap
Dario A. Bini The role of tensor rank
PreliminariesProperties of tensor rank
Open problems
Border rankLower bounds
Lower bounds and linear algebra
Let U,V ,W be the matrices defining a canonical factorization ofA of length rk(A)
A =
rk(A)∑`=1
u(`) ◦ v(`) ◦w(`)
Assume w.l.o.g. that the first p columns of W are linearlyindependent, that is
W =[
W1 W2
], W1 ∈ Fp×p, det W1 6= 0
Let A′k =
∑pj=1 w
(−1)k,j Aj , define
A′ = [A′1,A
′2, . . . ,A
′p] = A •3 W−1
1 ,
i.e., choose a different basis to represent the space spanned by theslabs A1, . . . ,Ap
Dario A. Bini The role of tensor rank
PreliminariesProperties of tensor rank
Open problems
Border rankLower bounds
Evidently, A′ has the same rank of A and a canonicaldecomposition is given by U ′ = U,V ′ = V ,W ′ = W−1