Page 1
×××××××××××××××××××××××××××××××××××××××××××
×
On Deflations in Extended QRAlgorithms
Thomas Mach Raf Vandebril
Dept. Computer Science, KU Leuven, Belgium
GAMM Workshop Appl. Numer. Linear Algebra09 September 2013Wuppertal, Germany
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 1/27
Page 2
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Hessenberg Matrix
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××
=
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××
QR factorization of the Hessenberg matrixRotation of the form:
[c s−s c
], |c |2 + |s|2 = 1
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 2/27
Page 3
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Hessenberg Matrix
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××
=
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
0×××××
��
QR factorization of the Hessenberg matrixRotation of the form:
[c s−s c
], |c |2 + |s|2 = 1
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 2/27
Page 4
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Hessenberg Matrix
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××
=
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
0×××××
��
QR factorization of the Hessenberg matrix
Rotation of the form:
[c s−s c
], |c |2 + |s|2 = 1
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 2/27
Page 5
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Hessenberg Matrix
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××
=
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
0××××
����
QR factorization of the Hessenberg matrixRotation of the form:
[c s−s c
], |c |2 + |s|2 = 1
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 2/27
Page 6
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Hessenberg Matrix
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××
=
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
0×××
����
��
QR factorization of the Hessenberg matrixRotation of the form:
[c s−s c
], |c |2 + |s|2 = 1
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 2/27
Page 7
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Hessenberg Matrix
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××
=
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
0××
����
����
QR factorization of the Hessenberg matrixRotation of the form:
[c s−s c
], |c |2 + |s|2 = 1
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 2/27
Page 8
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Hessenberg Matrix
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××
=
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
0×
����
����
��
QR factorization of the Hessenberg matrixRotation of the form:
[c s−s c
], |c |2 + |s|2 = 1
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 2/27
Page 9
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Hessenberg Matrix
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××
=
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××0
����
����
����
QR factorization of the Hessenberg matrixRotation of the form:
[c s−s c
], |c |2 + |s|2 = 1
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 2/27
Page 10
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Hessenberg Matrix
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××
=
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
QR factorization of the Hessenberg matrix
Rotation of the form:
[c s−s c
], |c |2 + |s|2 = 1
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 2/27
Page 11
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Extended Hessenberg Matrix
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
There is a relation to extended Krylov subspaces.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 3/27
Page 12
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Rotators
Fusion:
�� �� = c1−s1
s1c1
c2−s2
s2c2
= c3−s3
s3c3
= ��
Turnover:
����
��=
Ö
Ö
Ö
Ö
Ö
Ö
Ö
Ö
Ö
= ����
��
Transfer Through Upper Triangular:
����
��Ö
Ö Ö Ö Ö
Ö Ö Ö
Ö Ö=
��
��Ö
Ö Ö Ö Ö
Ö Ö Ö
Ö Ö Ö=
��
��
Ö
Ö
Ö
Ö
Ö
Ö
Ö
Ö
Ö
Ö
� �
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 4/27
Page 13
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××
1) Compute bulge.
A) Eigenvalues of trailing matrix provide shift.B) x = p(A)e1 = (A− µmI ) · · · (A− µ1I )e1.C) Q0x = γe1.D) Apply Q0.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 5/27
Page 14
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××
1) Compute bulge.
A) Eigenvalues of trailing matrix provide shift.B) x = p(A)e1 = (A− µ1I )e1.C) Q0x = γe1.D) Apply Q0.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 5/27
Page 15
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××
��
� �
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 5/27
Page 16
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××
×
2) Bulge chasing.
A) Compute zeroing similarity transformation.B) Apply transformation.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 5/27
Page 17
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××
��
� �
2) Bulge chasing.
A) Compute zeroing similarity transformation.B) Apply transformation.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 5/27
Page 18
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××
×
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 5/27
Page 19
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××
��
� �
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 5/27
Page 20
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××
×
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 5/27
Page 21
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××
��
� �
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 5/27
Page 22
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××
×
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 5/27
Page 23
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××
��
� �
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 5/27
Page 24
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
×××××××
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 5/27
Page 25
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
×××××××
��� �
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 5/27
Page 26
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××
Why Francis’s Algorithm Works [Watkins 1982, ’07, ’08, ’11]
Power method.
Krylov subspaces.
Shifts accelerate convergence.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 5/27
Page 27
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
QR Factorization
QR Factorization
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/27
Page 28
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Algorithm on the QR Factorization[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/27
Page 29
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Algorithm on the QR Factorization[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
� �
��
Fusion
Transfer through up. triang.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/27
Page 30
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Algorithm on the QR Factorization[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
��
Turnover
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/27
Page 31
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Algorithm on the QR Factorization[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
��
Similarity transformation
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/27
Page 32
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Algorithm on the QR Factorization[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
� �
Transfer through up. triang.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/27
Page 33
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Algorithm on the QR Factorization[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
��
Turnover
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/27
Page 34
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Algorithm on the QR Factorization[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
��
Similarity transformation
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/27
Page 35
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Algorithm on the QR Factorization[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
� �
Transfer through up. triang.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/27
Page 36
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Algorithm on the QR Factorization[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
��
Turnover
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/27
Page 37
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Algorithm on the QR Factorization[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
��
Similarity transformation
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/27
Page 38
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Algorithm on the QR Factorization[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
� �
Transfer through up. triang.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/27
Page 39
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Algorithm on the QR Factorization[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
��
Turnover
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/27
Page 40
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Algorithm on the QR Factorization[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
��
Similarity transformation
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/27
Page 41
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Algorithm on the QR Factorization[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
� �
Transfer through up. triang.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/27
Page 42
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Algorithm on the QR Factorization[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
��
Turnover
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/27
Page 43
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Algorithm on the QR Factorization[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
������
Similarity transformation
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/27
Page 44
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Algorithm on the QR Factorization[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
� �
Transfer through up. triang.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/27
Page 45
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Algorithm on the QR Factorization[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
���� ��
Fusion
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/27
Page 46
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Francis’s Algorithm on the QR Factorization[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/27
Page 47
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 8/27
Page 48
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
� �
��
Fusion
Transfer through up. triang.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 8/27
Page 49
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
��
Turnover
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 8/27
Page 50
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
��
Similarity transformation
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 8/27
Page 51
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
� �
Transfer through up. triang.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 8/27
Page 52
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
��
Move pattern
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 8/27
Page 53
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
��
Turnover
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 8/27
Page 54
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
��
Transfer through up. triang.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 8/27
Page 55
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
� �
Similarity transformation
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 8/27
Page 56
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
��
Turnover
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 8/27
Page 57
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
��
Transfer through up. triang.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 8/27
Page 58
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
� �
Similarity transformation
Move pattern
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 8/27
Page 59
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
����
����
��
Turnover
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 8/27
Page 60
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
��
��
������
Similarity transformation
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 8/27
Page 61
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
��
��
����
� �
Transfer through up. triang.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 8/27
Page 62
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
��
��
���� ��
Turnover
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 8/27
Page 63
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
��
��
����
��
Similarity transformation
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 8/27
Page 64
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
��
��
����
� �
Transfer through up. triang.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 8/27
Page 65
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
����
��
��
����
��
Fusion
Move pattern
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 8/27
Page 66
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
��
��
����
����
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 8/27
Page 67
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]
× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
��
��
����
����
Why the Extended QR Algorithm Works
Power method.
Rational Krylov subspaces.
Shifts accelerate convergence.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 8/27
Page 68
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Standard Deflation
Standard Deflation
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 9/27
Page 69
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Deflation Criterion
× × × × ×× × × ×× × ×× ××
××××
=
����
����
× × × × ×× × × ×× × ×× ××
A) |H(k + 1, k)| ≤ ε ‖H‖F
B) |H(k + 1, k)| ≤ ε (|H(k , k)|+ |H(k + 1, k + 1)|)
A = Gτ(1)Gτ(2) · · ·Gτ(n−1)R
∥∥I − Gτ(i)
∥∥2≤ ε
A = Gτ(1)Gτ(2) · · ·Gτ(i−1)Gτ(i+1) · · ·Gτ(n−1)R
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 10/27
Page 70
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Why is that a Useful Deflation Criterion?
Why is that a useful deflation criterion?
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 11/27
Page 71
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Bauer-Fike Theorem
Bauer-Fike Theorem [Bauer, Fike 1960]
minµ∈Λ(A)
|λ− µ| ≤ κ2(X )‖A− A‖2
‖A− A‖ ≤ ‖Gτ(1) · · ·Gτ(i−1)‖ ‖Gτ(i) − I‖ ‖Gτ(i+1) · · ·Gτ(n−1)‖ ‖R‖≤ ‖Gτ(i) − I‖ ‖R‖≤ ‖Gτ(i) − I‖ ‖A‖
Lemma
Let A be diagonalizable, A = XΛX−1, and λ an eigenvalue of A.Then there exists an eigenvalue µ of A, so that
|λ− µ| ≤ κ2(X )‖Gτ(i) − I‖2‖A‖2.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 12/27
Page 72
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
But we can do better:
Theorem [Eisenstat, Ipsen 1995]
A = A1A2, E = A− A
mini
|λ−µi ||µi | ≤ κ2(X )
∥∥A−11 EA−1
2
∥∥2.
A1 = Gτ(1) · · ·Gτ(i) and A2 = Gτ(i+1) · · ·Gτ(n−1)R
⇒ A−11 EA−1
2 = I − G−1τ(i)
‖I − G−1τ(i)‖2 = ‖G−1
τ(i)(Gτ(i) − I )‖2 ≤ ‖Gτ(i) − I‖2
Lemma
Let A be diagonalizable, A = XΛX−1, and λ an eigenvalue of A.Then there exists an eigenvalue µ of A, so that
|λ−µ||µ| ≤ κ2(X )‖Gτ(i) − I‖2.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 13/27
Page 73
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Conditioning of One Eigenpair
Theorem [Wilkinson 1965]
Let
µ ∈ Λ(A), Ax = µx , yHA = µyH , be a simple eigenvalue,
E = εF , with ‖F‖2 = 1.
For µ = µ(ε) ∈ Λ(A + E ) with µ(0) = λ, we have
|µ− λ| ≤ ‖E‖2
yH x+O(‖E‖2
2).
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 14/27
Page 74
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Relative Perturbation Estimate
Following the proofs of [Eisenstat, Ipsen 1995] we can proof thefollowing theorem.
Theorem [M., Vandebril ’13]
Let
A be invertible,
µ ∈ Λ(A), Ax = µx , yHA = µyH , be a simple eigenvalue,
E = εF , with ‖F‖2 = 1.
For µ = µ(ε) ∈ Λ(A + E ) with µ(0) = λ, we have
|µ−λ||λ| ≤
‖A−1E‖2
yH x+O(
∥∥A−1E∥∥2
2).
If A = A1A2 and A1 and A2 invertible, then
|µ−λ||λ| ≤
‖A−11 EA−1
2 ‖2
yH x+O
(‖A−1
1 EA−12 ‖2
2
).
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 15/27
Page 75
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Very Good Relative Perturbation Estimate
Corollary [M., Vandebril ’13]
Let A = Gτ(1)Gτ(2) · · ·Gτ(n−1)R and
A = Gτ(1)Gτ(2) · · ·Gτ(i−1)Gτ(i+1) · · ·Gτ(n−1)R. Let further A be
nonsingular, and let λ be a single eigenvalue of A. Then thereexists an eigenvalue µ of A, so that
|µ−λ||λ| ≤
‖I−Gτ(i)‖2
xH y+O
(‖I − Gτ(i)‖2
2
),
with x the right and y the left eigenvector to λ.
The deflation criterion∥∥I − Gτ(i)
∥∥2≤ ε is very good.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 16/27
Page 76
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Aggressive Early Deflation (AED)[Braman, Byers, Mathias ’02]
Aggressive Early Deflation
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 17/27
Page 77
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Aggressive Early Deflation (AED)[Braman, Byers, Mathias ’02]
× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
×××××××××
A
⇒
× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
××××××⊗⊗⊗
A′ = QHAQ
VH
× × × ×× × ×× ××
V
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 18/27
Page 78
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
AED for Extended QR Algorithms
����
����
����
����
��
× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 19/27
Page 79
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
AED for Extended QR Algorithms
× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 19/27
Page 80
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
AED for Extended QR Algorithms
× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ×××××
×××
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 19/27
Page 81
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
AED for Extended QR Algorithms
× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
VH V
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 19/27
Page 82
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
AED for Extended QR Algorithms
× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
���� ��
�� �� ��
���� ��
�� �� ��
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 19/27
Page 83
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
AED for Extended QR Algorithms
× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
���� ��
�� �� ��
������
������
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 19/27
Page 84
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
AED for Extended QR Algorithms
× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
���� ��×
�� �� ��× ×
������ ×
���� ××��
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 19/27
Page 85
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
AED for Extended QR Algorithms
× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 19/27
Page 86
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Wedge-to-Broadhead Transformation
����
�� =�� �
���
����
����
����
��=
����
�� ��
��
����
=
�� ��
�
�
�
�
�
�
��
��
Test for deflations.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 20/27
Page 87
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Restore Original Pattern
×××××××××××××××××××××
⇒×××××××××××××××××××××
⇒×××××××××××××××××××××
⇒
×××××××××××××××××××××
⇒×××××××××××××××××××××
⇒×××××××××××××××××××××
⇒
×××××××××××××××××××××
⇒×××××××××××××××××××××
⇒×××××××××××××××××××××
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 21/27
Page 88
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
The effect of aggressive early deflation for n = 500
1020
3040
50
020
4060
80100
0
200
400
*
*
# shifts#
deflation window
run
tim
ein
s
aggressive early deflationstandard deflation only
best without early deflation best with early deflation savingsn sh t #it·sh sh m t #it·sh
100 4 6.00 s 215 10 18 5.62 s 223 6% time100 2 6.42 s 198 2 28 8.60 s 163 18% iterations500 6 148.49 s 803 20 20 100.45 s 727 32% time500 2 161.38 s 732 2 70 314.85 s 391 46% iterations
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 22/27
Page 89
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
The effect of aggressive early deflation for n = 500
1020
3040
50
020
4060
80100
0
200
400
*
*
# shifts#
deflation window
run
tim
ein
s
aggressive early deflationstandard deflation only
best without early deflation best with early deflation savingsn sh t #it·sh sh m t #it·sh
100 4 6.00 s 215 10 18 5.62 s 223 6% time100 2 6.42 s 198 2 28 8.60 s 163 18% iterations500 6 148.49 s 803 20 20 100.45 s 727 32% time500 2 161.38 s 732 2 70 314.85 s 391 46% iterations
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 22/27
Page 90
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Middle Deflation
Middle Deflation
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 23/27
Page 91
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Middle Deflation
× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 24/27
Page 92
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Middle Deflation
× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 24/27
Page 93
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Middle Deflation
× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 24/27
Page 94
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Middle Deflation
× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
V VH V VH
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 24/27
Page 95
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Middle Deflation
���� ��
�� �� ��×
× ×
���� ��
������×××
����������
��
× ××
�� �� ���� ��
��
× ××
× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 24/27
Page 96
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Middle Deflation
× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 24/27
Page 97
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Middle Deflation
× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 24/27
Page 98
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Predicting Middle Deflations
One needs to define a deflation window that most likely leads to amiddle deflation.
g∏i=f
|si | < ε2
For random matrices a successful middle deflation window of size0.75 · n × 0.75 · n is found after a few iterations.
A ∈ C1500×1500, random, three multishift steps (150 shifts),deflation window 363:1026, leads to middle deflation
The corresponding deflation windows for other test-matrices(tols2000, pde2961) have not been successful.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 25/27
Page 99
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Predicting Middle Deflations
One needs to define a deflation window that most likely leads to amiddle deflation.
g∏i=f
|si | < ε2
For random matrices a successful middle deflation window of size0.75 · n × 0.75 · n is found after a few iterations.
A ∈ C1500×1500, random, three multishift steps (150 shifts),deflation window 363:1026, leads to middle deflation
The corresponding deflation windows for other test-matrices(tols2000, pde2961) have not been successful.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 25/27
Page 100
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Conclusions
We have investigated deflations for extended QR algorithms.
Deflating almost diagonal rotations preserves absolute andrelative accuracy also with respect to the conditioning ofsingle eigenvalues.
We have seen aggressive early deflation in extended QRalgorithms.
These ideas can be used for middle deflations in special cases.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 26/27
Page 101
Basics Standard Deflation Aggressive Early Deflation Middle Deflation
Acknowledgments
The research was partially supported by
the Research Council KU Leuven
CREA-13-012 Can Unconventional Eigenvalue AlgorithmsSupersede the State of the Art (CUASSA),F+/13/020 Exploiting Unconventional QR-Algorithms for Fastand Accurate Computations of Roots of Polynomials,OT/11/055 Spectral Properties of Perturbed Normal Matricesand their Applications,CoE EF/05/006 Optimization in Engineering (OPTEC),
DFG research stipend MA 5852/1-1,
Fund for Scientific Research–Flanders (Belgium) project G034212N,Reestablishing Smoothness for Matrix Manifold Optimization viaResolution of Singularities,
Interuniversity Attraction Poles Programme, initiated by the BelgianState, Science Policy Office, Belgian Network DYSCO – DynamicalSystems, Control, and Optimization.
Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 27/27