Gen inversion

7/24/2019 Gen inversion

http://slidepdf.com/reader/full/gen-inversion 1/6

R

i N

J . F . T R A U B , E d i t o r

A S i mp l e A l gor i th m for

C omp u t i n g th e Ge n e r a l i z e d

Inverse of a Matr ix

B. RUST

O a k R i d g e G a s e o u s D i f f u s i o n P l a n t

W . R . B U R R U S A N D C .

SCHNEEBERGER*

O a £ R i d g e N a t i o n a l L a b o r a t o r y

Oak R i dge , T ennessee

T h e g e n e r a l i z e d i n v e r s e o f a m a t r i x i s i m p o r t a n t i n a n a l y s i s

b e c a u s e i t p r o v i d e s a n e x t e n s i o n o f t h e c o n c e p t o f a n i n v e r s e

w h i c h a p p l i e s t o a l l m a t r i c e s . I t a l s o h a s m a n y a p p l i c a t i o n s i n

n u m e r i c a l a n a l y s i s , b u t i t i s n o t w i d e l y u s e d b e c a u s e t h e e x i s t -

i n g a l g o r it h m s a r e f a i r l y c o m p l i c a t e d a n d r e q u i r e c o n s i d e r -

a b l e s t o r a g e s p a c e . A s i m p l e e x t e n s i o n h a s b e e n f o u n d t o

t h e c o n v e n t i o n a l o r t h o g o n a l i z a t i o n m e t h o d f o r i n v e r t in g n o n -

s i n g u l a r m a t r ic e s , w h ic h g i v e s th e g e n e r a l i z e d i n v e r s e w it h

l it t le e x t r a e f f o r t a n d w i t h n o a d d i t i o n a l s t o r a g e r e q u i r e m e n t s .

T h e a l g o r i th m g i v e s t h e g e n e r a l i z e d i n v e r s e f o r a n y m b y n

m a t r i x A , i n c lu d i n g t h e s p e c i a l c a s e w h e n m = n a n d A is

n o n s i n g u l a r a n d t h e c a s e w h e n m > n a n d r a n k ( A ) = n .

I n t h e f ir s t c a s e t h e a l g o r i th m g i v e s t h e o r d i n a r y i n v e r s e o f A .

I n t h e s e c o n d c a s e th e a l g o r i th m y i e l d s t h e o r d i n a r y l e a s t

s q u a r e s t ra n s f o r m a t i o n m a t r i x A T A ) - I A T a n d h a s th e a d -

v a n t a g e o f a v o i d i n g t h e l o s s o f s i g n i f ic a n c e w h i c h r e su l ts

i n f o r m i n g t h e p r o d u c t A T A e x p l i c i t l y .

The generalized inverse is an import ant concept in

matrix theory because it provdes an extension of the con-

cept of an inverse which applies to all matrices. Penrose

[1] showed tha t for any m X n complex matrix A there

exists a uniqu e n X m matr ix X wh ich satisfies the follow-

ing relations:

A X A = A (1)

XA X = z

(2)

( A X ) H = A X (3)

( X A ) g = X A . (4)

These four relations are often called Penrose's Lernmas,

and the anatrix X is said to be the generalized inverse of A

Resenrch was sponsored by the US Atomic Energy Commission

under contract with the Union Carbide Corporation.

* Student employee, Department of Mathematics, University

of Michigan.

and is often de noted by A t. I n th e special case where

m = n and A is nonsingular, this generalized inverse is

simply the ordinary inverse of A. Also, in the special case

where m > n and the co lumns of A are linearly indepen d-

ent, we can write

A ' = ( A ' A ) - I A H. (5)

It is an easy matter to see that this matrix satisfies all of

Penrose's Lemmas. It is important in numerical analysis

because it solves the problem of minimizing the distance

p ( x ) = [ b - - A x [ ,

where b is a given vecto r in m-space and A is a given m X n

matrix with m > n and linearly independen t columns.

More generally, if A is any m X n mat rix a nd b is any

vector in m-space, there may exist many vectors x which

minimize the distance p ( x ) , but the vector defined by

x = A~b

(6)

is the shortest of all such vectors. The problem of finding

the vector x of shortest length Ix t which minimizes the

distance p ( x ) may be referred to as the generalized least

squares problem. It is solved by the generalized inverse.

Suppose that the matrix A can be partitioned in the fol-

lowing manner:

A = (R, S) (7)

where R is an (m X k )-ma trix (k < n) wit h linearly inde-

pendent columns and S is an (m X (n - k))- matr ix

whose columns are linear combinations of the columns

of R.

TEEOR.EM I.

R r R = I .

(8)

PROOF. The columns of R are linearly independent.

Therefore, by (5), R ~ = ( R H R ) - i R H. Hence R ~ R =

( R H R ) - I R H R = I .

THEOREM II. T he mat r i x S has a un i q ue f ac t or i za t i on

i n t he f orm

s = RU

(9)

and t he mat r i x U i s g i ven by

U = R ' S . (10)

P ro of t ha t t he fac t or i za t i on ex i s ts . Suppose

S = ( s ~ + l , s ~ + ~ , . . - , s , ) .

Each column of S is a linear combination of the columns

of R. Therefore s~ = R u~ from some vector u~, i = k+ l,

V o l u m e

9 / Numbe r 5 / May, 1966

C o m m u n i c a t i o n s o f t h e

ACM 381



• • •

~ n . t - I en ce

S = ( R u k + ~ , R u k + 2 , . . . , R u n )

= R ( u k + l , u k + 2 , • . . , u n ) ,

i .e. , S = R U w h e r e U = ( u k + l , uk + 2 , . . - , U n ) .

P r o o f t h a t t h e f a c l o r i z a t i o n i s u n i q u e . I t h a s j u st b e e n

s h o w n t h a t S = R U f o r s o m e U . T h e r e f o r e R I S = R r R U

= U s i n c e b y ( 8 ) ,

R ~ R = I .

T h u s

U = R ~ S .

W e n o w s h o w t h a t t h e p r o b l e m o f c o m p u t i n g t h e g e n -

e r a l i z e d i n v e r s e o f a n y m a t r i x A c a n b e r e d u c e d t o t h e

p r o b l e m o f c o m p u t i n g t h e g e n e r a l i z e d in v e r s e o f a m a t r i x

o f t h e f o r m A ' = ( R , S ) , w h e r e R a n d S a r e m a t r i c e s o f

t h e s a m e f o r m a s t h e R a n d S o f e q . ( 7 ) .

T H E O R E M I I I .

I f P i j i s a n y n l h o r d e r e l e m e n t a r y p e r -

m u t a t i o n m a t r i x t h e n ( A P i j ) ~ = P ~ i A r .

T h e t r u t h o f t h is t h e o r e m c a n e a si ly b e d e m o n s t r a t e d

p i

b y s h o w i n g t h a t t h e m a t r i x ~ jA d o e s a c t u a l l y s a t is f y

P e n r o s e ' s L e m m a s . F u r t h e r m o r e i t is e a s y t o s e e t h a t i f

P ~ , P 2 , • , P ~ i s a n y f i n it e s e t o f e l e m e n t a r y p e r m u t a -

t i o n m a t r i c e s , t h e n

( A P ~ P 2 P ~ )~ = P ~ . P 2 P ~ A ' .

( 1 1 )

T h u s i f A i s a n y m X n m a t r i x i t c a n b e r e d u c e d t o t h e

f o r m

A ' = A P ~ P 2

. . . P ~ = ( R , S ) w i t h a l l t h e l i n e a r l y

d e p e n d e n t c o l u m n s ( i f a n y ) o c c u r r i n g la s t . T h e n b y e q .

( 1 1 ) ,

P ~ . . . P 2 P 1 A I = ( R , S ) ~

a n d h e n c e

A ~ = P i P 2 ' P ~ ( R , S ) ' . ( 1 2 )

T h u s i t i s n o w n e c e s s a r y o M y to c o n s i d e r t h e p r o b l e m

o f c o m p u t i n g t h e g e n e r a l i z e d i n v e r s e o f m a t r i c e s o f t h e

f o r m A = ( R , S ) .

T o o b t a i n a n e x p r e s s i o n f o r t h e g e n e r M i z e d in v e r s e i n

t e r m s o f t h e m a t r i c e s R a n d U , w e a p p e a l t o t h e l e a s t

s q u a r e s p r o p e r t y o f A ' . L e t u s c o n f in e o u r s e l v e s f o r t h e

t i m e b e i n g t o r e a l m a t r i c e s A . T h e r e s u l t s w h i c h w e s h a l l

o b t a i n c a n e a s i l y b e g e n e r a l i z e d t o t h e c o m p l e x c a s e .

C o n s i d e r t h e s y s t e m

A s = b , ( 1 3 )

w h e r e b i s a n y v e c t o r i n t h e c o l u m n s p a c e o f A ; i .e ., t h e

s y s t e m w i l l h a v e e x a c t s o l u ti o n s . I n t h i s c a s e a ll t h e l e a s t

s q u a r e s s o l u t i o n s w i ll h a v e t h e p r o p e r t y

p ( s ) = ] b - - A s ]

= 0 , a n d t h e s h o r t e s t s u c h s is g i v e n b y s = A ~ b .

C o n s i d e r f or a m o m e n t t h e p r o b l e m o f m i n im i z i n g t h e

l e n g t h s w i t h t h e r e s t r i c t i o n A s = b . T h i s i s e q u i v a l e n t t o

m i n i m i z i n g ] s [ 2 = srs w i t h t h e r e s t r i c t i o n A s = b . A s -

s u m e t h a t A h a s a p a r t i t i o n i n g ( R , S ) w i t h M 1 t h e l i n e a r l y

d e p e n d e n t c o l u n m s l a s t a n d p a r t i t i o n S a s f o l lo w s :

= ; )

w h e r e x i s a v e c t o r o f o r d e r k a n d y is a v e c t o r o f o r d e r

n - k . T h e n t h e p r o b l e m i s t o m i n i m i z e th e q u a n t i t y

( xr , y r ) ( ~ ) = X T X + y r y ,

w i t h t h e r e s t r i c t i o n t h a t

o r s i m p l y

b = O ,

R x + S y - b = O.

L e t u s a p p l y t h e m e t h o d o f L a g r a n g e m u l t ip l ie r s . S e t

L = x r x + y T y + z r [ R x + S y - - b] ,

w h e r e z is t h e v e c t o r o f p a r a m e t e r s t o b e e l i m i n a t e d . S i n c e

b y e q s. ( 9 ) a n d ( 1 0 ) ,

S = R U

w h e r e

U = R ~ S ,

w e c a n

w r i t e

L = x r x + y r y + z r [ R x + R U y - b ].

D i f f e r e n t i a t i n g L w i t h r e s p e c t t o e a c h e l e m e n t o f t h e

v e c t o r s x a n d y a n d s e t t i n g t h e s e d e r i v a t i v e s e q u a l t o z e r o

g i v e s

O L

- 2 x + R r z = 0 ( 1 4 )

O x

O L

- 2 y + U r R r z

= 0 ( 1 5 )

O y

w h e r e O L / O x i s t h e v e c t o r w h o s e e l e m e n t s a r e th e d e r i v a -

t i v e s o f L w i t h r e s p e c t t o t h e c o r r e s p o n d i n g e l e m e n t s o f x

a n d O L / O y h a s a s i m i l a r i n t e r p r e t a t i o n . A d d i n g t h e r e -

s t r i c t i o n

R x + R U y - b = 0 , (16 )

e n a b l e s u s t o e l i m i n a t e t h e v e c t o r z a n d s o l v e f o r x a n d y .

P r e m u l t i p l y i n g e q . ( 1 4 ) b y U r g iv e s

2 U T x + u T R T z = O.

C o m b i n i n g t h i s r e s u lt w i t h e q . ( 1 5 ) g i v e s 2 y = 2 U r x o r

y = U r x . ( 1 7 )

I f w e n o w s u b s t i t u t e t h e e x p r e s s i o n f o r y i n to e q . ( 1 6 ) ,

w e h a v e

R x + R U U T x = b,

R ( I + U U r ) x = b.

N o w , b y T h e o r e m I ,

R ~ R = I .

T h e r e f o r e

( I + U U r ) x = R r b .

T h e m a t r i x ( I + U U T ) i s a s y m m e t r i c p o s i t i v e d e f i n i t e

m a t r i x a n d h e n c e i s n o n s i n g u l a r . T h e r e f o r e

x = ( I + U U r ) - I R ' b .

( 1 8 )

S u b s t i t u t i n g t h i s v a l u e f o r x in t o e q . ( 1 7 ) g i v e s

y = u T ( I + V U r ) - I R ' b .

( 1 9 )

N o w e q s . ( 1 8 ) a n d ( 1 9 ) l e a d t o t h e c o n j e c t u r e t h a t o f

a l l t h e v e c t o r s s s a t i s f y i n g t h e r e s t r i c t i o n

p ( s ) = [ A s - -

b I = O

3 8 2 C o m m u n i c a t i o n s o f t h e A C M V o l u m e 9 / N u m b e r 5 / M a y , 1 9 66



t h e s o f m i n i m a l l e n g t h i s g i v e n b y

( ; ) (

= = u T ( i + u u T ) _ i R ~ b ]

( I + u u r ) - ~ R r ~

= U r ( i + u u T ) _ ~ R , ] b .

F u r t h e r m o r e , s i nc e t h e r e q u i r e d s is g i v e n b y

s = A I b ,

i t c a n b e c o n j e c t u r e d t h a t t h e g e n e r a l i z e d i n v e r s e is g i v e n

b y

A ( UU ) - iR

= (20)

I t i s a s i m p l e m a t t e r t o v e r i f y t h a t t h i s m a t r i x a c t u a l l y

s a t is f ie s P e n r o s e ' s L e m m a s a n d i s a c t u a l l y t h e g e n e r a l i z e d

i n v e r s e o f A . I n f a c t , i f A i s a n y c o m p l e x m a t r i x w i t h a

p a r t i t i o n i n g o f t h e f o r m ( R , S ) t h e n A ~ i s g i v e n b y

A = ( ( i +

v vH ) - 'R'

U ( I + U U ~ I ) - ~ R ' ] ( 2 1 )

T h u s w e h a v e a n e x p r e s s i o n f o r A r i n t e r m s o f R I a n d

U . T h e r e m a i n i n g p r o b l e m i s t o c o m p u t e R r a n d U . F o r

t h i s p u r p o s e , l e t u s b ri e f l y r e v i e w th e G r a m m - S c h m i d t

o r t h o g o n M i z a t i o n p r o c e ss .

I f { a t , a 2 , . . . , a~} i s a n y s e t o f l i n e a r l y i n d e p e n d e n t

v e c t o r s i n m - s p a c e ( m > n ) , t h e n t h i s s e t c a n b e r e p l a c e d

b y a n o r t h o n o r m a l s e t { q t , q 2 , , q~} i n t h e f o ll o w i n g

m a n n e r :

a~

( i ) q t = 1al I

( i i ) c2 = a2 - - (a2 q~)q~

C2

I c l

H H

( i i i ) ca a3 (a3 ql ) ql

- -

- - (a3 q~)q2

C3

q 3 =

C o n t i n u e i n t h i s m a n n e r , a t e a c h s t e p ( i ) f o r m i n g c ~ f r o m

a l a n d t h e p r e v i o u s q ' s a n d t h e n n o r m a l i z i n g c ~ t o l e n g t h

i t o g e t q ~ . A f t e r n s u c h s t e p s t h e r e s u l t i s a s e t q l , q 2 ,

• - . , q~ o f o r t h o n o r m a l v e c t o r s , i . e .,

0,

i ~ j ,

q i q i = 1 , i = j .

I n p a r t i c u l a r , i f t h e v e c t o r s a ~ a r e t h e c o l u m n s o f a n

m N n m a t r i x A , t h e n t h e a b o v e p r o c e s s r e p l a c e s A w i t h

a m a t r i x Q s a ti s f y i n g

Q H Q = I . ( 2 2 )

S i n c e e a c h q~ d e p e n d s o n l y o n a ~ a n d t h e p r e v i o u s q i , t h e

c o l u m n s o f A c a n b e r e p l a c e d o n e c o l u m n a t a t i m e a s il -

l u s t r a t . e d i n t h e f o l l o w i n g s c h e m a t i c d i a g r a m :

A ( a l , a 2 , a 3 , " " , ~ )

(1) . . .

( q l

, a 2 , a a , , a n )

( ' ~ ( q , , q 2 , a3 , , a ~ )

. . . , q , ) .

F u r t h e r m o r e , i n th i s s c h e m e e a c h n e w c o l u m n q i i s o b -

t a i n e d f r o m a l i ne a r c o m b i n a t i o n o f t h e v e c t o r a i a n d t h e

p r e v i o u s q 's . H e n c e t h e c o l u m n s o f A a r e o r t h o g o n a l i z e d

b y a s e r ie s of e l e m e n t a r y c o l u m n o p e r a t i o n s . I f w h e n

c a r r y i n g o u t t h i s p r o c e s s o n t h e c o l u m n s o f A w e b e g i n

w i t h t h e n t h o r d e r i d e n t i t y m a t r i x a n d p e r f o r m t he s a m e

e l e m e n t a r y c o l u m n o p e r a t i o n s o n it , a m a t r i x Z i s o b -

t a i n e d s u c h t h a t

A Z = Q . ( 2 3 )

I f m = n , t h e n t h e m a t r i x A i s n o n s i n g u l a r a n d t h i s p r o c e s s

p r o v i d e s a m e t h o d f o r c o m p u t i n g t h e i n v e r s e o f A . B e -

g i n n i n g w i t h A a n d t h e n t h o r d e r i d e n t i t y , w e a p p l y t h e

G r a m m - S c h m i d t p r o ce s s t o o b t a i n t h e m a t r i c e s Z a n d Q .

T h i s p r o c e s s is i l l u s t r a t e d s c h e m a t i c a l l y b y t h e d i a g r a m :

N o w b y e q. ( 2 2 ) Q UQ = I o r Q -1 = Q H a n d b y e q . ( 2 3 )

Z i s a m a t r i x s a t i s f y i n g A Z = Q . H e n c e

A - 1 = Z Q H . ( 2 4 )

T h u s i f A i s n o n s i n g u la r i t s i n ve r s e c a n b e c o m p u t e d b y

t h e G r a m m - S c h m i d t o r t h o g o n a l i z a t i o n p r o c e s s .

W e n o w e x t e n d t h i s m e t h o d t o c o m p u t e t h e g e n e r al i ze d

i n v e r s e o f a n a r b i t r a r y c o m p l e x m a t r i x A .

I n g e n e r a l , t h e c o l u m n s o f A w i l l n o t b e l i n e a r l y i n d e -

p e n d e n t , a n d t h e G r a m m - S c h m i d t o r t h o go n a l i z a ti o n p ro c -

e s s w i ll n o t w o r k f o r a l i n e a r l y d e p e n d e n t s e t o f v e c t o r s .

I f w e t r y t o a p p l y i t to s u c h a l i n e a r l y d e p e n d e n t s e t i n

w h i c h t h e f i rs t ,~ v e c t o r s a r e l i n e a r l y i n d e p e n d e n t b u t t h e

( k + 1 ) - t h v e c t o r is a li n e a r c o m b i n a t i o n o f t h e p r e v i o u s

k , i t w i l l s u c c e s s f u l l y o r t h o g o n a l i z e t h e f i r s t / c v e c t o r s , b u t

u p o n c a l c u l a t i n g c k + l , w e w i l l f i n d

k

Ck+l ak+l E H O.

- - ( a ~ + l q i ) q l =

i =1

T h u s t h e p r o c e s s b r e a k s d o w n u p o n e n c o u n t e r i n g a

l i n e a r l y d e p e n d e n t v e c t o r . A l t h o u g h t h e c o l u m n s o f A

w i ll in g e n e r a l b e l i n e a rl y d e p e n d e n t , w e h a v e s e e n t h a t i t

c a n j u s t a s w e ll b e a s s u m e d t h a t A h a s a p a r t i t i o n i n g i n

t h e f o r m A = ( R , S ) w i t h a ll t h e l in e a r l y d e p e n d e n t

c o l u m n s l a s t.

T h e r e f o r e , l e t u s c a r r y o u t a m o d i f i e d G r a m m - S e h m i d t

p r o c e s s i n t h e f o l l ow i n g m a n n e r : a p p l y t h e n o r m a l o r t h o -

g o n a l i z a t i o n p r o c e s s to t h e c o l u m n s o f R a n d c o n t i n u e

o v e r t h e c o l u m n s o f S i n t h e s a m e m a n n e r e x c e p t t h a t a s

e a c h v e c t o r b e c o m e s z e r o n o n o r m M i z a t i o n s t e p is p e r -

V o l u m e 9 / N u m b e r 5 / M a y , 1 96 6 C o m m u n i c a t i o n s o f t h e A C M 3 8 3



f o r m e d . I f w e c a r r y o u t t h i s p r o c e s s a n d , b e g i n n i n g w i t h

t h e n t h o r d e r i d e n t i t y m a t r ix , c a r r y o u t e x a c t l y t h e s a m e

e l e m e n t a r y c o l u m n o p e r a t i o n s o n i t , w e h a v e

w h e r e

L o [o I _ k J

_

R, S) I . -k

a n d Q i s a m a t r i x w i t h t h e p r o p e r t y QHQ = I .

N o t e t h a t t h e ( n - - k ) - o r d e r i d e n t i t y m a t r i x i n t h e

l o w e r r i g h t h a n d c o r n e r of t h e b o o k k e e p i n g m a t r i x r e m a i n s

u n c h a n g e d b y t h e p r o c e s s . T h i s i s b e c a u s e a l l t h e c o l u m n s

o f S b e c o m e z e ro w h e n t h e p r o c e s s i s a p p l i e d t o t h e m , t h u s

e s s e n t i a l l y z e r o i n g a n y t e r m s t h a t m i g h t c h a n g e t h e I n - k

w h e n t h e s a m e e l e m e n t a r y c o l u m n o p e r a t i o n s a r e a p p li e d

t o t h e b o o k k e e p i n g m a t r i x .

F r o m e q . ( 2 5 ) i t c a n b e s e e n t h a t

R Z = Q ( 2 6 )

a n d

R X + S = 0 .

R e a r r a n g i n g t h e l a t t e r e q u a t i o n g i v e s R X = - - S , a n d b y

e q . ( 8 ) , X = - - R ~ S . S i n c e b y e q . ( 1 0 ) U = R * S w e h a v e

X = - U . ( 2 7 )

T h u s t h e m a t r i x U c o m e s o u t o f t h e b o o k k e e p i n g m a t r i x ;

i .e. ,

I I

0 I , . -kJ I~-k

A l s o i t i s e a s y t o s e e t h a t R ~ i s g i v e n b y

R ~ = ZQ H. ( 2 8 )

T o v e r i f y th i s o n e n e e d o n l y n o t e t h a t b y e q . ( 2 6 ) R

= Q Z -1, a n d i f t h i s e x p r e s s i o n is u s e d f o r R , t h e n t h e

m a t r i x Z Q H d o e s a c t u a l l y s a t i s f y P e n r o s e ' s L e m m a s a n d

h e n c e m u s t b e R z.

R e c a l l t h a t t h e e x p r e s s io n f o r A ~ w a s b y e q . ( 2 1 ) ,

A = ( ( I + U U n ) - ~ R t ~

\

u (i + uun)-iRV

a n d n o w w e h a v e a m e t h o d f o r o b t a i n in g U a n d R ~. T h e

o n l y r e m a i n i n g p r o b l e m i s t h e e v a l u a t i o n o f t h e e x p r e s s i o n s

( I + U U U ) - i a n d U H ( I + U U U ) - l .

F o r t h i s p u r p o s e , n o t e t h a t t h e f o r m e r t e r m c a n b e r e-

w r i t t e n

( I + u u H ) -~ = I - u ( u H u + I ) - ~ U H

a n d t h e l a t t e r t e r m ,

u ' ( I + U U H ) -~ = ( U H U + I ) - ~ U n .

T h e s e t w o e x p r e s s i o n s a r e e a s i l y v e r i f i e d m a t r i x i d e n t i t i e s

a n d m a k i n g t h e s e s u b s t i t u t i o n s i n t h e e x p r e s s io n fo r t h e

g e n e r a l i z e d i n v e r s e g i v e s

A ~ = ( [ I - - u ( u H u + I )- iU H ] R r ~

( u H u + i ) - ~ U H R , ] . ( 2 9 )

N o w r e ca ] l t h a t u p o n c o m p l e t i o n o f t h e o r t h o g o n a l l z a t i o n

p r o ce s s , th e m a t r i x

a p p e a r e d a s t h e la s t ( n - k ) c o l u m n s o f t h e b o o k k e e p i n g

m a t r i x . O b v i o u s l y t h i s m a t r i x h a s l i n e a r l y i n d e p e n d e n t

c o l u m n s ; s o i t s c o l u m n s c a n b e o r t h o g o n a l i z e d b y t h e

G r a m m - S c h m i d t p r o c e ss . I f w e c a r r y a l o n g a b o o k k e e p i n g

m a t r i x , t h e n

w h e r e

G-S)

\ In--h~

- - U

(

, . 0

C l e a r l y , b y t h e a b o v e r e l a t i o n s h i p

Y = - - U P .

W = P a n d

T h u s t h e r e i s n o n e e d t o c a r r y a l o n g a b o o k k e e p i n g

m a t r i x s in c e t h e m a t r i x W o f t h e r e s u l t c o n t a i n s t h e s a m e

i n f o r m a t i o n t h a t t h e b o o k k e e p i n g m a t r i x w o u l d . S o

w h e r e t h e c o l u m n s o f t h e r e s u l t a r e o r t h o g o n a l ; i . e. ,

v ) = , .

o r

(-V): ,

C a r r y i n g o u t t h e i n d i c a t e d m u l t i p l i c a t i o n s g i v e s

p H U n U P + P H P = I ,

a n d f a c t o r i n g o u t t h e p H a n d t h e P g i v e s

p H ( U H U + I ) P = I.

N o w , P i s a m a t r i x w h i c h c o u l d b e o b t a i n e d f r o m a n

i d e n t i t y m a t r i x b y e l e m e n t a r y c o l u m n o p e r a t i o n s a n d

t h e r e f o r e m u s t b e n o n s in g u l a r . H e n c e

(UH U + I ) = (pH) - - lp - -1 ,

w h e n c e

( u H u + i ) - I = p p , .

A l s o ,

I - - U ( U H U + [ ) - i u n = I - - U P P U n ,

( 3 0 )

3 8 4 C o m m u n i c a t i o n s o f t h e A C M V o h t m e 9 / N u m b e r 5 / M a y , 1 96 6



o r

I - - u ( u H u + I ) - ~ U u = I - - ( U P ) ( U P ) H.

( 3 1 )

T h u s w e c a s t s u b s t i t u t e t h e e x p r e s s i o n s o n t h e r i g h t o f

e q s . ( 3 0 ) a n d (31) i n t o e q. ( 2 9 ) t o o b t a i n

A " = ( [ I - - (U P ) ( U P ) H ] R X ~

p p . U H R ~ ] . (32)

A n d s u b s t i t u t i n g t h e v a l u e f o r R x g i v e n b y e q . ( 2 8 ) a n d

SUBROUTINE G I N V 2

(A,UoAFLAG,ATFMP,MR~NR~.NC)

C

C THIS ROUTINE CALCULATES THF GENERALIZED INVERSE OF A

C A N D S T O R E S T H E T R A N S P O S E O F I T I N A .

C MR#FIRST D I M E N S I O N N n o OF A.

C NR N O . O F P O I ; S I N A

C N C N O , C ~F C O L U M N £

I t l

A

C U I S THE BOOKKEEPING ~ATPIX .

C AFLAG AND AT¢~P ARF T~MP"~RARY WORKING STORAGe.

C

DI'qENSION A(MR,NC) ,U(Nc,,NC) ,AFLAG(NC) ,ATC'MP(r, C)

DO IO I

# I

, N C

OO 5 J I ,NC

5 U(I,J) # D . q

I0 U(I , I }#1.O

F A C D O T ( M P , N R + A ~ I . I }

FAC# I . O/SORT ( FAC )

DO

I 5 I I

,NR

15

A(I , I I#AI I , I i+~AC

DO 2 0 I I , N C

2 0 U(I , I }#U( I , I )~cAC

AFLAG( I }# I. ~

C

C D E P E N D E N T C O L T O L E R A N C E F O R N B I T F L O A T I N G P O I N T F R A C T I O N

C

N 2 T

T O L ( 1 0 . * O.~**N)**~

DO ION J 2,pNC

DOTI # DOTIMR,NR,A ,J~J)

J M I J - I

DO 5n L I ~7

DO 9 r~ K I , J M I

3Q ATEMP(K} DOT(r~R,NR+A,, I,K)

DO 45 K#I ,JMI

DO 35

I I , N P

3 5 A( I ,J) A( I J ) -ATEMP(K)*~( I K)*AFLAG(K)

DO 40 I 1 , N C

zoo U( I ,J )#U( I ,J I- ATE MP( KI *U ( ~K )

a5 CONTINUE

53 CONTINUE

DOT2 # DOT(MR,NR,A ,J~J)

IF( (DOT2/D~TI ) - TOL) m 5 . 5 5 ~ 7 0

5 5 DO 6 0 I I , J M I

ATEMP ( I } 3. q

DO 6 0 K I ,I

6 0 A T E M P ( I } A T E M P ( I ) + U ( K , I ) * U ( K , J }

DO 65 I I , N R

A { I . J ) O ° F ]

DO 65 K I,JM[

65 A( I

, J I A (

I

, J I -A (

I ,KI*ATEMP(K)*AFLAG(K)

AFLAG(J ) #I].[~

FAC DOT(NC,NC,U,J,J)

FAC I . D/SORT ( FAC }

GO T O 75

7 r ] AFL AGfJ) I .n

FAC 1 .O/SORT(DOT2)

7 5 D O 8 0 I

#

I',NR

80 A( I,J)# A( I,J) *FAC

DO 8 5 I I NC

8 5 U ( I ~ J ) U ( I , J ) * F A C

1 0 O C O N T I N U F

DO 130 J l + N C

D O 1 3 0 I l , N R

F A C n . q

DO 12Q K J,NC

120 FAC FAC+A( I ,K} *U{ J,K 1

1'3D AiI*J) FAC

RETURN

FNO

F U N C T I O N DmTIMP~NP+A,JC~KC)

COMPUTES THE I NNPP RPODUC T OF COLUMNS JC AND KC

O F

MATPlX A.

DIMENSION A ( M R ~ I )

DOT#P.n

D O 5 I I , N R

DOT DOT + A(I ,JC)*A(I ,KC}

R F T U P N

E N D

F | G . 1 .

r e a r r a n g i n g t h e b o t t o m s u b m a t r i x g i v e s

A ' = ( [ I - ( U P ) ( u p ) H ] z Q H ~

p ( u p )H Z Q n ] . ( 3 3 )

W e n o w h a v e a s i m p le s c h e m e f o r c o m p u t i n g t h e g e n -

era l i z ed inverse .

B e g i n n i n g w i t h t h e m a t r ix ( R , S ) a n d a n i d e n t i t y

m a t r i x , w e c a n i l l u s tr a t e t h e s c h e m e a s fo l l ow s :

r , l r 0 i 0 ,

- s - i v - u P .

[ 0 I , _ ~ J [ 0 I j p j

W e w o u l d t h e n h a v e a l l t h e i n f o r m a t i o n n e c e s s a r y t o

c o m p u t e t h e g e n e r a l i z e d in v e r s e o f A f r o m e q . ( 3 3 ) .

T h u s w e h a v e a s im p l e e x t e n s io n o f t h e G r a m m - S c h m i d t

m e t h o d f o r c o m p u t i n g t h e g e n e r a ] i z e d i n v e r s e .

I n c a r r y i n g o u t t h i s a l g o r i t h m o n a c o m p u t e r , a l l t h e

c a l c u l a t i o n s c o u l d b e p e r f o r m e d i n t h e s p a c e o f t h e m a t r i x

A i t s e l f p lus t he spac e req u ired f or an n X n b ook k eep in g

m a t r i x . I t i s c l e a r t h a t a l l t h e o r t h o g o n a l i z a t i o n r e q u i r e d

t o r e d u c e t h e s e m a t r i c e s t o t h e f o r m

p J

c a n b e d o n e i n t h i s s p a c e. W e c o u l d t h e n f o r m t h e p r o d u c t

( U P ) ~ Z i n t h e s p a c e o f t h e z e r o s u b m a t r i x i n t h e l o w e r

l e f t h a n d c o r n e r o f t h e b o o k k e e p i n g m a t r i x t o g e t

I Q 0 ]

Z - - U P

[ ( U P ) " Z P J

W e c o u l d t h e n f o r m t h e p r o d u c t [ ( U P ) I ' Z Q H ]

H

n t h e O -

s u b m a t r i x o f ( Q , O ) a n d t h e n r e s to r e t h e O - s u b m a t r i x i n

t h e l o w e r e f t h a n d p a r t o f t h e b o o k k e e p i n g m a t r i x t o g e t

[ ( U P ) ~ ZQH IH ]

- U P I

I

P )

W e t h e n w o u l d o n l y n e e d t o p e r f o r m t h e p r o d u c t

O H ( Z Q u - ( U P ) ( u p ) H Z Q ' ~

= A z

T h e t r a n s p o s e o f t h i s p r o d u c t c a n b e fo r m e d i n t h e s p a c e

o r i g i n a l l y o c c u p i e d b y

( Q , ( u p ) H Z Q ~ ) .

T h u s t h e n e t

r e s u lt o f c a r r y i n g o u t t h e a l g o r i t h m i s t o r e p la c e t h e m a t r i x

A b y t h e t r a n s p o s e o f A ' .

A F O R T R A N s u b r o u t i n e f o r c a r r y i n g o u t t h e a l g o r i t h m

i s g i v e n i n F i g u r e 1 . T h e p r o g r a m d o e s n o t c a r r y o u t t h e

a l g o r i t h m e x p l i c i t l y i n t h a t i t a v o i d s p e r m u t i n g t h e

c o l u m n s t o o b t a i n t h e f o r m ( R , S ) , a n d a s e a c h l in e a r l y

d e p e n d e n t c o l u m n b e c o m e s z e r o i n t h e o r t h o g o n a l i z a t i o n

p r o c e s s , i t i s i m m e d i a t e l y r e p l a c e d b y a c o r r e c t e d c o l u m m

T h e n e t r e s u lt , h o w e v e r , i s t h e s a m e a s w o u l d b e o b t a i n e d

( C o n t i n u e d on p a g e 8 8 7)

V o l u m e 9 / N u m b e r 5 / M a y , 1 96 6 C o m m u n i c a t i o n s o f t h e A C M 3 8 5



Option C

The longitudinal center line of the feed holes shall be located

within 0.300 inch maximum from the two-track edge

and

0.395 inch maxi-

mum from the three-track edge of the tape. The distance from this center

line to either edge shall not vary by more than .006 inch (total variation)

within any 6 inch length of tape.

To help clarify the above options, a sketch is submitted indicating the

tolerance from the feed hole to the guided edge (Figure 2).

5. A purpose of option C is to prevent interference between the tape

and the tape guide in readers containing both a feed wheel and a nar row

tape guide. However, preferred practice in the design of readers with feed

wheels is to make the tape guide wide enough to assure locating the tape

by the feed wheel only, at the sensi ng pins. The pu rpos e of the guide, then,

is to facilitate insertion of the tape into the reader and to preve nt exces-

sive skew.

G U I D E . ' ] ) E _ D G ~ _ .

• S -

I o

o

- - ~ - - / / / / / / / / ~ / / / / / /

O P T I O N ' ~ , O P T I O ix/ B PTt O N C

C O M t v lU N I C A T I O N S O F F I C E _ . P R O P O S E D

M A C H I N E S A L T E R N A T E

FIG. 2

Pertinent factors relating to the three options are as follows:

Option A

1. This type of dimensioning of paper tape punches has been standard

in the communications industry for the past 75 years. To change the

guiding of these machines would be economically impractical.

2. This type of dfinensioning permi ts variat ion in feed hole location of

0.006 inch from the two hole edge and 0012 inch from the three hole edge

of the tape.

3. Tapes punched according to this standard are sensed equally well by

communication and business machine readers which guide on the sprocket

holes only.

4. Tapes punched according to this standard are sensed well by readers

which guide only on the two hole edge of the tape. There may be some

loss of sensing margin (due to I tem 2 above) when such tapes are passed

through a reader which guides only in the three hole edge.

5. This method of dimensioning differs from that used in EIA RS-227

for one inch paper tape but conforms to that used in thousands of

domestic and foreign machines in use and manufacture today.

Option B

1. This type of dimensioning of paper tape punches has been stand ard

in the business machine industry for two decades. To change the guiding

edge of these machines would be economically impracticM.

2. This type of dimensioning permits a varia tion in feed hole location of

0.012 inch fro m the two hole edge and 0.006 inch from the three hole edge

of the tape.


communications and business machine readers which guide on the

sprocket holes only.

4. Tapes punched according to this standard are sensed well by readers

which guide only on the three hole edge of the tape. There may be some

loss of sensing margin (due to Item 2 above) when such tapes are passed

through a reader which guides only on the two hole edge.

5. This meth od of dimensionin g is the same as used in EIA RS-227 for

one inch paper tape and would permit a reader which guides on only the

three hole edge of the tape to read both 1~ 6 inch and 1 inch tape with

equal margins.

Option C

1. This type of dimensioning offers a compromise between Option A and

B. It recognizes the pres ent and continuing existence of tape perforators

producing tape in accordance with both Option A and Option B conven-

tions.

2. This type of dimensioning permits a variation in feed hole location of

up to 0.012 inch from either the two hole or the three hole edge of the

tape.


communications and business machine readers which guide on the

sprocket holes only.

4. This method of dimensioning requires that readers which guide only

on one edge of the tape be designed to accommodate tapes guided on

either edge during preparation. The number of readers which guide only

on one edge is small and the design proble ms encountered in such a reader

to allow for the possible maxi mum 0.012 inch variation ( Ite m 2 above) arc

considered minimal.

R U S T e t a l .m c o n t d f ro m p a g e 3 8 5

i n c a r r y i n g o u t t h e a l g o r i t h m i n t h e m a n n e r d e s c r i b e d

a b o v e . I n t h e i n t e r e s t o f a c c u r a c y th e p r o g r a m r e o r t h o -

g o n a ] i z e s e a c h c o l u m n a f t e r i t i s f i r s ~ o r t h o g o n a l i z e d . T h i s

i s a s t a n d a r d t e c h n i q u e i n c a r r y i n g o u t t h e G r a m m -

S c h m i d t o r t h o g o n a l i z a t i o n .

A n u m b e r o f p u b l ic a t i o n s h a v e a p p e a r e d i n t h e p a s t

f e w y e a r s w h i c h a r e a ls o c o n c e r n e d w i t h m e t h o d s f o r

c o m p u t i n g t h e g e n e r a l i z e d i n v e r s e . P y l e [ 2 ] d i s c u s s e s a

m e t h o d f o r f i n d i n g t h e g e n e r a l i z e d i n v e r s e o f a n a r b i t r a r y

m X n c o m p l e x m a t r i x A w i t h m _ < n i n w h i c h t h e G r a m m -

S c h m i d t p r o c e s s is a p p l i e d t o t h e c o l u m n s o f A g a n d t h e n

t o t h e c o l u m n s o f A i f r a n k ( A ) _< m . B e n I s r a e l a n d

W e r s a n [ 3] d e s c r ib e d i m i n a t i o n m e t h o d s i n w h i c h th e

e l i m i n a t i o n p r o c e s s i s a p p l i e d t o t h e s y m m e t r i c p r o d u c t

AHA o r th e s y m m e t r i c p r o d u c t o f s o m e s u b m a t r i x o f A .

I t i s im p o r t a n t t o n o t e t h a t a ll t h e s e m e t h o d s , i n c l u d i n g

t h a t o f t h e a u t h o r s , d e p e n d u p o n t h e c o r r e c t d e t e r m i n a -

t i o n o f t h e r a n k o f t h e m a t r i x . I n [ 4] G o l u b d i s c u s s e s t h e

s t r a t e g y o f u s i n g t h e g e n e r a l i z e d i n v e r s e t o s o l v e l e a s t

s q u a r e s p r o b l e m s w h e n t h e m a t r i x i s d e fi c i en t i n r a n k o r

p o o r l y c o n d i t i o n e d .

RECEIVED JANUARY, 1966

R E F E l t E N C E S

1. PENROSE, Z. A generalized inverse for matrices.

Proc. Cam-

bridge Philos. Soe. 51

(1954).

2. PYLE, L. D. Generalized inverse computations using the gra-

dient project ion method.

J.ACM 11

(1964).

3. -- . , AND WERSAN, S.J. An elimi natio n met hod for comp ut-

ing the general ized inverse of an arbi t rary complex matr ix .

J.ACM 10

(1963).

4. GOhUB, G. Numerical methods for solving linear least squares

problems.

Numer. Math. 7

(1965).

5. BEN-ISRAEn, A., AND CHARNES A. Contr ibut ion to the theory

of generalized inverses.

J.SIAM 11

(1963).

6. GREVILLE, W. N. E. The pseudoinverse of a rectangular or

s ingular matr ix and i t s appl icat ion to the solut ion of sys tems

of l inear equat ions .

SIAM Rev. 1

(1959).

7 . - - . Some appl icat ions of the pseudoinverse of a nmtr ix .

SIAM Rev. 2

(1960).

P r o p o s e d A m e r i c a n S t a n d a r d : T w e l v e - R o w P u n c h e d - C a r d

C o d e f o r I n f o r m a t i o n I n t e r c h a n g e

(X3.2/303, dated January

14, 1966) will ap pe ar in t he J une , 1966 issu e of the

Communications

of the ACM.

Advance orders for repr ints , in uni ts of 50, may be placed wi th

Waver ly P ress , Mt . Roya l & Gui l ford Ayes ., Bal t imore, Maryl and

21202 up to May 20, 1966. Autho rize d in stitu tion al pur chas e or-

der should give name of document , numbe r of uni ts of 50, name

and address of person to whom the invoice i s to be addressed,

shipping address .

Volu me 9 / Numb er 5 / May, 1966 Comm uni cat ion s of

t h e A C M

387

Gen inversion

Documents