The topic: Least squares method Numeriska beräkningar i Naturvetenskap och Teknik.

The topic:

Least squares method

Numeriska beräkningar i Naturvetenskap och Teknik

An exemple

80

70

60

50

40

30

20

T

83.54

48.53

76.51

71.50

60.50

19.50

56.48

)(TfL

T

L

Model

TccTfL 10)(

20 40 60 80

48

50

52

54

Why do the measured values deviate from the model if the measurement is correct?


How determine the ‘best’ straight line?

T

L

Model bTaTfL )(

20 40 60 80

48

50

52

54


Distance between line and measurements points...

T

L

20 40 60 80

48

50

52

54

1d

2d3d

4d5d

6d

7d


How to define the distance between the line and the measurement points?

id Largest deviation at minimum Approximation in maximum norm

2id

Sum of deviations squared as small as possible Approximation in Euclidian norm

Easier to calculate!

Norm


Matrix formulation: An example

3

2

0

1

3

x

9.0

6.0

6.0

9.0

1.2

)(

xf

x

f

xccxf 10)(

9.03

6.02

6.0

9.0

1.23

10

10

0

10

10

cc

cc

c

cc

cc

9.0

6.0

6.0

9.0

1.2

1

1

1

1

1

3

2

0

1

3

1

0

c

c

fAc with

More equations than unknowns!



9.0

6.0

6.0

9.0

1.2

1

1

1

1

1

3

2

0

1

3

1

0

c

c

fAAcA TT

3

1

1

1

0

1

2

1

3

1

1

1

1

1

1

3

2

0

1

3

1

0

c

c

3

1

1

1

0

1

2

1

3

1

9.0

6.0

6.0

9.0

1.2



fAAcA TT

23 1

1 5

1

0

c

c

11.1

1.2

1

0

c

c

0.505

521.0

xxf 505.0521.0)(



xxf 505.0521.0)(


General Statement of the Problem:

Depending on the model, the measurement data can of course be described by other expressions than the straight line.

In general terms one seeks a function f* that approximates f’s given values as well as possible in euclidian norm.

Specifically, above we looked for a solution expressed as

xccxf 10)(

but we could as well have looked for a solution given by another function (possibly then for different data)

)cos()sin()( 10 xcxcxf

etc...


Generally one can thus write:

nncccxf ...)( 1100

f(x) is in other words a linear combination of given functions

n ,...,, 10 Where the coefficients

nccc ,...,, 10 are sought

One can in accordande with a vector space look at it so that

n ,...,, 10

Spans a function space (a space of this kind which fulfills certain conditions is called a Hilbert space, cmp. quant. mech)


In the case of the straight line we have

x

1

0 1

In a geometrical comparision these two functions, which can be seen as two vectors in the function space, span a plane U:

10

x1

*f

”vector” 0

”vector” 1

Approximating function

f sought function

The smallest distance from the plane is given by a normal. The Smallest deviation between f* och f is for f*-f orthogonal to the plane U!


Normal equations

Since we are interested in fitting m measured values weleave the picture of the continuous function space and view f(x) as an m-dimensional vector with values:

)(

...

)(

)(

2

1

mxf

xf

xf

)(0

)2(0

)1(0

...m

)(1

)2(1

)1(1

...m

That should be expressed by

and0 1f

For the straight line:

1

...

1

1

0

mx

x

x

...1

1

1


The orthogonality condition now gives the equations:

0),*(

0),*(

1

0

ff

ff1100* ccf where

0),(

0),(

11100

01100

fcc

fcc

€

c0(ϕ 0,ϕ 0) + c1(1ϕ1,ϕ 0) − ( f ,ϕ 0) = 0

c0(ϕ 0,ϕ1) + c1(ϕ1,ϕ1) − ( f ,ϕ1) = 0

€

c0(ϕ 0,ϕ 0) + c1(1ϕ1,ϕ 0) = ( f ,ϕ 0)

c0(ϕ 0,ϕ1) + c1(ϕ1,ϕ1) = ( f ,ϕ1)

the equations for the normal:

Which gives


),(),(),(

),(),(),(

1111100

0011000

fcc

fcc

The equations for the normal :

),( ),(

),( ),(

1110

0100

1

0

c

c

),(

),(

1

0

f

f

1

0

. .

. .

. .

. .

. .

. .

10 1

0

c

c

1

0

.

.

.

.

.

.

f


Back to the exemple:

3

2

0

1

3

x

9.0

6.0

6.0

9.0

1.2

)(

xf

xccxf 10)(

1

1

1

1

10

3

2

0

1

3

)(1

x

Model:

3

2

0

1

3

3 2 0 1- 3 1 1 1 1 1

1

1

1

1

1

1

0

c

c

9.0

6.0

6.0

9.0

1.2

1 1 1 1 1 3 2 0 1- 3

TA A TAc f

Data:


Conclusion:

2||*|| ff

the minimum of

ff * is orthogonal to the basis vectors

n ...,, 3,210

Assuming the model

nncccccf ...* 33221100

Given data f

is obtained when

The coefficienterna c1, c2, c3, cn are determined from


The equations

),(),(...),(),(

...

),( ),(...),(),(

),(),(...),(),(

1100

11111100

00011000

nnnnnn

nn

nn

fccc

fccc

fccc

or

fAAcA TT

Where the colomuns in A are: n ...,, 3,210


Note 1:

The func’s Have to be linearly independent

n ...,, 3,210

(cmp vectors in a vector space)

Note 2:

993

994

996

997

999

x

9.0

6.0

6.0

9.0

1.2

)(

xf

Assume our problem would have been (x koord -996)

4958111 4979

4979- 5

1

0

c

c

2102.7

1.2

23 1

1 5

1

0

c

c

11.1

1.2

cmp to


The topic: Least squares method Numeriska beräkningar i Naturvetenskap och Teknik.

Documents

model numeriska berkningar

example numeriska berkningar

norm numeriska berkningar

mech numeriska berkningar

function f

continuous function

normal equations

vector space note