Announcements Motivation Robo Warm-up Parametric Regression Least Squares Cholesky Factorization Sparsity Special Designing and Analyzing Linear Systems CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Doug James (and Justin Solomon) CS 205A: Mathematical Methods Designing and Analyzing Linear Systems 1 / 35
38
Embed
Designing and Analyzing Linear Systems - graphics.stanford.edugraphics.stanford.edu/.../designing_and_analyzing_systems.pdf · Designing and Analyzing Linear Systems CS 205A: Mathematical
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Announcements Motivation Robo Warm-up Parametric Regression Least Squares Cholesky Factorization Sparsity Special Structure
Designing and AnalyzingLinear Systems
CS 205A:Mathematical Methods for Robotics, Vision, and Graphics
Doug James (and Justin Solomon)
CS 205A: Mathematical Methods Designing and Analyzing Linear Systems 1 / 35
Announcements Motivation Robo Warm-up Parametric Regression Least Squares Cholesky Factorization Sparsity Special Structure
Announcements
I Section and office hours: Finalized. Posted onwebsite.
I HW0 due, HW1 out
I Check midterm/final dates. Conflicts?
CS 205A: Mathematical Methods Designing and Analyzing Linear Systems 2 / 35
Announcements Motivation Robo Warm-up Parametric Regression Least Squares Cholesky Factorization Sparsity Special Structure
Theorist’s Dilemma
“Find a nail for this really interesting hammer.”
A~x = ~b
CS 205A: Mathematical Methods Designing and Analyzing Linear Systems 3 / 35
Announcements Motivation Robo Warm-up Parametric Regression Least Squares Cholesky Factorization Sparsity Special Structure
Today’s Lesson
Linear systems areinsanely important.
CS 205A: Mathematical Methods Designing and Analyzing Linear Systems 4 / 35
Announcements Motivation Robo Warm-up Parametric Regression Least Squares Cholesky Factorization Sparsity Special Structure
Planar Serial Chain Manipulator
q1
q2
q3
...qn
p
Problem: How to change redundant joint angles tomove toward target?
Joint angles: ~q = (q1, q2, . . . , qn)T
End-effector position: ~p =(xy
)Kinematic model: ~p = ~f(~q)Linearized model: ∆~p = J ∆~qMinimum-norm solution for ∆~q given ∆~p.
CS 205A: Mathematical Methods Designing and Analyzing Linear Systems 5 / 35
Announcements Motivation Robo Warm-up Parametric Regression Least Squares Cholesky Factorization Sparsity Special Structure
Linear Regression
f (~x) = a1x1 + a2x2 + · · · + anxn = ~aT~x
Find {a1, . . . , an}.
CS 205A: Mathematical Methods Designing and Analyzing Linear Systems 6 / 35
Announcements Motivation Robo Warm-up Parametric Regression Least Squares Cholesky Factorization Sparsity Special Structure
n Experiments
~x(k) 7→ y(k) ≡ f(~x(k))
y(1) = f(~x(1)) = a1x(1)1 + a2x
(1)2 + · · ·+ anx
(1)n
y(2) = f(~x(2)) = a1x(2)1 + a2x
(2)2 + · · ·+ anx
(2)n
...
CS 205A: Mathematical Methods Designing and Analyzing Linear Systems 7 / 35
Announcements Motivation Robo Warm-up Parametric Regression Least Squares Cholesky Factorization Sparsity Special Structure
Linear System for ~a
− ~x(1)> −− ~x(2)> −
...
− ~x(n)> −
a1
a2...
an
=
y(1)
y(2)
...
y(n)
CS 205A: Mathematical Methods Designing and Analyzing Linear Systems 8 / 35
Announcements Motivation Robo Warm-up Parametric Regression Least Squares Cholesky Factorization Sparsity Special Structure
General Case
f(~x) = a1f1(~x) + a2f2(~x) + · · ·+ amfm(~x)
f1(~x
(1)) f2(~x(1)) · · · fm(~x(1))
f1(~x(2)) f2(~x
(2)) · · · fm(~x(2))...
... · · · ...f1(~x
(m)) f2(~x(m)) · · · fm(~x(m))
a1a2...am
=
y(1)
y(2)
...y(m)
f can be nonlinear!
CS 205A: Mathematical Methods Designing and Analyzing Linear Systems 9 / 35
Announcements Motivation Robo Warm-up Parametric Regression Least Squares Cholesky Factorization Sparsity Special Structure
Two Important Cases
f (x) = a0 + a1x + a2x2 + · · · + anx
n
“Vandermonde system”
f (x) = a cos(x + φ)
Mini-Fourier
CS 205A: Mathematical Methods Designing and Analyzing Linear Systems 10 / 35
Announcements Motivation Robo Warm-up Parametric Regression Least Squares Cholesky Factorization Sparsity Special Structure
Something Fishy
Why should you have to doexactly n experiments?
What if y(k) is measuredwith error?
CS 205A: Mathematical Methods Designing and Analyzing Linear Systems 11 / 35
Announcements Motivation Robo Warm-up Parametric Regression Least Squares Cholesky Factorization Sparsity Special Structure
Overfitting
OverfittingFinding patterns in statistical noise
CS 205A: Mathematical Methods Designing and Analyzing Linear Systems 12 / 35
Announcements Motivation Robo Warm-up Parametric Regression Least Squares Cholesky Factorization Sparsity Special Structure
Interpretation of Linear Systems
b1b2...bn
=
− ~r>1 −− ~r>2 −... · · · ...− ~r>n −
x1x2...xn
=
~r1 · ~x~r2 · ~x
...~rn · ~x
“Guess ~x by observing its dot products with ~ri’s.”
CS 205A: Mathematical Methods Designing and Analyzing Linear Systems 13 / 35
Announcements Motivation Robo Warm-up Parametric Regression Least Squares Cholesky Factorization Sparsity Special Structure
What happens when m > n?
Rows are likely to be incompatible.
Next best thing:
A~x ≈ ~b
CS 205A: Mathematical Methods Designing and Analyzing Linear Systems 14 / 35
Announcements Motivation Robo Warm-up Parametric Regression Least Squares Cholesky Factorization Sparsity Special Structure
Least Squares
A~x ≈ ~b⇐⇒ min~x‖A~x−~b‖2
⇐⇒ A>A~x = A>~b
CS 205A: Mathematical Methods Designing and Analyzing Linear Systems 15 / 35
Announcements Motivation Robo Warm-up Parametric Regression Least Squares Cholesky Factorization Sparsity Special Structure
Least Squares
A~x ≈ ~b⇐⇒ min~x‖A~x−~b‖2
⇐⇒ A>A~x = A>~b
CS 205A: Mathematical Methods Designing and Analyzing Linear Systems 15 / 35
Announcements Motivation Robo Warm-up Parametric Regression Least Squares Cholesky Factorization Sparsity Special Structure
Normal Equations
A>A~x = A>~bA>A is the Gram matrix.
CS 205A: Mathematical Methods Designing and Analyzing Linear Systems 16 / 35
Announcements Motivation Robo Warm-up Parametric Regression Least Squares Cholesky Factorization Sparsity Special Structure