Optimization/Learning on the GPU (supplement figure slides) CIS 665 Joe Kider.

Optimization/Learning on the GPU (supplement figure slides)

CIS 665

Joe Kider

Pictures/Slides thanks to…• Jonathan Shewchuk• Nico Galoppo• Jeff Bolz

• (Most of this was a blackboard lecture, these slides supplement that, since drawing the graphs of quadratic forms can be difficult. For the most part the lecture came from the following 3 sources: – Jonathan Richard Shewchuk, An Introduction to the Conjugate Gradient

Method Without the Agonizing Pain– Nico Galoppo et Al., LU-GPU: Efficient Algorithms for Solving Dense

Linear Systems on Graphics Hardware– Bolz et Al., Sparse Matrix Solvers on the GPU: Conjugate Gradients

and Multigrid

Gauss-Jordon

Graph of a quadratic form f(x)

The minimum point of this surface is the solution to Ax=b

Contours of the quadratic form

Gradient f’(x) of the quadric form

Gradient Descent

Problem graphs

Method of Orthogonal Directions

Conjugate Directions

Gram-Schmidt Conjugation

Conjugate Directions

Conjugate directions using theAxial unit vectors, also knowAs Gaussian Elimination

Conjugate Gradients

Conjugate Gradients

Conjugate Gradients

Conjugate Gradients on the GPU

Conjugate Gradients on the GPU

Example Applications

• Just a few uses:

– GPU sim demo– Heart wave demo– Flesh Simulation– Water Simulation