Regularization of the Inverse Laplace Transform with Applications in Nuclear Magnetic Resonance Relaxometry Candidacy Exam Christiana Sabett Applied Mathematics, Applied Statistics, & Scientific Computation University of Maryland, College Park [email protected]Advisors: John J. Benedetto, Alfredo Nava-Tudela Mathematics, IPST Mentor: Richard Spencer Laboratory of Clinical Investigations, NIA December 6, 2016 Christiana Sabett ILT Regularization in NMR Relaxometry
40
Embed
Regularization of the Inverse Laplace Transform with ... · Regularization of the Inverse Laplace Transform with Applications in Nuclear Magnetic Resonance Relaxometry Candidacy Exam
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Regularization of the Inverse LaplaceTransform with Applications in Nuclear
Magnetic Resonance RelaxometryCandidacy Exam
Christiana SabettApplied Mathematics, Applied Statistics, & Scientific
December 6, 2016Christiana Sabett ILT Regularization in NMR Relaxometry
Outline
Nuclear Magnetic Resonance (NMR) RelaxometryBackgroundObjectiveMotivation from Celik
1D Discrete Model2D Model ExtensionOrdinary Least SquaresRegularization
Regularized Least SquaresMethods to Choose the Regularization Parameter
Hansen’s L-CurveL-Curve as an Analytical ToolFINDCORNER Algorithm
Problem Extensions
Christiana Sabett ILT Regularization in NMR Relaxometry
Nuclear Magnetic Resonance (NMR) Relaxometry
Figure: Clockwise from top left: a. Local magnetization M emergesfrom alignment with magnetic field B0. b. With an RF pulse, M alignswith the magnetic field B1 in the transversal plane. c. After the pulse,M begins to realign with B0. d. Components Mlon(t) and Mtr (t),characterized by decay rates T1 and T2, respectively, describe M(t) attime t . Images courtesy of Alfredo Nava-Tudela.
Christiana Sabett ILT Regularization in NMR Relaxometry
Objective
A 1-dimensional continuous NMR relaxometry signal takes theform
y(t) =∫ ∞
0f (T2)e−t/T2 dT2 + n(t) (1)
where T2 is the transversal decay rate, f (T2) corresponds to theamplitude of the associated component, and n(t) is additivenoise.
Objective:Recover the distribution of amplitudes f (T2) present in thesignal via an inverse Laplace transform (ILT).
Christiana Sabett ILT Regularization in NMR Relaxometry
Toy Example
Consider a signal
y(t) = 0.6e−t/T2,1 + 0.4e−t/T2,2 + n(t) (2)
where the exact distribution f (T2) is
f (T2) = 0.6 δT2,1(T2) + 0.4 δT2,2(T2) (3)
The recovery of f (T2) is unstable due to the sensitivity of theinversion to noise.
Christiana Sabett ILT Regularization in NMR Relaxometry
Motivation
Celik et al [1] demonstrated stabilization of the ILT through theintroduction of a second, indirect dimension.
Figure: The experimental process applied by Celik [1]. The 2D ILTpath illustrated by the solid arrows produced better resolution ofpeaks in a sparse signal than the 1D ILT path (dashed arrow).
Christiana Sabett ILT Regularization in NMR Relaxometry
Motivation
Figure: The experimental process applied by Celik [1]. Inversionsfrom 12 different noise realizations are shown, demonstrating thestability of the 2D ILT. Top: 1D ILT. Bottom: 2D ILT projection.
Christiana Sabett ILT Regularization in NMR Relaxometry
Discrete Model
A 1D NMR signal takes discrete form
z(ti) =K∑
j=1
F (T2,j)e−ti/T2,j (4)
In matrix form:z = AF (5)
where
[A]ij = e−ti/T2,j (6)zi = z(ti) (7)
Christiana Sabett ILT Regularization in NMR Relaxometry
Discrete Model
A 1D NMR signal takes discrete form
z(ti) =K∑
j=1
F (T2,j)e−ti/T2,j (4)
In matrix form:z = AF (5)
where
[A]ij = e−ti/T2,j (6)zi = z(ti) (7)
Christiana Sabett ILT Regularization in NMR Relaxometry
2D NMR Model
The 2D continuous NMR relaxometry signal takes the form
z (t , t) =∫ ∞
0
∫ ∞0
F (T1,T2)e−t/T1e−t/T2dT1dT2 (8)
In discrete form,
z (ti , tj) =K1∑
k=1
K2∑m=1
F (T1,k ,T2,m)e−ti/T1,k e−tj/T2,m (9)
Christiana Sabett ILT Regularization in NMR Relaxometry
2D NMR Model
Define A1,A2, F , and Z such that
[A1]ik = e−ti/T1,k
[A2]jm = e−tj/T2,m
[F ]km = F (T1,k ,T2,m)
[Z ]ij = z (ti , tj)(10)
ThenZ = A1FAT
2 (11)
(M1 × M2) = (M1 × K1)(K1 × K2)(K2 × M2)
which becomes
vec(Z ) = (A2 ⊗ A1)vec(F ) (12)
Christiana Sabett ILT Regularization in NMR Relaxometry
2D NMR Model
Define A1,A2, F , and Z such that
[A1]ik = e−ti/T1,k
[A2]jm = e−tj/T2,m
[F ]km = F (T1,k ,T2,m)
[Z ]ij = z (ti , tj)(10)
ThenZ = A1FAT
2 (11)
(M1 × M2) = (M1 × K1)(K1 × K2)(K2 × M2)
which becomes
vec(Z ) = (A2 ⊗ A1)vec(F ) (12)
Christiana Sabett ILT Regularization in NMR Relaxometry
2D NMR Model
Define A1,A2, F , and Z such that
[A1]ik = e−ti/T1,k
[A2]jm = e−tj/T2,m
[F ]km = F (T1,k ,T2,m)
[Z ]ij = z (ti , tj)(10)
ThenZ = A1FAT
2 (11)
(M1 × M2) = (M1 × K1)(K1 × K2)(K2 × M2)
which becomes
vec(Z ) = (A2 ⊗ A1)vec(F ) (12)
Christiana Sabett ILT Regularization in NMR Relaxometry
Ordinary Least Squares
With 1D ordinary least squares (OLS), we solve
minF∈RK
||AF − y ||22 (13)
Christiana Sabett ILT Regularization in NMR Relaxometry
Ordinary Least Squares
Define the singular value decomposition of A as
A =L∑
i=1
σiuivTi (14)
where σi are the singular values and ui , vi are the left and rightsingular vectors, respectively. Then the OLS solution is
FOLS =N∑
i=1
uTi y vi
σi. (15)
Christiana Sabett ILT Regularization in NMR Relaxometry
Regularization
To increase stability in the inversion, we add a penalty termtuned by the parameter α.
The most common form is Tikhonov regularization, aka ridgeregression.
minF∈RK
||AF − y ||22+α2||F ||22 (16)
Christiana Sabett ILT Regularization in NMR Relaxometry
Regularization
Other common penalty forms include:
Lp regularization, p ≥ 1
minF∈RK
||AF − y ||22+α||F ||p (16)
L1 regularization is known as LASSO. Lp regularization for1 < p < 2 is called bridge regression.
Christiana Sabett ILT Regularization in NMR Relaxometry
Regularization
Other common penalty forms include:
differential operator, L
minF∈RK
||AF − y ||22+α||LF ||22 (16)
Christiana Sabett ILT Regularization in NMR Relaxometry
Regularization
We will discuss Tikhonov regularization.
minF∈RK
||AF − y ||22 + α2||F ||22 (16)
Christiana Sabett ILT Regularization in NMR Relaxometry
Regularized Least Squares
The regularized problem can be expressed as a normal leastsquares problem
minF∈RK
||AF − y ||22 (17)
where
A =
[AαIK
]y =
[y
0K
]
Christiana Sabett ILT Regularization in NMR Relaxometry
Regularized Least Squares
For an appropriate choice of α,
FTikh =N∑
i=1
fiuT
i y vi
σi(18)
where, in the case of Tikhonov regularization, the filter factors fitake the form
fi =σ2
i
α2 + σ2i. (19)
Christiana Sabett ILT Regularization in NMR Relaxometry
Regularization Parameter
The choice of regularization parameter α strongly influencesthe character of the solution.
Christiana Sabett ILT Regularization in NMR Relaxometry
Methods to Choose the Regularization Parameter
Numerous methods have been proposed to choose the idealregularization parameter. We will consider the following:
Christiana Sabett ILT Regularization in NMR Relaxometry
Discrepancy Principle
The discrepancy principle attempts to minimize the residualbased on a prespecified error bound ε, such that
||AFα − y || = ε (20)
for the optimal α.
Disadvantages:Requires a priori knowledge of the errorOften oversmooths the solution
Christiana Sabett ILT Regularization in NMR Relaxometry
Generalized Cross Validation
Generalized Cross Validation (GCV) extends the idea of“leave-one-out” cross-validation, minimizing the function
G(α) =||AFα − y ||2
(τ(α))2 (21)
withτ(α) = trace(I − A(AT A + α2LT L)−1AT ). (22)
where L is a differential operator or the identity matrix (Hansen).
Disadvantages:G(α) is difficult to minimize numerically due to its flatnessDoes not perform well with correlated errors
Christiana Sabett ILT Regularization in NMR Relaxometry
The L-Curve
Introduced by P.C. Hansen in 1993, the L-curve was originallyused as an analytical tool.
It plots the residual ||AFα − y ||2 against the size of the solution||Fα||2 as a function of α.
Christiana Sabett ILT Regularization in NMR Relaxometry
L-Curve Method
The L-curve method was proposed by Hansen and O’Leary asa means of choosing the regularization parameter α. Idea: Findthe “corner” of the L-curve.
Advantages over the other methods:Well-defined numericallyRequires no prior knowledge of the errorsNot heavily influenced by large correlated errors whenconsidered on a log scale
Christiana Sabett ILT Regularization in NMR Relaxometry
L-Curve Method
When GCV performs well, the chosen α value is close to thevalue chosen by the L-curve method.
Christiana Sabett ILT Regularization in NMR Relaxometry
L-Curve Method (Hansen, O’Leary)
Let (ρ, η) define a point on the L-curve in log scale.
FINDCORNER
1. Calculate several points (ρi , ηi) on each side of the corner.
Christiana Sabett ILT Regularization in NMR Relaxometry
L-Curve Method (Hansen, O’Leary)Let (ρ, η) define a point on the L-curve in log scale.
FINDCORNER
2. Fit a 3-dimensional cubic spline to those points (ρi , ηi , αi) afterfirst performing local smoothing.
Christiana Sabett ILT Regularization in NMR Relaxometry
L-Curve Method (Hansen, O’Leary)Let (ρ, η) define a point on the L-curve in log scale.
FINDCORNER
3. Compute the point of maximum curvature and find thecorresponding regularization parameter α0.
Christiana Sabett ILT Regularization in NMR Relaxometry
L-Curve Method (Hansen, O’Leary)Let (ρ, η) define a point on the L-curve in log scale.
FINDCORNER
4. Solve the regularized problem and add the new point(ρ(α0), η(α0)) to the L-curve.
Christiana Sabett ILT Regularization in NMR Relaxometry
L-Curve Method (Hansen, O’Leary)
Let (ρ, η) define a point on the L-curve in log scale.
FINDCORNER
5. Repeat until convergence.
Christiana Sabett ILT Regularization in NMR Relaxometry
Model Extensions
Characterize the “optimal” choice of penalty term using theL-curve method to optimize the regularization parameter.
L2 penalty:min
F∈RK||AF − y ||22 + α||F ||22
L1 penalty:min
F∈RK||AF − y ||22 + α||F ||1
Elastic Net:
minF∈RK
||AF − y ||22 + α1||F ||1 + α2||F ||2
Lp penalty:min
F∈RK||AF − y ||22 + α||F ||p
Christiana Sabett ILT Regularization in NMR Relaxometry
Model Extensions
Characterize the “optimal” choice of penalty term using theL-curve method to optimize the regularization parameter.
L2 penalty:min
F∈RK||AF − y ||22 + α||F ||22
L1 penalty:min
F∈RK||AF − y ||22 + α||F ||1
Elastic Net:
minF∈RK
||AF − y ||22 + α1||F ||1 + α2||F ||2
Lp penalty:min
F∈RK||AF − y ||22 + α||F ||p
Christiana Sabett ILT Regularization in NMR Relaxometry
Model Extensions
Characterize the “optimal” choice of penalty term using theL-curve method to optimize the regularization parameter.
L2 penalty:min
F∈RK||AF − y ||22 + α||F ||22
L1 penalty:min
F∈RK||AF − y ||22 + α||F ||1
Elastic Net:
minF∈RK
||AF − y ||22 + α1||F ||1 + α2||F ||2
Lp penalty:min
F∈RK||AF − y ||22 + α||F ||p
Christiana Sabett ILT Regularization in NMR Relaxometry
Model Extensions
Characterize the “optimal” choice of penalty term using theL-curve method to optimize the regularization parameter.
L2 penalty:min
F∈RK||AF − y ||22 + α||F ||22
L1 penalty:min
F∈RK||AF − y ||22 + α||F ||1
Elastic Net:
minF∈RK
||AF − y ||22 + α1||F ||1 + α2||F ||2
Lp penalty:min
F∈RK||AF − y ||22 + α||F ||p
Christiana Sabett ILT Regularization in NMR Relaxometry
Conclusion
An NMR relaxometry signal can be inverted via an inverseLaplace transform. The measurement of an additional, indirectdimension provides increased stability in the inversion.
RegularizationForm a least-squares problem.Add a Tikhonov regularization term for stability.
L-CurveCritical to the quality of the inversion is the choice ofregularization parameter α.Use parametric plot of ||AFα − y ||2 versus ||Fα||2 to find theoptimal α.
Christiana Sabett ILT Regularization in NMR Relaxometry
References I
Primary Material:
Celik, H., Bouhrara, M., Reiter, D. A., Fishbein, K. W., & Spencer,R. G. (2013). Stabilization of the inverse Laplace transform ofmultiexponential decay through introduction of a seconddimension. Journal of Magnetic Resonance, 236, 134-139.
Hansen, P. C. & O’Leary, D. P. (1993). The Use of the L-Curve inthe Regularization of Discrete Ill-Posed Problems. SIAM Journalon Scientific Computing, 14(6), 1487-1503.
Secondary Material:
Fu, W. J. (1998). Penalized Regressions: The Bridge versus theLasso. Journal of Computational and Graphical Statistics, 7(3),397-416.
Varah, J. M. (1983). Pitfalls in the Numerical Solution of LinearIll-Posed Problems. SIAM Journal on Scientific and StatisticalComputing, 4(2), 164-176.
Christiana Sabett ILT Regularization in NMR Relaxometry
References II
Berman, P., Ofer, L., Parmet, Y., Saunders, M., Wiesman, Z.(2013). Laplace Inversion of Low-Resolution NMR RelaxometryData Using Sparse Representation Methods. Concepts inMagnetic Resonance, 42(3), 72-88.
Hansen, P. C. (1992). Analysis of Discrete Ill-Posed Problems byMeans of the L-Curve. SIAM Review, 34(4), 561-580.
Tibshirani, R. (1996). Regression Shrinkage and Selection viathe Lasso. Journal of the Royal Statistical Society, 58(1),267-288.
Venkataramanan, L., Song, Y., & Hurlimann, M. D., (2002).Solving Fredholm Integrals of the First Kind with Tensor ProductStructure in 2 and 2.5 Dimensions. IEEE Transactions on SignalProcessing, 50(5), 1017-1026.
Christiana Sabett ILT Regularization in NMR Relaxometry