Deep convolutional neural networks for accelerated dynamic magnetic resonance imaging Christopher M. Sandino 1 , Neerav Dixit 1 Purpose References Discussion / Conclusions 1 Department of Electrical Engineering, Stanford University, Stanford, CA • Magnetic resonance imaging (MRI) scan times are relatively slow, especially for dynamic acquisitions like in the heart • Scan time can be accelerated by compressed sensing 1 (CS) schemes that exploit data redundancy to reconstruct undersampled MR images • However, CS-based reconstruction times are long because they employ iterative algorithms to solve optimization problems • Critical time between patient exam and diagnosis is extended by hours – making MRI infeasible for urgent clinical situations Goal: Use convolutional neural networks to efficiently and accurately reconstruct highly undersampled dynamic MRI data Background • CS-based image reconstruction 1 is based on iteratively solving non-linear inverse problems of the form: • CNNs are well-suited for modelling this task 2 and have previously been used to learn CT 3 and MRI 4 static CS recon pipelines Input time-series 20 64 64 256 x 256 256 x 256 256 x 256 64 128 2 128 2 128 2 128 128 128 64 2 256 256 64 2 64 2 256 32 2 512 512 32 2 32 2 512 16 2 16 2 1024 16 2 1024 1024 32 2 512 512 32 2 32 2 512 64 2 256 256 64 2 64 2 128 2 128 2 128 128 256 128 2 128 256 x 256 64 64 256 x 256 256 x 256 20 256 x 256 conv 3x3, stride 1, pad 1; batch norm; ReLU max pool 2x2, stride 2 convT 2x2, stride 2 conv 1x1, stride 1, pad 0 copy and concatenate Output time-series Modified U-net 5 architecture: Methodology • Each dataset is expanded into N=896 (2D+time) examples • Each example has 20 time frames (50 ms temporal resolution) • Datasets are not fully sampled, but are diagnostic quality • Acquisition time: ~15 min, CS reconstruction time: ~2 hours 4-D cardiac MRI datasets: Deep Reconstruction Workflow: • Still many questions: Similar performance for dynamic data? Best loss function to train on? Upper limit for undersampling? How to evaluate CNN reconstructions? (Courtesy of Dr. Shreyas Vasanawala) N train = 2688 N val = 896 N test = 896 PSNR (dB) SSIM Speed CS (Truth) - - 2hr19 m Naive 57.792 0.629 3s CNN-L2 62.976 0.734 57s CNN-L1(F) 62.923 0.743 53s CNN-SSIM 62.526 0.720 200s Experiment #2: Does loss function impact image quality? Figure from Lustig et al. 1 [1] M Lustig, et al. "Compressed sensing MRI." IEEE signal processing magazine, 2008. [2] K Gregor, and Y LeCun. "Learning fast approximations of sparse coding." ICML, 2010. [3] KH Jin, et al. "Deep Convolutional Neural Network for Inverse Problems in Imaging." arXiv, 2016. [4] K Hammernik, et al. "Learning a Variational Network for Reconstruction of Accelerated MRI Data." arXiv, 2017. [5] O Ronneberger, et al. "U-net: Convolutional networks for biomedical image segmentation." International Conference on Medical Image Computing and Computer-Assisted Intervention. 2015 • We present a CNN model that can reconstruct dynamic MR images comparable to standard techniques in under one minute (150x faster than CS) • Potential to also accelerate scan time (2x faster than standard) • Not able to resolve temporal dynamics well – look to RNNs? • CNNs provide faster scan and recon times – potential to make MRI cheaper and more feasible in urgent clinical situations Performance Statistics: • L1 / L2 similar performance • SSIM better at preserving structure and edges Experiment #1: Can CNN reconstruct undersampled data better than CS? L2: L1(F): SSIM: ||y - ˆ y || 2 ||F (y - ˆ y )|| 1 1 2 (1 - SSIM(y, ˆ y )) minimize x ||F s x - y || 2 2 + λ||φ(x)|| 1 Results PSNR=56.7 dB PSNR=62.4 dB PSNR=51.0 dB PSNR=57.3 dB • Used L2 loss • Ground truth is diagnostic quality CS recon • CNN outperforms Naïve recon by ~6 dB