Mehdi S. M. Sajjadi, Bernhard Schölkopf, Michael Hirsch msajjadi.com Single Image Super-Resolution Through Automated Texture Synthesis EnhanceNet Max Planck Institute for Intelligent Systems s tate of the art by PSNR o ur result Mehdi S. M. Sajjadi Bernhard Schölkopf Michael Hirsch
13
Embed
EnhanceNet - webdav.tuebingen.mpg.dewebdav.tuebingen.mpg.de/pixel/enhancenet/files/... · ¤ DRCN, VDSR (Kim et al., CVPR 2016) ¤ NTIRE challenge on image super-resolution (CVPR
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
EnhanceNet: Single-Image Super-Resolution Through Automated Texture Synthesis
Mehdi S. M. Sajjadi, Bernhard Schölkopf, Michael Hirsch msajjadi.com
Single Image Super-ResolutionThrough Automated Texture Synthesis
EnhanceNet
Max Planck Institutefor Intelligent Systems
state of the art by PSNR our result
Mehdi S. M. Sajjadi
BernhardSchölkopf
Michael Hirsch
EnhanceNet: Single-Image Super-Resolution Through Automated Texture Synthesis
Mehdi S. M. Sajjadi, Bernhard Schölkopf, Michael Hirsch msajjadi.com
Single Image Super-Resolution
2
Low-resolution input
?
Original high-res image
EnhanceNet: Single-Image Super-Resolution Through Automated Texture Synthesis
Mehdi S. M. Sajjadi, Bernhard Schölkopf, Michael Hirsch msajjadi.com
Method: Classical approach
¤ Minimize mean-squared error¤ MSE = %
&'∑ ∑ 𝐼*+, − 𝐼+,
./,
0+
¤ Evaluation by Peak signal-to-noise ratio (PSNR)¤ PSNR = −10 log%6 𝑀𝑆𝐸
¤ The field is dominated by convolutional neural nets¤ SRCNN (Dong et al., ECCV 2014)¤ DRCN, VDSR (Kim et al., CVPR 2016)¤ NTIRE challenge on image super-resolution (CVPR 2017)
3
EnhanceNet: Single-Image Super-Resolution Through Automated Texture Synthesis
Mehdi S. M. Sajjadi, Bernhard Schölkopf, Michael Hirsch msajjadi.com
State of the art by PSNR
4
CNN
Low-resolution input High-resolution output
EnhanceNet: Single-Image Super-Resolution Through Automated Texture Synthesis
Mehdi S. M. Sajjadi, Bernhard Schölkopf, Michael Hirsch msajjadi.com
Is PSNR the right metric?
5
Low-resolution input Original imageGenerated image
EnhanceNet: Single-Image Super-Resolution Through Automated Texture Synthesis
Mehdi S. M. Sajjadi, Bernhard Schölkopf, Michael Hirsch msajjadi.com
EnhanceNet’s output
6
Low-resolution input Original imageOur result
EnhanceNet: Single-Image Super-Resolution Through Automated Texture Synthesis
Mehdi S. M. Sajjadi, Bernhard Schölkopf, Michael Hirsch msajjadi.com
PSNR vs. visual similarity
7
High-resolutionimage
Low-resolutionimage
Optimal PSNR Realistic imageLow PSNR
EnhanceNet: Single-Image Super-Resolution Through Automated Texture Synthesis
Mehdi S. M. Sajjadi, Bernhard Schölkopf, Michael Hirsch msajjadi.com
Method: residual CNN with loss
¤ Euclidean distance / mean squared error (MSE)
¤ 𝐼* − 𝐼.
.= %
:;<∑ 𝐼*+, − 𝐼*+,
.�+,
¤ Perceptual loss (Dosovitskiy and Brox 2016, Johnson et al. 2016)
¤ 𝜙 𝐼* − 𝜙 𝐼.
.MSE in VGG feature space
¤ Texture loss / style transfer (Gatys et al. 2015)