National Lab of Radar Signal Processing Bo Chen National Lab. of Radar Signal Processing Xidian University, China Deep Learning and Its Applications to Radar ATR Joint work with Bo Feng, Jun Ding, Gungor Polatkan, Guillermo Sapiro, David Blei, David B. Dunson and Lawrence Carin MLA 2014
39
Embed
Deep Learning and Its Applicationsto Radar ATRsee.xidian.edu.cn/vipsl/MLA2014/ChenBo.pdf · · 2014-11-20Deep Learning and Its Applicationsto Radar ATR Joint work with Bo Feng,
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
National Lab of Radar Signal Processing
Bo Chen National Lab. of Radar Signal Processing
Xidian University, China
Deep Learning and Its Applications to Radar ATR
Joint work with Bo Feng, Jun Ding, Gungor Polatkan, Guillermo Sapiro, David Blei, David B. Dunson and Lawrence Carin
MLA 2014
National Lab of Radar Signal Processing
Outline
Deep/Multilayered Models
Deep Models with Bayesian Nonparametric
Convolutional Factor Analysis with
(Hierarchical) Beta Process
Summary and Future Work
2014-11-20
National Lab of Radar Signal Processing
Outline
Deep/Multilayered Models
Deep Models with Bayesian Nonparametric
Convolutional Factor Analysis with
(Hierarchical) Beta Process
Summary and Future Work
2014-11-20
National Lab of Radar Signal Processing2014-11-20
Success Stories• Computer Vision:
− Image inpaiting/denoising, segmentation− Object recognition/detection, scene understanding− Video analysis
• Information Retrieval/NLP:− Text, audio, and image retrieval− Parsing, machine translation, text analysis
Predictive Sparse Decomposition[Kavukcuoglu et al.,‘09]
National Lab of Radar Signal Processing2014-11-20
Denoising Autoencoder[Vincent et. Al., 2008]
Figure credit for Vincent
National Lab of Radar Signal Processing2014-11-20
Deep Belief Networks (Greedy)
• Construct an RBM with an input layer v and a hidden layer h
• Stack another hidden layer on top of the RBM to form a new RBM
• And so on.
[Hinton et. al., 2006]
National Lab of Radar Signal Processing2014-11-20
Deep Belief Networks (Greedy),[Hinton et. al., 2006]
Generating samples
National Lab of Radar Signal Processing2014-11-20
Deep Boltzmann Machine
1 2 3 1 1 1 2 2 2 3 3, , , ; T T TE v h h h θ v W h h W h h W h
The energy of the state is defined as: 1 2 3, , ,v h h h
1 2 1 2 21| ,k ik i km mi m
p h g W v W h
v h
2 1 3 2 1 3 31| ,k ik i km mi m
p h g W h W h h h 3 2 3 21|k jk j
jp h g W h
h
1 1 11|k kj jj
p v g W h
hInference:
Figure credit for Ruslan
National Lab of Radar Signal Processing
Outline
Deep/Multilayered Models
Deep Models with Bayesian Nonparametric
Convolutional Factor Analysis with
(Hierarchical) Beta Process
Summary and Future Work
2014-11-20
National Lab of Radar Signal Processing2014-11-20
Bayesian Nonparametric Modeling• What is Bayesian nonparametric?
− It doesnot mean “no parameters”, a really large parametric model
− “not parametric,” not restricted to objects whose dimensionality stays fixed as more data is observed. More flexible according flexible data structure
− A model over infinite dimensional function or measure spaces
− A family of distributions that is dense in some large space
• Why nonparametric models?− broad class of priors that allows data to “speak for itself”− side-step model selection and averaging
National Lab of Radar Signal Processing2014-11-20
Dirichlet distribution Dirichlet process
Beta distribution Beta processGaussian distribution Gaussian processPoisson distribution Poisson process
Bayesian Nonparametric Models
Dirichlet Process / Chinese Restaurant Process Beta Process / Indian Buffet Process
National Lab of Radar Signal Processing2014-11-20
Autoencoder with Nonparametric Priors• Deep Sparse Graphical Models via CIBP (R. Adam et. al., 2010)
• Autoencoder with Gaussian Process (J. Snoek et. al., 2012)
National Lab of Radar Signal Processing2014-11-20
Autoencoder with Nonparametric Priors• Beta Process RBMs (R. Mittelman et. al., 2013)
Sparseness Analysis• The impact of sparse hyperparameters on sparseness and model performance
the sparseness of filters:
2014-11-20
National Lab of Radar Signal Processing
Sparseness Analysisthe sparseness of hidden nodes:
the sparseness of binary indicators:
2014-11-20
National Lab of Radar Signal Processing
Layered Representation
Layer 1 Layer 2 Layer 3
2014-11-20
National Lab of Radar Signal Processing
Online Learning
Held-out RMSE with different sizes of minibatches on Caltech101 data, as in Fig. 6. (a) Layer 1, (b) Layer 2.
2014-11-20
National Lab of Radar Signal Processing
HBP on Caltech101Task=102; Images:1020; 1000 Layer-2 filters ranked by the global usage
It appears that as the range of image classes considered within an HBP analysis increases, the form of the prominent filters tend toward simple filter forms.
2014-11-20
National Lab of Radar Signal Processing2014-11-20
Classification
National Lab of Radar Signal Processing
Outline
Deep/Multilayered Models
Deep Models with Bayesian Nonparametric
Convolutional Factor Analysis with
(Hierarchical) Beta Process
Summary and Future Work
2014-11-20
National Lab of Radar Signal Processing
Summary and Future Work Build new convolutional deep networks based on factor
analysis with BP/IBP Infer the number of filters at each layer of the deep model
from the data by an IBP/BP construction Multi-task feature learning for simultaneous analysis of
different families of images via HBP Future work:
combine topic modeling with the model develop a classifier special for convolutional
property with better generalization build deep model with the encoder style