머신러닝 시그 세미나_(deep learning for visual recognition)

Observable Data

Model Information

Observable Data

Model Information

Speech Recognition

I am a boy

Observable Data

Model Information

ImageClassify

Cat

Observable Data

Model Information

HOW?

• Core visual object recognition

Feedback

• Weakness in kernel machine(SVM …):

• It does not scale well with sample size.

• Based on matching local templates.

• the training data is referenced for test data

• Local representation VS distributed representation

• N N(Neural Network) -> Kernel machine -> Deep NN

• Deep learning is all about deep neural networks

• 1949 : Hebbian learning

• Donald Hebb : the father of neural networks

• 1958 : (single layer) Perceptron

• Frank Rosenblatt

- Marvin Minsky, 1969

• 1986 : Multilayer Perceptron(Back propagation)

• David Rumelhart, Geoffrey Hinton, and Ronald Williams

• 2006 : Deep Neural Networks

• Geoffrey Hinton and Ruslan Salakhutdinov

Hand-CraftedFeatures

TrainableGenericClassifier

F(X;𝜃)

𝜃

SimpleClassifier

Layer1

SimpleClassifier

Layer2

LayerN

Layer1

SimpleClassifier

Layer2

LayerN

Layer1

SimpleClassifier

Layer2

LayerN

Layer1

SimpleClassifier

Layer2

LayerN

Layer1

SimpleClassifier

Layer2

LayerN

Trainable Generic Classifier

Hand-crafted Features

Layer1

SimpleClassifier

Layer2

LayerN

Trainable Generic Classifier

Hand-crafted Features

Layer1

SimpleClassifier

Layer2

LayerN

Shallow learning Deep learning

feature extraction by domain experts(SIFT, SURF, orb...)

automatic feature extraction from data

separate modules(feature extractor + trainable classifier)

unified model : end-to-end learning(trainable feature + trainable classifier)

i j

• http://www.cs.toronto.edu/~hinton/MatlabForSciencePaper.html

• convolutional neural networks (popular): LeCun

• Alex Krizhevsky: Hinton (python, C++)

• https://code.google.com/p/cuda-convnet/

• Caffe: UC Berkeley (C++)

• http://caffe.berkeleyvision.org/