Machine Learning: Advanced Topics Overview

Sergii Shelpuk

August 2013

Machine Learning

Advanced Topics

Overview

Agenda

1. Machine Learning: current state and open problems

2. Best conferences and publications in Machine Learning

3. Deep Learning: overview4. Reinforcement Learning: overview5. ICML 2013 overview6. Current state of the art in ML7. Q&A

Current State and Open Problems

Years State Examples

Before 1960 Initial approach Samuel’s checkers

1960 - 1970 Rise of neural networks Perceptron

1970 - 1980 Fall of neural networks

1980 - 1990 Jeff Hinton popularizes backpropagation. Interest to NN comes back

Convolutional networksLeNET and others

1990 - 2000 Era of large margin classifiers SVM, Kernels

2000 - present Era of neural networks. Jeff Hinton and others pioneered deep learning

Google Image SearchSpeech recognitionFace recognition

TOP ML Events

Microsoft Academic Research

Deep Learning - Overview

What features should I put in?

Deep Learning - Overview

Deep Learning - Overview

Deep Learning - Overview

Deep Learning - Overview

ML Classifier

Data

Feat

ure

ext

ract

or Features

Class

All specialized computer vision/speech recognition/etc algorithmsPrevious state of the arts

Deep Learning - Overview

See also: Jeff Hawkins, On Intelligence

Deep Learning - Overview

Deep Learning - Overview

wheels

steering wheel

Deep Learning - Overview

Autoencoder

See more: Stanford UFLDL Tutorial

Deep Learning - Overview

See more: Convolutional Deep Belief Networksfor Scalable Unsupervised Learning of Hierarchical Representations

Deep Learning - Overview

See more: Building high-level features using large scale unsupervised learning

Deep Learning - Overview

Pre-trained as AutoencoderTypical classification

neural network

Deep Learning - Overview

Results: Video Recognition

Deep Learning - Overview

Results: Phoneme Classification

Deep Learning - Overview

Other Results

Deep Learning - OverviewImage-Net Competition

Deep Learning - OverviewImage-Net CompetitionResults from the testset

See more: ImageNet Classification with Deep Convolutional Neural Networks

Deep Learning - Overview

Error rates on the ILSVRC-2012 competition

▪ University of Tokyo

▪ Oxford University Computer Vision Group

▪ INRIA (French national research institute in CS) + XRCE (Xerox Research Center Europe)

▪ University of Amsterdam

▪ 26.1% 53.6%

▪ 26.9% 50.0%

▪ 27.0%

▪ 29.5%

classificationclassification&localization

▪ University of Toronto (Alex Krizhevsky) ▪ 16.4% 34.1%▪

Reinforcement Learning - Overview

System

Control Agent

Observe state of the system

Send control signals

Reinforcement Learning - Overview

State 1

State 3

State 4

State 5

State 8

State 7

State 6

State 2State 10

State 9

Current state

Undesirable state

Undesirable state

Desirable state

Reinforcement Learning - Overview

State 1 State 2 State 3 State 4 State 5 … State n

State 1 0.05 0.01 0.02 0.15 0.03 … 0.08

State 2 0.02 0.01 0.15 0.18 0.11 … 0.12

State 3 0.01 0.15 0.19 0.5 0.12 … 0.18

… … … … … … … …

State n 0.8 0.05 0.01 0.01 0.02 … 0.1

For action 1

State 1 State 2 State 3 State 4 State 5 … State n

State 1 0.04 0.02 0.01 0.35 0.01 … 0.08

State 2 0.02 0.01 0.15 0.18 0.11 … 0.12

State 3 0.01 0.15 0.19 0.5 0.12 … 0.18

… … … … … … … …

State n 0.8 0.05 0.01 0.01 0.02 … 0.1

For action 2

…For action n

Reinforcement Learning - Overview

Inverted pendulum problem

http://www.youtube.com/watch?v=aH_d_d_DTkU

Stanford robotic helicopter

http://www.youtube.com/watch?v=Idn10JBsA3Q

ETH Zurich Quadrocopter Pole Acrobatics

http://www.youtube.com/watch?v=pp89tTDxXuI

ICML 2013 overview

4 days conference2 days workshopTop universities and researchers

Tracks:• Deep Learning• Compressed Sensing• Reinforcement Learning• Topic Modeling• General SVM and Decision Tree Methods• Spectral Learning & Tensors• Online Learning• Structured Labeling• Dimensionality Reduction• Statistical Methods• Transfer Learning• General Methods• Optimization• Clustering• Learning Theory• Dimensionality Reduction and Semi-Supervised Learning• Computer Vision• Kernel Methods• Matrix Factorization

All materials are available online:http://icml.cc/2013/

Just few photos :-)

ICML 2013 Materials

Multiple Clustering:http://dme.rwth-aachen.de/en/DMCS

Maxout networks:http://jmlr.org/proceedings/papers/v28/goodfellow13.pdf

Large scale deep belief networks on GPU:http://jmlr.org/proceedings/papers/v28/coates13.pd

The End