Intelligent Information Technology Research Lab, Acadia University, Canada 1 Getting a Machine to Fly Learn Extending Our Reach Beyond Our Grasp Daniel.

Intelligent Information Technology Research Lab, Acadia University, Canada1

Daniel L. Silver

Acadia University, Wolfville, NS, Canada

[email protected]


Intelligent Information Technology Research Lab, Acadia University, Canada

Key Take Away

A major challenge in artificial intelligence has been how to develop common background knowledge

Machine learning systems are beginning to make head-way in this area

Taking first steps to capture knowledge that can be used for future learning, reasoning, etc.

3


Outline

Learning – What is it? History of Machine Learning Framework and Methods ML Application Areas Recent and Future Advances Challenges and Open Questions

4


What is Learning?

Animals and Humans① Learn using new experiences and prior

knowledge

② Retain new knowledge from what is learned

③ Repeat starting at 1.

Essential to our survival and thriving

5


What is Learning?(A little more formally)

Inductive inference/modeling Developing a general model/hypothesis from

examples Objective is to achieve good generalization for

making estimates/predictions It’s like … Fitting a curve to data

Also considered modeling the data Statistical modeling

7


What is Learning?

Generalization through learning is not possible without an inductive bias

= a heuristic beyond the data


Inductive Bias

ASH ST

THI RDSEC OND

ELM ST

FIR ST

PINE ST

OAK ST

Inductive bias depends upon:• Having prior knowledge• Selection of most related knowledge

Human learners use Inductive Bias


What is Learning?

Requires an inductive bias

= a heuristic beyond the data

Do you know any inductive biases?

How do you choose which to use?


Inductive Biases

Universal heuristics - Occam’s Razor Knowledge of intended use – Medical

diagnosis Knowledge of the source - Teacher Knowledge of the task domain Analogy with previously learned tasks

Tom Mitchell, 1980


What is Machine Learning?

The study of how to build computer programs that: Improve with experience Generalize from examples Self-program, to some extent


History of Machine Learning

1950 20001980

PDP GroupMulti-layerPerceptrons,New apps

Renaissance

1990

AI Success

Data mining,Web mining,User models,New alg.,Google

Present

Big Data,Web Analytics,Parallel alg.,Cloud comp.,Deep learning

Advances

1890

WilliamJames,Neuronal learning

Origins

1940

Donald Hebb,Math models, The PerceptronLimited value

Promise

1960

Minsky &Papert paper,Researchwanes

Hiatus

1970

Genetic alg,Version spaces,Decision Trees

Exploration


Of Interest to Several Disciplines

Computer Science – theory of computation, new algorithms

Math - advances in statistics, information theory Psychology – as models for human learning, knowledge

acquisition and retention Biology – how does a nervous system learn Physics – analogy to physical systems Philosophy – epistemology, knowledge acquisition Application Domains – new knowledge extracted from

data, solutions to unsolved problems

17


Classes of ML Methods Supervised – Develops models that predict the value of

one variable from one or more others: Artifical Neural Networks, Inductive Decision Trees, Genetic

Algorithms, k-Nearest Neighbour, Bayesian Networks, Support Vectors Machines

Unsupervised – Generates groups or clusters of data that share similar features K-Means, Self-organizing Feature Maps

Reinforcement Learning – Develops models from the results of a final outcome; eg. win/loss of game TD-learning, Q-learning (related to Markov Decision Processes)

Hybrids – eg. semi-supervised learning


Focus: Supervised Learning

Function approximation (curve fitting)

Classification (concept learning, pattern recognition)

x1

x2

AB

f(x)

x

21


Supervised Machine Learning Framework

Instance Space X

TrainingExamples

TestingExamples

(x, f(x))

Model ofClassifier

hInductive

Learning System

h(x) ~ f(x)


Supervised Machine Learning

Problem: We wish to learn to classifying two people (A and B) based on their keyboard typing.

Approach: Acquire lots of typing examples from each person Extract relevant features - representation!

M = number of mistakes T = typing time

Transform feature representation as needed Use an algorithm to fit a model to the data - search! Test the model on an independent set of examples of typing from

each person


Classification

Mistakes

Typing Speed

A

B

B

B

B

BB

B

BB

B

B

B

B

BB

B

B B

B

B

AA

AA

AA

AA

AA

A

A

A

A

A

A

A

A

A

B

B

B

B

B

B

BB

B

Logistic Regression

Y

Y=f(M,T)0

1

M T

Y


Classification

A

B

B

B

B

BB

B

BB

B

B

B

B

BB

B

B B

B

B

AA

AA

AA

AA

AA

A

A

A

A

A

A

A

A

A

B

B

B

B

B

B

BB

B

Artificial Neural Network

A

Mistakes

Typing Speed

M T

Y

…


Classification

A

B

B

B

B

BB

B

BB

B

B

B

B

BB

B

B B

B

B

AA

AA

AA

AA

AA

A

A

A

A

AA

A

A

B

B

B

B

B

B

BB

B

Inductive Decision Tree

AA

Mistakes

Typing Speed

M?

T? T?

Root

LeafAB

Blood Pressure Example


Application AreasData Mining: Science and medicine: prediction, diagnosis, pattern

recognition, forecasting Manufacturing: process modeling and analysis Marketing and Sales: targeted marketing, segmentation Finance: portfolio trading, investment support Banking & Insurance: credit and policy approval Security: bomb, iceberg, fraud detection Engineering: dynamic load shedding, pattern recognition

31


Application Areas

Web mining – information filtering and classification, social media predictive modeling

User Modeling – adaptive user interfaces, speech/gesture recognition

Intelligent Personal Agents – email spam filtering, fashion consultant,

Robotics – image recognition, adaptive control, autonomous vehicles (space, under-sea)

Military/Defense – target acquisition and classification, tactical recommendations, cyber attack detection

32


Recent and Future Advances

Robotics Neuroprosthetics Lifelong Machine Learning Deep Learning Architectures ML and Growing Computing Power NELL – Never-Ending Language Learner Cloud-based Machine Learning

33


OASIS: Onboard Autonomous Science Investigation System

Since early 2000’s Goal: To evaluate,

and autonomously act upon, science data gathered by spacecraft

Including planetary landers and rovers

34


Stanford’s Sebastian Thrun holds a $2M check on top of Stanley, a robotic Volkswagen Touareg R5

212 km autonomus vehicle race, Nevada Stanley completed in 6h 54m Four other teams also finished

Source: Associated Press – Saturday, Oct 8, 2005

DARPA Grand Challenge 2005

35


The Competition

36


Autonomous Underwater Vehicles

Arctic ExplorerAUV designed and built by International Submarine Engineering Ltd. (ISE) of Port Coquitlam, B.C.Used to map the sea floor underneath the Arctic ice shelf in support of Canadian land claims under the UN Convention on the Law of the Sea. Various military uses; e.g. mine detection, elimination

(Source: ISE, Mae Seto)

37


Literally Extending Our Reach – Neuroprosthetic Decoders

Dec, 2012 Andy Schwart, Univ.

of Pittsburgh Jan Scheuermann,

quadriplegic Brain-machine

interface, 96 electrodes

13 weeks of training High-performance neuroprosthetic

control by an individual with tetraplegia, The Lancet, v381, p557-654, Feb 2013

39


Lifelong Machine Learning (LML)

Considers methods of retaining and using learned knowledge to improve the effectiveness and efficiency of future learning

We investigate systems that must learn: From impoverished training sets For diverse domains of tasks Where practice of the same task happens

Applications: Intelligent Agents, Robotics, User Modeling, DM


Supervised Machine Learning Framework

Instance Space X

TrainingExamples

TestingExamples

(x, f(x))

Model ofClassifier

hInductive

Learning System

h(x) ~ f(x)

After model is developed and used it is thrown away.


Lifelong Machine Learning Framework

Instance Space X

TrainingExamples

TestingExamples

(x, f(x))

Model ofClassifier

h

Inductive Learning Systemshort-term memory

h(x) ~ f(x)

DomainKnowledge

long-term memoryRetention &ConsolidationInductive

Bias SelectionKnowledgeTransfer


Lifelong Machine Learning Framework

Instance Space X

TrainingExamples

TestingExamples

(x, f(x))

Model ofClassifier

h

Inductive Learning Systemshort-term memory

h(x) ~ f(x)

DomainKnowledge

long-term memoryRetention &ConsolidationInductive

Bias SelectionKnowledgeTransfer


Lifelong Machine Learning One Implementation

Instance Space X

TrainingExamples

TestingExamples

(x, f(x))

Model ofClassifier

h

h(x) ~ f(x)

Retention &ConsolidationKnowledge

Transfer

f2(x)

x1 xn

f1(x) f5(x)

Multiple Task Learning (MTL)

InductiveBias Selection

f3(x)f2(x) … f9(x) fk(x)

Consolidated MTL

DomainKnowledge

long-term memory


An Environmental Example

Stream flow rate prediction [Lisa Gaudette, 2006]

x = weather data

f(x) = flow rate

11

12

13

14

15

16

0 1 2 3 4 5 6Years of Data Transfered

MA

E (

m^

3/s)

No Transfer Wilmot Sharpe Sharpe & Wilmot Shubenacadie


Lifelong Machine Learning with csMTL

Example: Learning to Learn how

to transform images Requires methods of

efficiently & effectively Retaining transform

model knowledge Using this knowledge to

learn new transforms

(Silver and Tu, 2010)52


Lifelong Machine Learning with csMTL

55Demo


Deep Learning Architectures Hinton and Bengio (2007+)

Learning deep architectures of neural networks

Layered networks of unsupervised auto-encoders efficiently develop hierarchies of features that capture regularities in their respective inputs

Used to develop models for families of tasks

57


Deep Learning Architectures

Consider the problem of trying to classify these hand-written digits.


Deep Learning Architectures

2000 top-level artificial neurons2000 top-level artificial neurons

00500 neurons

(higher level features)500 neurons

(higher level features)

500 neurons(low level features)

500 neurons(low level features)

Images of digits 0-9

(28 x 28 pixels)

Images of digits 0-9

(28 x 28 pixels)

11 22 33 44

55 66 77 88 99

Neural Network:- Trained on 40,000 examples - Learns: * labels / recognize images * generate images from labels- Probabilistic in nature- Demo

2

3

1


ML and Computing Power

Moores Law Expected to

accelerate as the power of computers move to a log scale with use of multiple processing cores

60



IBMs Watson – Jeopardy, Feb, 2011: Massively parallel data processing system capable

of competing with humans in real-time question-answer problems

90 IBM Power-7 servers Each with four 8-core processors 15 TB (220M text pages) of RAM Tasks divided into thousands of stand-alone

jobs distributed among 80 teraflops (1 trillion ops/sec)

Uses a variety of AI approaches including machine learning

61



Andrew Ng’s work on Deep Learning Networks (ICML-2012)Problem: Learn to recognize human faces, cats, etc from unlabeled dataDataset of 10 million images; each image has 200x200 pixels9-layered locally connected neural network (1B connections)Parallel algorithm; 1,000 machines (16,000 cores) for three days

62

Building High-level Features Using Large Scale Unsupervised LearningQuoc V. Le, Marc’Aurelio Ranzato, Rajat Monga, Matthieu Devin, Kai Chen, Greg S. Corrado, Jeffrey Dean, and Andrew Y. NgICML 2012: 29th International Conference on Machine Learning, Edinburgh, Scotland, June, 2012.



Results: A face detector that is 81.7%

accurate Robust to translation, scaling,

and rotation

Further results: 15.8% accuracy in recognizing

20,000 object categories from ImageNet

70% relative improvement over the previous state-of-the-art.

63


Never-Ending Language Learner Carlson et al (2010)

Each day: Extracts information from the web to populate a growing knowledge base of language semantics

Learns to perform this task better than on previous day

Uses a MTL approach in which a large number of different semantic functions are trained together

64


Cloud-Based ML - Google

69

https://developers.google.com/prediction/


Machine Flight vs. Machine Learning

71

Factor Machine Flight Machine Learning

Effectiveness Travel higher, father Learn more things, accurately

To places not reachable Model complex phenomena

Efficiency Travel faster Learn faster

Lower cost Lower cost

Satisfaction Safe travel, beauty Confidence, elegance

Reach the moon, and beyond

Reach new knowledge, solve new problems


Thank You!

[email protected] http://plato.acadiau.ca/courses/comp/dsilver/ http://ML3.acadiau.ca

Intelligent Information Technology Research Lab, Acadia University, Canada 1 Getting a Machine to Fly Learn Extending Our Reach Beyond Our Grasp Daniel.

Documents

data slide

canada outline learning

silver acadia university

inductive bias slide

future learning

hours of learning

extent slide

deep learning advances