Learning School of Computing, University of Leeds, UK AI23 – 2004/05 – demo 2.

Learning

School of Computing,

University of Leeds, UK

AI23 – 2004/05 – demo 2

part 1: what is learning?

• what would you say learning is?

Part 1 : what is learning?

• meaning of learning is subject to

discussion

• recap some ideas

• high-level: “experience alters behaviour”

• low-level: “weights (on neuron connections)

change”

example 1: Yamauchi/Beer’s alternate worlds

• one agent, one goal, one landmark

• two kinds of world: landmark-far/near

a/b: landmark opposite to goal

c/d: landmark between agent and goal

• agent’s task: reach goal (how? what if it knows

the type of world it is in?)

example 1 [cont.]

• so, if world is known, a fixed strategy can be applied

• now, suppose a coin is tossed every 10 trials, and the kind of

world is changed accordingly

• how can the problem be solved? The agent must learn to detect

the kind of world it is in

• Yamauchi/Beer’s solution

• separately obtained (through artificial evolution) 3 distinct networks

that solve subtasks: world detection, LF and LN goal-finding

• integrated networks: agent uses first trial in the 10-trial sequence to

“learn” what world he is in; with that knowledge, he then switches to

the right strategy for that world, for the next 9 trials. On average,

95% success

example 1 [cont.]

• is that learning???

• can be seen as “experience altering behaviour”?

• no “weights changing”; rather, internal state of the

agent is changed (by setting a world-type flag) – does it

matter?

• the network is only learning one thing (the world the

agent is in); can that still be called learning?

example 2: c.elegans

1-mm worm

example 2: c.elegans

• no evidence of synaptic plasticity in c.elegans, i.e. no mechanisms for changing the weights between neurons

• however, c.elegans exhibits various kinds of learning capabilities (behavioural plasticity)

• habituation / sensitisation, associative learning

• this would mean that changing weights on neuron connections is not the only way in which learning occurs in nature

• lots to discover and understand yet!

Part 2: different forms of learning

• activity: recall different forms of learning

forms of learning

• neural networks

• Gradient-descent algorithms for the McCulloch

and Pitts neuron and for Feed-Forward Neural

networks

• delta rule

• backpropagation

• feed-forward nets used in some demos in

BEAST

forms of learning

• reinforcement learning

• agent interacts with environment and receives

• rewards (positive reinforcement)

• punishments (negative reinforcement)

• different to delta rule / backprop

• the agent is not given the correct answer, but only a good/bad signal; “quantitative v. qualitative”

• only desired results are needed to specify the problem, rather than intermediate actions; think of riding a bike, mazes, tic-tac-toe, backgammon

[see demo, pendulum]

forms of learning

• conditioning

• Pavlov’s experiments

• repeated pairing of two stimuli so that a

previously neutral (conditioned) stimulus

eventually elicits a response (conditioned

response) similar to that originally elicited by

nonneutral (unconditioned) stimulus

• notion of reward for artificial purposes

forms of learning

• Hebbian Learning

• form of learning in natural and artificial neural

networks

• potentiation of effective synaptic connections

and decay / depression of ineffective ones

• concept of simultaneous / concurrent /

correlated activation

forms of learning

• winner takes all

• a form of competitive learning in natural and artificial neural networks

• neurons compete on activation over an input

• winner neuron gets reinforced

• Hebbian-like rule

• will be seen in this module

forms of learning

• evolutionary algorithms

• search algorithms inspired by natural evolution:

population evolves, improving its “fitness”

• concepts of assessment (of an individual), selection,

variation (of population’s individuals over time)

• can be used as optimisation tools, even to "train"

neural networks

• Yamauchi/Beer

• also the way we use them in BEAST

• will be seen in this module

forms of learning

• imitation

• a form of learning in nature and (recently) in robotics

• individuals learn by replication and repetition of behaviour observed in others

[see demo, CogVis] work by CogVis lab @ SOC

• behaviour is adapted to their particulars

[see demo, tennis]

forms of learning

• mimicry

• a form of evolutionary learning: species / groups learn

by mimicking desirable genetic traits from other

species / groups

• “wasp-like” insects work by J.Noble / D. Franks, SOC

• social learning

• learning is achieved via the communication of

information within a social structure

• schools, books; birds, mammals

learning

• activity: where are the above used in

nature and in bio-inspired algorithms?

thank you!