[email protected] You wanted to know what the Matrix is? Matthew Casey.

[email protected]

You wanted to know what the Matrix is?

Matthew Casey

2The Matrix (1999), © 1999 Warner Bros.

3

Modelling the Action of the Brain

• You wanted to know what the Matrix is?– ‘What you can feel […] smell […] taste and see […] are

simply electrical signals interpreted by your brain’ (The Matrix 1999)

– The outlook may be dystopian, but in artificial intelligence, the dream is still to build ‘intelligent machines’ (even if they do take over the world)

• Even Alan Turing was similarly motivated, for example, in 1947, when he wrote:– "I am more interested in the possibility of producing

models of the action of the brain than in the practical applications to computing.“ (Hodges 1992:363)

• Yet modelling “the action of the brain”– Requires knowledge of the actions to be modelled– Requires a robust understanding of the tools used– Has practical applications as well

Hodges, A. (1992). Alan Turing: The Enigma. London: Vintage, Random House.

4

Knowledge of the Actions

• Why?– To understand ‘the brain’ better– To use knowledge of the brain to build more

‘intelligent machines’

• Do we know enough about the brain?– We have a developing understanding (e.g.

Carter 2000)– …and detailed models of specific aspects (e.g.

Feigenson et al 2004)– …but our knowledge appears quite poor (e.g.

Olshausen 2005)

Olshausen, B.A. & Field, D.J. (2005). How Close are we to Understanding V1? Neural Computation, vol. 17, pp. 1665-1699.

Carter, R. (2000). Mapping the Mind. London, UK: Phoenix.Feigenson, L., Dehaene, S. & Spelke, E. (2004). Core systems of number. Trends in Cognitive Sciences, vol. 8(7), pp. 307-314.

5

Knowledge of the Tools• Why?

– To explore new architectures (theory and application)– To understand how biological systems can give rise to

behaviour• Which architectures and algorithms?

– What types of neuron, network or algorithm?– Our focus is on combining neural networks (Sharkey 1999)– Limited theory, but wide application, not just computational

neuroscience (e.g. Kittler at al 1998)• But we need a sufficiently robust understanding of these

– To understand model behaviour– To generalise and apply elsewhere– So far, this robust understanding exists only for a small

number of multi-net architectures (negative correlation learning and mixture-of-experts)Sharkey, A.J.C. (1999). Multi-Net Systems. In Sharkey, A. J. C. (Ed), Combining Artificial Neural Nets:

Ensemble and Modular Multi-Net Systems, pp. 1-30. London: Springer-Verlag.

Kittler, J., Hatef, M., Duin, R.P.W. & Matas, J. (1998). On Combining Classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20(3), pp. 226-239.

6

Practical Applications

• The grand aim of artificial intelligence?– To build ‘intelligent machines’?

• Grand Challenges (Hoare & Milner 2004)– Architecture of Brain and Mind– ‘Bottom-up specification […] of computational

models’– ‘Top-down development of a new kind of theory’

• But– Knowledge spread across different disciplines– Foresight Cognitive Systems (Sharpe 2003): we

need to develop an inter-disciplinary understanding

Hoare, T & Milner, R. (2004). Grand Challenges in Computing: Research. UK Computing Research Committee (UKCRC).Sharpe, B. (2003). Foresight Cognitive Systems Project: Applications and Impact. London: Department of Trade and Industry, Office of Science and Technology.

7

Solutions?

• Modelling the ‘action of the brain’– Use available tools to prototype aspects of cognition:

neural networks and signal processing, …– Use inter-disciplinary knowledge: psychophysics and

neurobiology

• Grand Challenge suggests:– Build increasingly more complex models– Combine theory and empirical results to build better

models

• Most of the brain is dedicated to some form of sensory processing– A good place to start…– …building upon the wealth of computational work that has

already been done, but hasn’t quite solved the problem

8The Matrix (1999), © 1999 Warner Bros.

9

Multi-sensory Processing• Could you understand what was being said in the film clip?

– You should be able to, even without the sound– Your other senses, memory, emotions, etc. work together

• But, can we get a machine to do the same thing?– Computer vision, speech recognition, etc. are hard tasks– The brain does it very well – but how?

• Uni-modal or multi-modal processing?– Typically, only single modalities have been modelled– Yet evidence suggests that there is some low-level cross-

sensory processing (Thesen et al 2004)– Can computational models benefit from a similar approach?

• Aim– To explore multi-sensory processing to see if it can help us

build models/machines that are closer to being intelligent– We therefore need to build ever more complex models of the

brain that can process different sensory inputs in an integrated way

Thesen, T., Vibell, J.F., Calvert, G.A. & Österbauer, R.A. (2004). Neuroimaging of Multisensory Processing in Vision, Audition, Touch, and Olfaction. Cognitive Processing, vol. 5(2), pp. 84-93.

10

Multi-modal Processing

• Multi-sensory integration leads to a multi-modal understanding: the whole ‘picture’

• Numerical cognition– Exploring a multi-modal understanding in numeracy– Integrated cognitive abilities for manipulating numbers:

subitization, counting, addition and number representation (Casey et al, Casey 2004, Ahmad et al 2002)

• Multi-sensory processing and tools– Combines visual and linguistic inputs to produce a single

output– Combining supervised and unsupervised learning in

parallel and in sequence– But better (and less specific) models of vision, audition,

etc. needed

Ahmad, K., Casey, M.C. & Bale, T. (2002). Connectionist Simulation of Quantification Skills. Connection Science, vol. 14(3), pp. 165-201.

Casey, M.C. & Ahmad, K. (accepted). A Competitive Neural Model of Small Number Detection. Neural Networks.Casey, M.C. (2004). Integrated Learning in Multi-net Systems. Unpublished doctoral thesis. Guildford, UK: University of Surrey.

11

Low-level Vision

• Modelling low-level human vision (with Sowden)– Category learning task (Notman et al 2005)– Task dependence tunes low-level processing– Categorical perception effect: measurable difference in

‘within class’ versus ‘between class’ discrimination– What causes this CP Effect?

• Well-established area of investigation and computational modelling– However, new understanding of sensory processing: low-

level vision is not static– Dynamic changes to how we process low-level visual

input depending upon task (visual and task inputs)– Linked to low-level cross-modal processing

Notman, L.A., Sowden, P.T. & Özgen, E. (2005). The Nature of Learned Categorical Perception Effects: A Psychophysical Approach. Cognition, vol. 95(2), pp. B1-B14.

12

Categorical PerceptionDo these belong to the same or a different category?

13

Category Learning

Category B

Category A

Notman et al 20050o

45o

90o

135o 180o

315o

270o

225o

Distance changes through learning

3f phase angle

Images combine an f and 3f grating

14

Receptive Field Modelling• Modelling low-level vision:

– 2-D Gabor filtering: frequency, phase and orientation (cf. Itti & Koch 2001)

– Split into receptive fields– Neuron per field, fed into discrimination model– Task driven: discrimination/categorisation– Meant to learn how to combine receptive field values

• But…– Grappling with ‘plausible’ models of vision– MLP only: needs to model ‘templates’ and lateral inhibition

(competitive learning?)– Assumes a model of vision that may be wrong (cf. Olshausen

2005)– Relies upon simplistic grating patterns (as used in human tests)

• Despite the problems– This simple model is starting to show that the CP Effect can be

reproduced because of the process of learning categoriesItti, L. & Koch, C. (2001). Computational Modelling of Visual Attention. Nature Reviews Neuroscience, vol. 2(3), pp. 194-203.Olshausen, B.A. & Field, D.J. (2005). How Close are we to Understanding V1? Neural Computation, vol. 17, pp. 1665-1699.

15

What Next?

• Work so far has:– Built multi-net adaptive models of specific

cognitive abilities– Focussed on single vision, linguistic and task inputs

• Need to:– Build better model of vision and other senses– Need to combine these models– Perhaps demonstrate via simple robotics

• But…– We still lack a robust understanding of the tools– Despite exploring novel architectures and

algorithms: sequential and parallel systems (Casey et al 2004)

Casey, M.C. & Ahmad, K. (2004). In-situ Learning in Multi-net Systems. In Yang, Z.R., Everson, R. & Yin, H. (Ed), Proceedings of the 5th International Conference on Intelligent Data Engineering and Automated Learning (IDEAL 2004), Lecture Notes in Computer Science 3177, pp. 752-757. Heidelberg: Springer-Verlag.

16

Work in Progress

• Limited theory on combined systems:– Ensemble (NCL) and modular systems (mixture-of-experts)

• Individual networks are well understood– Are multi-nets just single networks?– Are neural network ensembles just partially connected

feedforward systems (cf. Brown 2004)?• We need a better understanding of useful architectures

(without getting lost in the detail)– Ensembles: game theory approach (Zanibbi, Casey & Brown)– Ensembles: application to classification (Zhang & Casey)– Recurrence and single-nets (Taskaya-Temizel & Casey)

• …of generic architectures:– Set theoretic (Shields & Casey)– Infer properties of combined system from components

• …and to think about other aspects of the brain:– Emotion (Pavlou & Casey)

Brown, G. (2004). Diversity in Neural Network Ensembles. Unpublished doctoral thesis. Birmingham, UK: University of Birmingham.

17

Coming Soon…

Workshop on Biologically Inspired Information Fusion

UniS 22nd and 23rd August 2006Matthew Casey, Paul Sowden, Tony Browne, Hujun

Yin

Bringing together computer scientists, engineers, psychologists and biologists to discuss multi-sensory processing

http://www.soc.surrey.ac.uk/ias/workshops/biif/

[email protected]

Thank you

Questions?

19

Numeracy and the Brain

Verbal System

Comparison

Arithmetic Facts

Visual Number Form

Temporal Lobe

Magnitude Representation

Parietal Lobe

Frontal Lobe

Left Hemisphere

Occipital Lobe Visual Number

Form

Temporal Lobe

Magnitude Representation

Parietal Lobe

Frontal Lobe

Right Hemisphere

Comparison

Dehaene, S. (2000). The Cognitive Neuroscience of Numeracy: Exploring the Cerebral Substrate, the Development, and the Pathologies of Number Sense. In Fitzpatrick, S.M. & Bruer, J.T. (Eds), Carving Our Destiny: Scientific Research faces a New Millennium, pp. 41-76. Washington: Joseph Henry Press.