Brain-inspired equivalence structure extraction technique for generating frames

Hiroshi Yamakawa

FUJITSU LABORATORIES LTD.JAPAN

Brain-inspired equivalence structure (ES) extraction technique for generating frames

Outline

1. Human-level intelligence can explore from neocortex learning.

Artificial intelligence (AI) lacks flexible sampling function of neocortex Equivalence structure (ES) extraction is key for such a function

2. Use local sequences to extract equivalence structures (ESs). Inspired by theta phase precession of hippocampal formation

3. Simple simulation of ES extraction

4. Summary Frame generation:

Promising way to achieve artificial general intelligence (AGI)

2

HLDL mostly means human-level artificial intelligence (HLAI)

3

Complete intelligenceDeep learning (DL) High-performance unsupervised machine learning technology corresponding to neocortex

Creativity

Intuition

General intelligence

Artificialintelligence

Humanintelligence

D

Emotion(Amygdala)

Reinforcement learning

(Basal ganglia)

Control theory(Cerebellum) Efficient arithmetic

operation and logic inference

Retrieval from big

data

Pattern recognition

Deep learning(DL)

HLDL (Neocortex + Hippocampus)

…because untrodden machine intelligences D

are concentrated on neocortex.

Human-level DL (HLDL) Fully simulate neocortex computing and its learning functions. (with help of hippocampus).

Feasible intelligence (with limited resources)

What is the problem in achieving HLDL?

Convolution layer ： • Well-developed for

machine learning: Simple cell, Auto encoder

network, SOM, Boltzmann machine, Info-MAX, Manifold learning, ...

Eye

Visualcortex

Deep learning lacks flexible sampling

4

CaudateQuick generation of best next-move

V1

V2

V3

MTG/V6

Sampling/Pooling layer: • Human encode structure of

hierarchical retinotopy:→ Complex cell, Max-

pooling, ...• Supports visual invariances → Need flexible sampling

Example: Intuitive “decision making” for chess-like game

Chess-like game

Sampling

Convolution

Sampling

Convolution

Sampling

Convolution

Sampling

Convolution

High-level featuresSupports intuitive decision making - Cannot be explained by experts - Cannot be acquired by deep learning

Hippocampussupportlearning

PrecuneusPerception of board patternHippocampus

Support learning of neocortex

(Wan, Science 2011)

5

C

AB

DEFZ

XY

Time

1 2 3 4 5 6 7

Time

1 2 3 4 5 6 7

Subspace Subspace

Combinedframe

Equivalence structures for flexible sampling

Time: t

Var

iabl

e se

t:

x

D

FE

G

A

H1 2 3 4 5 6 7

BC

Original frame

InvarianceIncreased events enhance deductive inference.

Equivalence structure (ES) …indicates portions of subspace that could be regarded as equivalent.

Invariance in basic image processing Invariance for face recognition

D

FE

G

A

H

Input sequence

1 2 3 4 5 6 7

BC

Need more flexibility for higher-level

sampling

Outline

1. Human-level intelligence can explore from neocortex learning.

Artificial intelligence (AI) lacks flexible sampling function of neocortex Equivalence structure (ES) extraction is key for such a function

2. Use local sequences to extract equivalence structures (ESs). Inspired by theta phase precession of hippocampal formation

3. Simple simulation of ES extraction

4. Summary Frame generation:

Promising way to achieve artificial general intelligence (AGI)

6

7

Static patterns are poor for ES extraction

If using common static binary patterns

as similarity to compare subspaces,

Needs similarity with rich variation.

C

AB

DEFZ

XY

ESs Subspace of d variables

Combined frame

Time: t

Set

of N

var

iabl

es

D

FE

G

A

H

Input sequence

1 2 3 4 5 6 7

BC

Original frame

Too many other

subspaces

…this could exist in

neocortex.

Problem:

Variation in static

patterns 2d is not

enough to categorize

thousands of

subspaces NCd(~Nd).

8

Subspaces can be compared using local sequences

Theta phase precessionSeveral sequential events are packed in each phase (~5 Hz)

( Sato and Yamaguchi : Neural Computation 2003)

Inspired by information representation in

hippocampus.

Local sequences are used to compare subspaces.

(Skipping a detailed explanation.)

C

AB

DEFZ

XY

Time

1 2 3 4 5 6 7

Time

1 2 3 4 5 6 7Combined

frame

Time: t

D

FE

G

A

H1 2 3 4 5 6 7

BC

Original frame

D

FE

G

A

H

Input sequence

1 2 3 4 5 6 7

BC

ESs Subspace of d variables

Set

of N

var

iabl

es

Outline

1. Human-level intelligence can explore from neocortex learning.

Artificial intelligence (AI) lacks flexible sampling function of neocortex Equivalence structure (ES) extraction is key for such a function

2. Use local sequences to extract equivalence structures (ESs). Inspired by theta phase precession of hippocampal formation

3. Simple simulation of ES extraction

4. Summary Frame generation:

Promising way to achieve artificial general intelligence (AGI)

9

10

Simple simulation to validate this idea

dim. ID 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

3 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1

4 1 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0

5 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0

6 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0

7 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0

8 0 0 0 0 0 0 0 0 1 0 1 0 1 0 0 0 0 0 0 0

Time

A swinging dot image in sequence of one-dimensional spaces,representing an idealized video image of natural scenes (up to 300 frames)

D

F

E

G

A

H

B

C

Input image for experiment: Dot wave sequence

Expected ES: A cluster of adjacent variable sets

300

Cluster of 12 subspaces, each of which consisting of 3 adjacent variables, is expected to be extracted depending on spatial continuity of input sequence.

C

A

B

E

F

G Z

X

Y

D

B

C

E

C

D

F

G

H

D

E

F

C

A

B

E

F

GD

B

C

E

C

D

F

G

H

D

E

F

Combined frame Cluster of subspaces

11

All

perm

utat

ion

of s

ubsp

ace

s (

366

patt

erns

)

Index of local sequences (Only non-zero elements shown)

Num

bers

of l

ocal

seq

uenc

es

Expected ES containing adjacent variable set is extracted as cluster from a numbers of local sequences clustering.

Combined frame

X Y Z

E D CB C DG F ED E FF E DE F GD C BC D EC B AA B CH G FF G H

Result: Expected ES is extracted as a cluster

Su

bsp

aces

Outline

1. Human-level intelligence can explore from neocortex learning.

Artificial intelligence (AI) lacks flexible sampling function of neocortex Equivalence structure (ES) extraction is key for such a function

2. Use local sequences to extract equivalence structures (ESs). Inspired by theta phase precession of hippocampal formation

3. Simple simulation of ES extraction

4. Summary Frame generation:

Promising way to achieve artificial general intelligence (AGI)

12

Summary

Untrodden machine intelligent functions are concentrated on neocortex, so emergence of HLDL mostly means emergence of HLAI.

Learning of sampling layer is minimally needed to generate high-level features for HLDL. This learning is assumed to be equivalence structure extraction.

13

(Buzsaki, 2007)

Where is responsible sub-region for ES extraction in theta loop of

hippocampal formation?

Inspired by theta phase precession, I introduced “numbers of local sequences” for each subspace. Clustering of subspaces by these frequencies enabled extraction of ESs in a simple demonstration.

I'd like to specify the sub-region of the hippocampal formation within theta loopsthat perform ES extraction.

Future work includes constructing a neocortex-hippocampus model implementing ES extraction.

Human-level general AI needs ability to generate frames.

14

123456

EA B C

Eve

nts

VariablesD

Values

General intelligence systems should be able to learn to solve problems that were unknown at time of their creation.

NeuronColumn

Combined new frame

Ⅰ

Ⅱ

Ⅲ

Ⅳ

Ⅴ

Ⅵ

Equivalence structure

(ES)

Obviously, human brain can generate new frames to solve various new problems using learning ability of neocortex.

Designing HLDL by referring to neocortex is a promising approach.

frame