Artificial general intelligence (AGI) building thinking machines © 2007 General Intelligence Research Group.

Artificial general intelligence(AGI)

“building thinking machines”

© 2007 General Intelligence Research Group

AGI vs “narrow” AI

• examples of narrow AI:– face recognition– spam filtering– data mining– Google

Common objections

• intelligence is not well-defined• it’s too hard• computing power is not there yet• no unifying theory of AI• we don’t understand the brain• etc…

All this is bull shit!

AI pioneers• Alan Turing (1912-1954)• John von Neumann (1903-1957)

John McCarthy (1927-) Marvin Minsky (1927-)

Implications of AGI

• complete automation• ethical issues• “Technological Singularity”

Vernor Vinge (1944-)Ray Kurzweil (1948-)

Representative AGI projects

Cyc

• most-funded AI project in history ($10s of millions)• based on predicate logic• complete ontology• millions of facts, concepts

Doug Lenat (1950-)

Soar• Allen Newell (1927-1992) John E Laird

• based on production rules & rete algorithm• learning – “chunking”

Novamente• Ben Goertzel (1966-)• probabilistic logic based on

“uncertain probabilities”• graph-based

knowledge representation• genetic algorithms

for learning• robot living in virtual reality• 2007 book:

Artificial General Intelligence

NARS• Non-Axiomatic Reasoning System• Pei Wang• can learn from experience• work with insufficient

knowledge and resources• unified cognition:

reasoning, learning,planning, etc…

• 2006 book:Rigid Flexibility

SNePS

• Semantic Network Processing System• Stuart C Shapiro• extends first-order logic• belief revision /

assumption-basedtruth maintenance

• natural languageunderstanding

AIXI

• Marcus Hutter• highly abstract• based on Kolmogorov

complexity theory• KC is incomputable• learning may take

forever!

Polyscheme

• Nick Cassimatis• integrates multiple methods

of representation, reasoning,and problem-solving

• procedural substrate• not “one model”

CAM-brain

• Hugo de Garis (1947-)• neural network• evolvable hardware• cellular automata• currently at

Wuhan University

SAIL• John Weng neural network-based• navigates and learns from environment

autonomously

Jeff Hawkins (1957-)

• inventor of “Palm Pilot”• founded Redwood

Neuroscience Institute• 2005 book:

On Intelligence• HTM (Hierarchical

Temporal Memory)• neurally-inspired

Brain-inspired

AI

visual cortex

Wiring of 6-layer cortex

Neurally-inspired AI

• feedforward neural network• Jeff Hawkins’ approach• problem:

invariant recognition:translation,rotation,scaling

Statistical learning

• takes place in a vector space• requires many examples• target = manifold• difficult to learn

concepts withvariableseg:On(apple,table),On(car,road), etc…

“Spatial” pattern recognition

ANN,SVM,PCA,Clustering,etc…

Logic-based vision

• visual features logical representation

Logical-vision exampleQuadrilateral() :-∃e1:edge

e∃ 2:edgee∃ 3:edgee∃ 4:edgev∃ 1:vertexv∃ 2:vertexv∃ 3:vertexv∃ 4:vertex

Connects(e1,v1,v2) ^Connects(e2,v2,v3) ^Connects(e3,v3,v4) ^Connects(e4,v4,v1)

“Syntactic” pattern recognition

predicate logic formula: featurei relation1(feature1, feature2, …) ^

relation2(feature3, feature4, …) ^…

Spatial interpretation?

Logic-based AI

Avoid reinventing the wheel!

Logic-based AI

• first-order predicate logic (Prolog)• common objections:

“brittle”“rigid”“binary” “not numerical”“just a theorem prover”

• probabilistic / fuzzy logic• non-deductive mechanisms

eg: abduction, induction

Modules

• perception (eg vision)• pattern recognition• inference• natural language• learning• truth maintenance• planning

Architecture

Pattern recognition• “neural characteristics” “soft computing”• Prolog:

chair(X) :- leg1, leg2, leg3, leg4, seat, back, horizontal(seat), vertical(back),...

leg1

chair

leg2 leg3 leg4 … ...

fuzzy values

Pattern recognition– “chairs”

more chairs

still more chairs

Pattern recognition

• how humans recognize “concepts”?• [Michalski 1989] “2-tiered approach”

rule-based vs instance-based• Prolog:

chair :- chair1

chair :- chair2

chair :- chair3

...chair :- (rule for general chair)

Probabilistic logic• classical resolution [JA Robinson 1965]• Bayesian networks [eg Judea Pearl]

Resolution algorithm

• try to resolve formulas repeatedly until no more can be resolved

P V Q ~P V R

Q V R

Bayesian network

• propositional

First-order Bayes net

BeltStatus(belt) RoomTemp(room)

EngineStatus(machine)

Bayesian vs classical logic

• Conditional Probability Table (CPT) classical

Bayesian (A ^ B)A B

C

A B CT T 1.0T F 0.0F T 0.0F F 0.0

A B CT T TT F FF T FF F F

KBMC• Knowledge-Based Model Construction• [Wellman et al 1992]• generate Bayesian networks “on-the-fly” to

answer specific queries

KB

KBMC example

KBMC example

Belief bases vs belief sets

• belief set = Cn( belief base )

set of consequences• belief sets are too large to manipulate• for AGI, must use belief base

Fuzzy logic• “John’s girl friend is probably very pretty”

• fuzziness probability• Lotfi Zadeh (1921-)

1965 fuzzy sets1973 fuzzy logic

Confidence

• Example:A. 10 girls, 5 have long hairB. 1000 girls, 500 have long hair

p = 0.5but A and B are not the sameB has higher confidence

• used in Pei Wang’s NARS logic

Probabilistic-fuzzy inference

( P, C, Z )n ( P, C, Z ) x1 x2 . . .

Ps and Zs can be point-valued or interval-valued

probabilityconfidencefuzziness

Probability intervals

• Example:marry fool [p = 0.8]! marry loser [p = 0.7]

p( fool V loser ) =0.7 + 0.1 * p( marry ) [ 0.7, 0.8 ]

unknown

Conditional probability table (CPT)

• All permutations of fuzzy values

• Or, store in a“distribution-free”format?

a b Cz1 … (P1, C1, Z1)

z2 … (P2, C2, Z2)

z3 … (P3, C3, Z3)

z4 … (P4, C4, Z4)

… … …

“Rules of thought”

• “If cats have claws,and Juney is a cat,then Juney has claws.”

• P,x,y P(x) ^ isa(y,x) P(y)• modus ponens: • syllogisms

Q Q P P,

reasoning

deduction retroduction

induction abduction

Abduction

• “finding explanations”• eg glass is wet it was raining• algorithm:

reverse of deduction (eg resolution)• very high complexity

(within the arithmetical complexity class )0 2

Abduction algorithm

Induction vs abduction

• abduction: answer = ground literalseg “grass is wet” “it was raining”

• induction: answer = general formulaeeg daughter(X,Y) :- father(Y,X) ^ female(Y)

Induction• learning general patterns statistically• ILP (Inductive Logic Programming)[Stephen Muggleton]1990s

Induction example

Given data:

male(mary) = falsefemale(mary) = truemother(mary, louise) = truefather(mary, bob) = truedaughter(bob, mary) = true

daughter(X,Y) :- father(Y,X) ^ female(Y)

Natural language

• unifying framework• language = knowledge-based inference• [Jerry R Hobbs] “Abduction as Interpretation”

eg “The Boston office called.”• “apple pie” “door knob” “street hawker”

• all we need is a lot of rules• can inductively learn the rules

Belief maintenance

• Truth Maintenance System (TMS)• belief revision• to attain “consistency”• avoid “cognitive dissonance”

Truth maintenance

justifications

Belief revision

• “Epistemic entrenchment” Belief Base• [Mary-Anne Williams

1995]

entrenchmentranking

“Click” feeling

Perhaps an effect of successful inference, abduction, or belief revision?

Paraconsistency

• holding 2 contradictory beliefs in the knowledge base at the same time

Associative memory

• knowledge base = database• special indexing to allow associative recall• hard disk = long-term memory• RAM = working memory

Planning

Conclusions

• “neural” is problematic• “blank slate” is problematic• “logic-based” is very promising

Agendafor Logic-based AI

1. design probabilistic-fuzzy logic2. develop algorithms for:– abduction–belief maintenance

3. acquire common sense knowledge

“Web 2.0”-style collaboration• branching• voting• commercial• problem: too few members

Thank you

• [Aliseda 2006] Abductive Reasoning: Logical Investigations into Discovery and Explanation. Synthese Library Series vol 330, Springer

• [Antoniou 1997] Nonmonotonic Reasoning, MIT Press• [Cussens 2001] Integrating probabilistic and logical reasoning. In David

Corfield and Jon Williamson eds Foundations of Bayesianism, volume 24 of Applied Logic Series, pages 241-260. Kluwer, Dordrecht

• [2000 Flach & Kakas eds] Induction and Abduction, Springer Applied Logic Series #18

• [Haddawy 1994] Generating Bayesian networks from probability logic knowledge, in Proceedings of the 10th conference on uncertainty in AI, 1994.

• [Hobbs 200?] Abduction as Interpretation• [Jaeger 1997] Relational Bayesian networks. In Proceedings of the 13th Annual

Conference on Uncertainty in AI (UAI-97), p266-273, San Francisco, CA, 1997, Morgan Kaufman Publishers

• [Kakas, Kowalski, Toni 1992] Abductive Logic Programming, Journal of Logic and Computation 2(6):719-770. http://citeseer.ist.psu.edu/kakas93abductive.html

• [Laskey 2006] MEBN: A logic for open-world probabilistic reasoning. GMU C4I Center Technical Report C4I-06-01. George Mason Univ, USA.

• [Milch & Russell 2007] First-Order Probabilistic Languages: Into the Unknown In ILP: Proceedings of the 16th International Conference on Inductive Logic Programming. Berlin: Springer

• [Michalski 1989] Two-tiered concept meaning, inferential matching, and conceptual cohesiveness. In Vosniadou & Ortony eds, Similarity and analogical reasoning, p122-145. Cambridge University Press, New York.

• [Muggleton 1996] Stochastic logic programs. In de Raedt, ed, Advances in Inductive Logic Programming, p254-264, IOS Press 1996.

• [Ngo, Haddawy, & Helwig 1995] A theoretical framework for context-sensitive temporal probability model construction with application to plan projection. In Proceedings of the 11th Annual Conference on Uncertainty in Artificial Intelligence (UAI-95), p419-426, Montreal, Quebec, Canada.

• [Norvig & Russell 2003] Artificial Intelligence: A Modern Approach, Prentice Hall.

• [Poole 1993] Probabilistic horn abduction and Bayesian networks, Artificial Intelligence, 64(1), 81-129, 1993

• [Poole 2003] First-order probabilistic inference, Proc, IJCAI-03, Acapulco, August 2003, p985-991

• [Wellman, Breese, Goldman 1992] From knowledge bases to decision models. Knowledge Engineering Review 7(1): 35-52

• [Williams 1995] Changing nonmonotonic reasoning inference relations, in Proceedings of the second world conference on the fundamentals of AI, 469-482, Ankgor, Paris, 1995