History of Artificial Intelligence

1

History of Artificial Intelligence

Dana Nejedlová

Department of Informatics

Faculty of Economics

Technical University of Liberec

2

What is Intelligence?

• Common definition of artificial intelligence:– AI is a field which attempts to build intelligent

machines and tries to understand intelligent entities.

• But what is intelligence?– Learning, manipulating with facts, but also creativity,

consciousness, emotion and intuition.

• Can machines be intelligent?– Up to the present day it is not sure whether it is

possible to build a machine that has all aspects of intelligence.

– This kind of research is central in the field of AI.

3

What Is Artificial Intelligence?• Building machines that are able of symbolic processing,

recognition, learning, and other forms of inference• Solving problems that must use heuristic search instead

of analytic approach• Using inexact, missing, or poorly defined information

– Finding representational formalisms to compensate this

• Reasoning about significant qualitative features of a situation

• Working with syntax and semantics• Finding answers that are neither exact nor optimal but in

some sense „sufficient“• The use of large amounts of domain-specific knowledge• The use of meta-level knowledge (knowledge about

knowledge) to effect more sophisticated control of problem solving strategies

4

Before the Creation of Electronic Computers

• Ancient and medieval myths– Talos, Pandora, Golem

• artificial men, robots, automatons

• Research in the antiquity till the 17th century– Aristotle, Gottfried Wilhelm Leibniz

• automation of reasoning

– Thomas Hobbes, René Descartes• mechanistic understanding of living beings

• 20th century, 1948– Norbert Wiener – Cybernetics: Or the Control and

Communication in the Animal and the Machine.• Intelligent behavior is the result of the feedback mechanism.

5

The Beginnings of Electronic Computers

• John Louis von Neumann (1903 – 1957)– Von Neumann’s architecture of a computer

• Consultations on the EDVAC Project (1945)– Game Theory (1944)

• It can be applied to the interacting intelligent agents.– Cellular automata (1966)

• They have computational capacity.

• Alan Mathison Turing (1912 – 1954)– Turing Machine (1936)

• formalization of algorithm, abstraction of computer– Turing Test (1950)

• proposal how to test the ability of a machine to demonstrate thinking

– Programming of “Manchester Mark I” computer (1949)

6

The birth of “Artificial Intelligence”• John McCarthy used the term “Artificial

Intelligence” for the first time as the topic of the Dartmouth conference in 1956.– Venue:

• Dartmouth College, Hanover, state New Hamphshire, USA

– Organizers:• John McCarthy, Marvin Minsky, Nathaniel Rochester, and

Claude Shannon

– Participants:• Ray Solomonoff, Oliver Selfridge, Trenchard More, Arthur

Samuel, Herbert Simon, and Allen Newell

– Proposal:• To prove that every aspect of learning or any other feature of

intelligence can be so precisely described that a machine can be made to simulate it.

7

Approaches to Artificial Intelligence• Good Old-fashioned Artificial Intelligence (GOFAI) or

symbolic artificial intelligence (John Haugeland, 1985)– Program (e.g. classifier) in the GOFAI style is composed of parts

(e.g. rules), that have clear relation to the real world.

• New-fangled Artificial Intelligence– The most important branch was connectionism – artificial neural

networks (McCulloch – Pitts, 1943).– Genetic algorithms (Holland, 1975) and other kinds of

biologically inspired information processing

• Strong AI (John Searle, 1980)– Artificial intelligence is real intelligence.– Solution of complex problems, e.g. robotics.

• Weak AI– Artificial intelligence is a mere imitation of human real

intelligence.– Solution of a specific problems that do not cover the whole scale

of human capabilities, e.g. OCR or chess.

8

Motivations for Biologically Inspired Information Processing

• Danny Hillis: The Connection Machine (1985)– Machines programmed in a GOFAI style tend to slow down as

they acquire more knowledge.• They must search their knowledge base.

– Humans have the opposite property.• They have massively parallel brain architecture.

– Humans were not produced by an engineering process.• They are the result of evolution.

• Marvin Minsky: The Society of Mind (1986)– Model of human intelligence which is built from the interactions

of simple parts called agents which are themselves mindless.• It would be difficult to imagine how evolution could shape a single

system as complex as mind.• Evolution could, however, shape individual specialized cognitive

units and form the mechanisms that enable the modules to interact.

• Marvin Minsky: The Emotion Machine (2006)– Emotions are different ways to think that our mind uses to

increase our intelligence.

9

Artificial Intelligence Philosophy• What is intelligence and thinking?

– Turing test (1950)– According to GOFAI thinking is symbol manipulation,

that is why program in the GOFAI style is thinking.– Chinese Room Problem (John Searle, 1980)

• Thinking of humans and computers is different.

• Is human intelligence inseparable from mind and emotions?

• In what sense can we say that a computer can understand natural language?

• Who is responsible for the decisions made by AI?

• What should be the ethics of people of dealing with the creations of artificial intelligence?

10

Hard Versus Soft Computing

• Good Old-fashioned Artificial Intelligence– IF – THEN Rules– Heuristics

• New-fangled Artificial Intelligence– Neural networks– Fuzzy logic– Probabilistic reasoning

• belief networks (Bayes networks)• genetic algorithms• chaos theory• parts of learning theory (machine learning)

11

Heuristics

• Problem-solving method that is usually successful, but can fail i some situations

• Unclearly defined problems with missing or ambiguous data– Medical diagnosis– Vision, speech recognition

• Helps to decide among infinite number of possible interpretations.

• A problem may have an exact solution, but the computational cost of finding it may be prohibitive.– Chess, tic-tac-toe, 15 or 8-puzzle, scheduling, path-finding…– Heuristic evaluation function

• Evaluates each stage of solution.– Number of conflicts in a number of possible schedules

• Helps to decide about the next step leading to the goal.– Selecting the schedule with minimum number of conflicts for the next

small changes attempting to find some correct schedule

12

Expectations from Artificial Intelligence

• Predictions of Herbert Simon and Allen Newell (Heuristic Problem Solving, 1958), that within ten years– a digital computer will be the world's chess

champion,– a digital computer will discover and prove an

important new mathematical theorem,– a digital computer will compose critically

acclaimed music,– most theories in psychology will take the form

of computer programs.

13

Typical AI Problem

• 8 Queens Puzzle• Is there a way of

placing 8 queens on the chessboard so that no two queens would be able to attack each other?

14

Hard Problem for AI

• Truncated Chessboard Problem

• Is there a way of placing dominos on the board so that each square is covered and each domino covers exactly two squares?

15

Limitations of Artificial Intelligence• David Hilbert (1862 – 1943) and Kurt Gödel (1906 –

1978)– Gödel‘s Incompleteness Theorem (1931)

• Consistency of a formal system cannot be proved within the system, because it can contain statements with self-reference – logical paradoxes of the type:

– This statement is false.

– Some tasks have no algorithms.• The halting problem

– It is not decidable whether the algorithm will halt or not.– The algorithms in question contain again self-reference.

• Complexity Theory (NP-completeness, 1971)– Some tasks have algorithms, but the computation cannot be

completed in practice (on a real computer), because it would take too much time.

• Roger Penrose (books The Emperor‘s New Mind, Shadows of the Mind)– It may not be possible to completely simulate biological

intelligence by computational approaches as it may be based on (apparently quantum) phenomena that we do not know and are not able to imitate.

16

Gödel‘s Incompleteness Theorem• There are unprovable statements in every axiomatic

mathematical system expressive enough to define the set of natural numbers.

• Example theorem 1 = 2• Proof of the theorem:

• If a = b, a ≠ 0, b ≠ 0,• then the two following equalities are also true:a2 – b2 = (a – b) ∙ (a + b),a2 – b2 = a2 – ab.• And the following statements can be derived from them: a2 – ab = (a – b) ∙ (a + b) a ∙ (a – b) = (a – b) ∙ (a + b) a = a + b a = a + a a = 2a 1 = 2

• Truth can be verified only when knowledge beyond the natural finite numbers arithmetic is used.

17

The Logic Theorist – The First Artificial Intelligence Program

• Allen Newell, J.C. Shaw and Herbert Simon at Carnegie Institute of Technology, now Carnegie Mellon University, in 1955

• It did logic proofs from the book “Principia Mathematica” (Bertrand Russell and Alfred North Whitehead, 1910).

• It used mental processes of human experts.– cognitive science

• To implement Logic Theorist on a computer, the three researchers developed a programming language, IPL, a predecessor of Lisp.

18

Programming Languages• Tasks like natural language processing,

knowledge representation, or theorem proving needed a special language allowing processing of symbolic data.

• Lisp (John McCarthy, USA, 1958)– functional paradigm / list processing

• Program consists of functions of nested functions.• Data and programs are represented the same way: a list.

– (+ 1 2 3) is a both a list of 4 atoms and a function returning value 6.

• Program can serve as data for another program!– Powerful feature allowing flexible and productive coding.

• Prolog (Alain Colmerauer, Europe, 1972)– declarative paradigm / logic programming

• Program consists of facts and rules.– Programmer describes (i.e. declares) a problem.

• Compiler deduces new facts from them.– Programmer does not write the algorithm for the solution.

19

Programs with Symbolic Artificial Intelligence

• The General Problem Solver (1957)– It was solving formalized symbolic problems, e.g.

mathematical proofs and chess.• The Geometry Theorem Prover (1958)

– It was proving theorems with the help of explicitly represented axioms.

• SAINT (Symbolic Automatic INTegrator)– Integral calculus (1961)

• ANALOGY (1963)– The picture A is to picture B like picture C to picture

D.• IQ tests are used for measuring the intelligence of people.• Computers can be programmed to excel in IQ tests.• But those programs would be stupid in real-world situations.

20

Natural Language Processing• STUDENT (1964, 1967)

– It was solving word problems in algebra.

• SIR (Semantic Information Retrieval, 1968)– It was reading simple sentences and answered

questions.

• ELIZA (1965)– It was simulating psychologist.

• TLC (Teachable Language Comprehender) (1969)– It was reading text and making semantic network.

• SUR (Speech Understanding Research) (1971)– 5-year plan of the ARPA (today DARPA) agency of a

research in continuous speech recognition

21

Expert Systems• They belong to the symbolic AI.

• They use a set of rules and heuristics.

• MACSYMA (MIT, 1968 -1982)– It was doing symbolic math calculations.

• DENDRAL (SRI, 1965)– It is identifying chemicals.

• MYCIN (SRI, Edward Shortliffe, 1974)– It diagnosed infectious blood diseases.– The following systems: EMYCIN, PUFF,

INTERNIST - CADUCEUS

22

Commercial Expert Systems

• PROSPECTOR (SRI, 1974 – 1983)– It is analyzing geological data and searching

for deposits of minerals.

• XCON – eXpert CONfigurer (CMU, 1978)– It was configuring DEC’s VAX computers.

• TEIRESIAS (SRI, Randall Davis, 1976)– Knowledge Acquisition System (KAS)– It is acquiring knowledge from human experts.– It is building knowledge bases for expert

systems.

23

Robotics

• Marvin Lee Minsky (* 1927)

• Freddy (University of Edinburgh,1973)

• SHAKEY (SRI, 1969)

• SHRDLU (MIT, Terry Winograd, 1970)

• blocks worlds (MIT, 1970)– Robot has to manipulate building blocks

according to instructions.• computer vision• natural language understanding• planning

24

The First Artificial Neural Networks

• Warren McCulloch and Walter Pitts– Model of artificial neuron (1943)– Neuron represents functions.

• Donald Olding Hebb– Rule for neural network training (1949)

• Marvin Minsky and Dean Edmonds have built the first computer with neural network.– SNARC (1951)

25

Other Artificial Neural Networks• Frank Rosenblatt

– Perceptron (1957)• a single-layer network and its learning rule capable

of learning linearly separable functions

• Bernard Widrow and Marcian Ted Hoff– Minimization of network’s root square error– Delta rule (learning rule of a neural network)– ADAptive LINEar Systems or neurons or

ADALINEs (1960)– MADALINEs (1962)

• multi-layer versions of ADALINEs

26

Neural Networks Critique• Book „Perceptrons“ (Marvin Minsky and

Seymour Papert, 1969)– When single-layer neural networks of a

Perceptron type cannot learn XOR function (it is linearly inseparable), also multi-layer networks cannot learn it.

– Hence funding of neural network research was stopped until the beginning of the 20th century 80’s.

• But multi-layer neural networks can learn the XOR function.

• All that is needed for this is to find the right algorithm for their training.

27

Neural Networks Resurrection

• Hopfield net (John Hopfield, 1982)– It can learn a couple of pictures (patterns).

• Self-Organizing Map (SOM) (Teuvo Kohonen, 1982)– It can do unsupervised learning.

• Backpropagation (Arthur Bryson and Yu-Chi Ho, 1969)– algorithm for training of a multilayer neural network– It needs network’s neurons not to have a sharp threshold.– Because it was not noticed, it was then rediscovered several

times in the 70’s and the 80’s of the 20th century and popularized in 1986.

• NETtalk (Terry Sejnowski and Charles Rosenberg, 1986)– Multi-layer neural network, that learned English pronunciation

and could generalize.– It used the backpropagation algorithm.

28

The Most Important AI Laboratories• MIT (Massachusetts Institute of

Technology)– 1959 - John McCarthy and Marvin Minsky

founded Artificial Intelligence Laboratory.

• SRI (Stanford Research Institute)– 1963 - John McCarthy founded AI Laboratory.

• CMU (Carnegie Mellon University)– 1980 - Raj Reddy founded The Robotics

Institute.

• IBM• AT&T Bell Labs• University of Edinburgh

29

Present• Robotic toys, space probes• Robotics in machinery• Home appliances (washers, vacuum cleaners)• Data Mining, fraud detection, spam filtering• Searching on the Internet (web agents)• Modeling of interactive processes (agents)• E-business – e-shops personalization• Intelligent tutoring systems and SW interfaces• Role-playing games, chess programs• Speech and video recognition• Machine translation