A brief introduction of Artificial Intelligence Method and Fuzzy Logic Control S. C. Chen 2009/07/17.

A brief introduction of Artificial Intelligence Method and Fuzzy Logic Control

S. C. Chen

2009/07/17

Outline

Artificial Intelligence

Artificial Neural Network

Data Mining

Genetic Algorithm

Fuzzy Logic Control

Artificial intelligence

Thinking Machines

Agent

Robot


AI (artificial intelligence) is a combination of computer science, physiology, and philosophy.

AI is a broad topic, consisting of different fields, from machine vision to expert systems. The element that the fields of AI have in common is the creation of machines that can "think".

In order to classify machines can thinking, it is necessary to define intelligence for the machines.


What does the degree of intelligence of machines to consist of, for example, solving complex problems, or making generalizations and relationships? And what about perception and comprehension? AI is also call machine intelligence that means people who create a system to solve complex problems by intelligence.Research into the areas of learning, of language, and of sensory perception have aided scientists in building intelligent machines.

Artificial Intelligence---historycentury 1940~ 1950~ 1960~ 1970~ 1980~ 1990~

computer 1945 computer (ENIAC)

1957 Fortran

Research of AI 1953 David Chess

1956 Dartmouth

1977Universal Declaration of Knowledge Engineering

1982

Fifth generation computer

1991

Neural

computer

Language of AI 1960

LISP

1973

PROLOG

Knowledge representation

1973

Production

system

1976

Theoretical framework

Expert System 1965

DENDRAL

1975

MYCIN

Artificial Intelligence---research areas

Natural Language Processing

Knowledge of performance

Intelligent Search

Reasoning

Perception problem

Pattern recognition

The management of imprecise and uncertain

Knowledge acquisition

Artificial Intelligence---research areas

Data mining

ANN (Artificial Neural Network)

Fuzzy Theory

Genetic Algorithm

Machine Learning

Artificial Neural Network---Introduction

ANN (Artificial Neural Network) is also known as:Parallel distributed processors

Adaptive systems

Self-organizing systems

Connectionism

Neurocomputers

NN (neural networks)

Artificial Neural Network---Introduction

The definition of ANN:ANN is a kind of computing system that is created by hardware or software. ANN used a lot of artificial neuron to simulate the ability of an organism neuron. Artificial neuron is a simple simulation of an organism neuron that gathered input data from external environment or other artificial neuron. Afterward, to obtain the result data by the procedure of complex computing, than to output the result to external environment or next artificial neuron.

Artificial Neural Network---Neuron

a Inputs

W Weight

SUM Summation

f Activation function

t Output

http://zh.wikipedia.org/w/index.php?title=File:Ncell.png&variant=zh-cn

Artificial Neural Network---Neuron to Neural Network

Different problems have different combination method of network.

Using usage samples to train the network, and than changing the weight of network foot by foot. Finally, making the value of output Y to close to our purpose value.

Artificial Neural Network---Neural Network Learning

Supervised Learning NetworkPrediction, identify, classify

Unsupervised Learning NetworkClustering

Hybrid Learning Network

Associate Learning NetworkData acquisition, Noise filter

Optimization Application NetworkDesign, Scheduling

Artificial Neural Network---Supervised Learning Network

According to the field of problem, providing the sample include input and output data for training.

The network learning from input and output data to adjust the weights of hidden layer to adaptive input and output.

This module just like mother to watch over children for learning.

Back-propagation neural network is most representative in the supervised learning network.

Artificial Neural Network---Back-propagation neural network

The module of perceptron was be proposed in 1957. This module is lack for hidden layer of neural network. So, the learning ability of NN is much more restricted.

In 1985, P. Werbos, D. Parker, G. E. Hinton proposed the learning algorithm of hidden layer. The proposed learning algorithm makes BP-NN to enter the new generation.



Examples:Traveling Salesman Problem

3D Kohonen Feature Map

http://www.patol.com/java/TSP/index.html

http://www.patol.com/java/TSP/index.html

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-sample.html

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-sample.html

Data Mining---Introduction

Data explosion problemData explosion problem Automated data collection

tools and mature database

technology lead to

tremendous amounts of data

stored in databasesdatabases, data data

warehouseswarehouses and other other

information repositoriesinformation repositories

We are drowning in datadrowning in data,

but starving for knowledgestarving for knowledge!

SolutionSolution: Data warehousing and data mining

Data warehousing and on-line analytical processing

Extraction of interesting knowledgeknowledge (rulesrules, regularitiesregularities, patterns patterns,

constraintsconstraints) from data in large databases

Data Mining---Introduction

Data Mining ---Introduction, history of database

1960s:

Data collection, database creation, IMS (IP Multimedia Subsystem ) and network DBMS (Database Manage System)

1970s: Hierarchical and network database systemsRelational data model, relational DBMS implementation

1980s: RDBMS, advanced data models (extended-relational, OO, deductive, etc.) Application-oriented DBMS (spatial, temporal, engineering, etc.)

1990s: Data mining, data warehousing, multimedia databases, and Web databases

2000sStream data management and miningData mining and its applicationsWeb technology (XML, data integration) and global information systems

Data Mining ---Introduction

Data mining (knowledge discovery fromknowledge discovery from data data) Extraction of interestinginteresting (non-trivialnon-trivial, implicitimplicit, previously unknownpreviously unknown and potentially usefulpotentially useful) patternspatterns or knowledgeknowledge from huge amount huge amount of dataof data

Data mining: a misnomer?Data mining is not only just data ， but it can finding knowledge!!

Alternative namesKnowledge discoveryKnowledge discovery (mining) in databases (KDD), knowledgknowledge extractione extraction, business intelligencebusiness intelligence, data/pattern analysis, data archeology, data dredging, information harvesting, etc.

Many people treat data mining as a synonymsynonym for another popularly used term, Knowledge Knowledge Discovery from DataDiscovery from Data (KDD) — GeneralizedGeneralized Data Data

miningmining

Alternatively, other view data mining as simply an essential stepan essential step in the process of knowledge discovery — NarrowlyNarrowly Data miningData mining

Data Mining ---Introduction

Data Mining ---Knowledge Discovery (KDD) Process

Data mining—core of knowledge discovery process

Data Cleaning and Data Integration

Databases

Data Warehouse

Task-relevant Data

Selection and Transformation

Data Mining

Evaluation and Presentation

Patterns

Data Mining---KDD Process: Several Key Steps

1.1. Data cleaning Data cleaning Remove noisenoise and inconsistent datainconsistent datamay take 60% of effort!

2.2. Data integration Data integration Where multiple data source may be combined

3.3. Data selection Data selection Where data relevant to the analysis taskrelevant to the analysis task are retrieved from the DB

4.4. Data transformation Data transformation Where data are transformed or consolidated into forms appropriate for mining by performing summarysummary or aggregationaggregation operations, for instancefor instance.

5.5. Data mining Data mining Intelligent methods are applied in order to extract data patterns.Choosing the mining algorithm(s)Choosing the mining algorithm(s) for searching patterns of interest

6.6. Pattern evaluation Pattern evaluation To identify the truly interesting patternsidentify the truly interesting patterns representing knowledge based on some interestingness measures.

7.7. Knowledge presentation Knowledge presentation Where visualization and knowledge representation techniques are used to presepresent the mined knowledge to the usernt the mined knowledge to the user.

We adopt a broad view of data mining functionality:

Data mining is the process of discovering interesting the process of discovering interesting

knowledgeknowledge from large amounts of datalarge amounts of data stored in

databases, data warehouses, or other information

repositories.

Data Mining

Data Mining--- Typical Data Mining System

data cleaning, integration, and selection

Database or Data Warehouse Server

Data Mining Engine

Pattern Evaluation

Graphical User Interface

Knowledge-Base

DatabaseData

WarehouseWorld-Wide

WebOther Info

Repositories

OLAP: OLAP: On line analytical ProcessingOn line analytical Processing

Data Mining and Business Intelligence

Increasing potentialto supportbusiness decisions

End User

Business Analyst

DataAnalyst

DBA

Decision

MakingData Presentation

Visualization Techniques

Data MiningInformation Discovery

Data ExplorationStatistical Summary, Querying, and Reporting

Data Preprocessing/Integration, Data Warehouses

Data SourcesPaper, Files, Web documents, Scientific experiments, Database Systems

Data Mining---What Kind of Data?

Relational databases

Data warehouses

Transactional databases

Advanced DB and information repositoriesObject-oriented and object-relational databases

Spatial and Spatiotemporal Databases

Temporal, Sequence, and Time-Series Databases

Text databases and multimedia databases

Heterogeneous and legacy databases

Data Streams

WWW

Relational databases

Data warehouses

Transactional databases

Data Mining---What kinds of patterns can be mined?

Data mining functionalities are used to specify the kinds of patterns the kinds of patterns to be foundto be found in data mining tasks.

Data mining tasks can be classified into two categories:

Descriptive Descriptive :

Characterize the general propertiesCharacterize the general properties of the data in the database.

Predictive Predictive :

Perform inferenceinference on the current data in order to make predictions.

In some cases, users may have no idea regarding what kinds of patterns in their data may be interesting interesting, and hence may kind to search for several different kinds of patterns in parallelin parallel.

Thus it is important to have a data mining system that can mine multiple kinds of patternsmultiple kinds of patterns to accommodate different user expectations or applications.

Data mining functionalities, and the kinds of

patterns they can discover, are described below:Concept descriptionConcept description: Characterization and discrimination

Association AnalysisAssociation Analysis

ClassificationClassification and PredictionPrediction

ClusterCluster analysisanalysis

Outlier analysisOutlier analysis

Trend Trend and evolution analysisevolution analysis

Data Mining---What kinds of patterns can be mined?

Data Mining---Association analysis

From transactional databases, RDBMS or other large amounts of data item of storage systems. Finding the interested pattern and frequent pattern to analysis the associationsassociations and the correlationscorrelations between each data item.

this associations doesn’t present directly in data

The best example is to ensure association rules.

Data Mining--- Data Cube Aggregation

Data cubes store multidimensional aggregated inmultidimensional aggregated in

formationformation.

For example:

Data Mining ---Potential Applications

Data analysis and decision supportData analysis and decision support

Market analysis and management

Target marketing, customer relationship management (CRM), market

basket analysis, cross selling, market segmentation

Risk analysis and management

Forecasting, customer retention, improved underwriting, quality

control, competitive analysis

Fraud detection and detection of unusual patterns (outliers)

Other ApplicationsOther Applications

Text mining (news group, email, documents)

Web mining

Bioinformatics and bio-data analysis

Genetic Algorithm---Introduction

Genetic algorithms are a part of evolutionary computing, which is a rapidly growing area of artificial intelligence. Based on Darwinian principles of biological evolution.

First proposed by Prof. John Holland in 1975, and his colleague at Univ. of Michigan.

Genetic Algorithm---Introduction

Chromosomes are strings of DNA and serves as a model for the whole organism.

A chromosome consists of genes.

Each gene encodes a trait.

Complete set of genetic material (all chromosomes) is called genome.

Particular set of genes in genome is called genotype.

Genetic Algorithm ---Encoding

The chromosome should in some way contain information about solution which it represents.

The most used way of encoding is a binary string.

Chromosome 1 1101100100110110Chromosome 2 1101111000011110

Each bit in this string can represent some characteristic of the solution. One can encode directly integer or real numbers.

Genetic Algorithm---Procedure

The initial population generate randomly

Evaluating the fitness function

Is reach the condition?Optimal solution

YES

Reproduction

Crossover

Mutation

NO

Genetic Algorithm---Fitness function

Algorithm is started with a set of solutions (represented by chromosomes) called population. Solutions from one population are taken and used to form a new population. The new population (offspring) will be better than the old one (parent).Solutions which are selected to form new solutions are selected according to their fitness - the more suitable they are the more chances they have to reproduce.

Genetic Algorithm---Reproduction

Reproduction is a kind of computing process that used to decide population elimination by the fitness degree. Fitness degree is calculate by fitness function.

Selection methods:Roulette Wheel Selection

Competitive Selection

Genetic Algorithm ---Crossover

Crossover selects genes from parent chromosomes and creates a new offspring.

Chromosome 111011 | 00100110110Chromosome 211011 | 11000011110Offspring 1 11011 | 11000011110Offspring 2 11011 | 00100110110 “ | “ is the crossover point

Genetic Algorithm ---Crossover

Single point crossover: 11001011+11011111 = 11001111

Two point crossover: 11001011 + 11011111 = 11011111

Uniform crossover: 11001011 + 11011101 = 11011111

Arithmetic crossover: 11001011 + 11011111 = 11001001

Genetic Algorithm---Mutation

Prevent falling all solutions in population into a local optimum of solved problem

Mutation changes randomly the new offspring.

Original offspring 11101111000011110Mutated offspring 11100111000011110Original offspring 21101100100110110Mutated offspring 21101101100110110

Genetic Algorithm---Roulette Wheel Selection

Parents are selected according to their fitness.

The better the chromosomes are, the more chances to be selected they have.

Fuzzy logic control

Introduction

Fuzzify

Rule-based

Inference

Defuzzification

Fuzzy logic control---Part1---Introduction

1965 Fuzzy Set (Prof. Lotfi A.Zadeh,UCB)

1966 Fuzzy logic (Dr. Peter N.Marinos, Bell Lab)

1972 Fuzzy Measure

(Prof.Michio Sugeno)Fuzzy Set

Fuzzy Event

CrispElement

Fuzzy logic control ---Introduction

Knowledge Representation

example: age (Man Old)

traditional

Age (Man Gt 60)

30 　　 60 　　　 Ages

1

Membership Function

Fuzzy logic control ---Introduction

Fuzzy

Age (Man Old)

30 　　 60 　　　 Ages

1

Membership Function

0.5

Fuzzy Logic

AsubsetfuzzytheinxelementtheofmembershipxA :)(Esetreferencetheofelementanx :

EofsubsetFuzzyBA ,,

]1,0[,),(),( baxbxa BA

),( baMINba

),( baMAXba

aa 1

)()( bababa

Fuzzy Logic Control---Introduction

abbaityCommutativ )()(

aa )(

stheoremsDeMorgan '

baba )(

)()( cbacbaeAssociativ

)()()( cabacbavityDistributi

baba )(


二值理論推論形式（事實）　麻雀是鳥（規則）　鳥會飛（結論）　麻雀會飛AI Language as LISP,Prolog “Pattern Matching”Fuzzy 推論形式：（事實）　這番茄很紅（規則）　蕃茄若是紅了就熟了（結論）　這蕃茄很熟了


(facts) X is

(rule) if X is A then Y is B

希望得到的結論是(result) Y is B

A

1

0

1

0

A

B

A B

Mamdani 法

Fuzzy Logic Control---Part2 ---Architecture

Fuzzy Logic Control---Crisp set

Set A, whose elements are x1,…. xn , is always written as A={x1,…,xn}

A={x | P(x) } ,x has the property P

Set A is defined by its characteristic function,

AxforAxforA x 1

0)(

A

An example set A={1,2,3,4,5}, U={0,1,2,3,4,5,6,7,8,9}. z(x) isa characteristic function of set A. z(1)=1, z(2)=1, z(3)=1, z(4)=1, z(5)=1, z(6)=0, z(0)=0z(7)=0, z(9)=0

Membership function denoted as ,

There are three stand type of membership

functions, triangle, trapezoid and Gaussian.

A fuzzy set F in X may be represented as

When X is continuous ：

When X is discrete ：

Fuzzy Logic Control---Definition

A 10 A

XxxxF F |))(,( xxF

X

F /)(

xxFX

F /)(

Fuzzy Logic Control---Basic characteristics of fuzzy sets

1. Vertical dimension: height, normalization (maximum form: at least one element with 1.0 membership and one element with membership 0).

2. Horizontal dimension: universal discourse, support sets, alpha cuts.

3. Representation schemes: membership function, ordered set of pairs, polynomial-like (integral-like).

Fuzzy Logic Control---BASIC CONCEPTS

A ofset level thein allfor ,)A(

by defined is function membership whoseN on defined number)(fuzzy set fuzzy a :ycardinalitfuzzy .7

).(A:ycardinalitscalar 6.

.continuous piecewise is function membership whose Ron definedset fuzzy normalized andconvex a :numberfuzzy .5

}.)(|{A :cut-4.

grade. membership possible maximum theattains elements its of oneleast at :setfuzzy Normalized3.

set. that in elementany by attained grade membershiplargest the:Height2.

0}.(x)μ|X{x A supp :Support1.

A~

A

Xx

A

A

x

xXx

).f(x/)f(x/)f(x/

)///f(f(A)

,///A

:principle extension13.

)](),(min[)( :onintersecti12.

)](),(max[)( :union.11

).(1)( :complement10.

B.A )()( :eequivalenc9.

subset.proper B.A )()( :subset.8

n2211

2211

2211

BABA

BABA

AA

BA

BA

n

nn

nn

xxx

xxx

xxx

xxx

xx

xx

xx

─

─

Fuzzy Logic Control---Types of Membership Functions

1. Linear type: linear/step functions. Approximating an unknown or poorly understood concept that is not a fuzzy number. (often expressed as shouldered sets)

2. Triangular type:

often used to model process control systems.


when you decompose a variable into fuzzy sets, the amount of overlap must vary between 10% to 50%.In modeling dynamic systems, it can approximate their behaviors to nearly any degree of precision.

cx

cxbbcxc

bxaabax

ax

cbaxTri

0

)/()(

)/()(

0

),,;(


3. Trapezoidal Type:

dx

dxccdxd

cxb

bxaabax

ax

dcbaxTra

0

)/()(

1

)/()(

0

),,,;(


4. Sigmoid/Logistic Type:

* modeling population dynamics where the sampling of individual values approximates a continuous random variable.

* frequency (proportional) representation: usually, most, always.

cx

cxbaccx

bxaacax

ax

cbaxS

1

))/()((21

)/()(2

0

),,;( 2

2


1. Fuzzy numbers and around representation: around, close to, few, some.

2. PI, Beta, Gaussian fuzzy sets (bell-shaped): the slope and width of the bell curves indicate the degree of compactness associated with the fuzzy number.

3. PI curves: is not asymptotic. Zero point is at a discrete and specified point.

xbcbccxS

xcbcbcxScbx

),2/,;(1

),2/,;(),;(


4. Beta curves: more tightly compacted than PI.

5. Similar to beta curves, but the slope goes to zero very quickly with a very short tail.

6. Irregularly shaped and arbitrary fuzzy sets: (domain value, membership) pairs linear interpolation.

2)(

12

),;(

))/)((1(),;(xckeckxG

bcxbcxB

Fuzzy Logic Control---Concept of membership function

μ

a b c

1

(a) Triangular function

μ

a b c

1

d

(b) Trapezoid function

Matching degree μ

1

0

Matching degree μ gradually changing between 0 and 1

Matching degree μ

1

0

0.3

0.8

μ = 1

Membership function 1

Membership function 2

Fuzzy Logic Control ---An example of fuzzy set

)(xA

There three linguistic variables are temp is cold, temp is okay and temp is hot.Each of linguistic variable means a membership function.

1

0

The temp is cold

The temp is okay

The temp is hot

Temperature 。 C 18 23 32

32

0

30

2.0

29

38.0

27

63.0

25

82.0

23

1

20

47.0

19

2.0

18

0)(xAb

)(xokayistempthe Ab

Fuzzy Logic Control---Extension principle

)(0

)(),....,(minsup)(

1

11))(...1( 1

yfif

xxy

rAAyfxrx

B

r

BAyx ff

The extension principle was introduced by Zadeh in 1975 and is an important tool in fuzzy set theory

Ux A xxA /)(

)(/)(/)(/)(

/)()(

2211 yyxyxyx

xxfAfB

BnnAAA

Ux A

Vyxy Ayfx

B

)()( max)(1


x

-3-2-1012

0.50.61.00.90.40.1

8116101

16

max{0.5}=0.5max{0.6,0.1}=0.6max{1,0.4}=1max{0.9}=0.9max={1,0.4}=1max={0.6,0.1}=0.6

)( xA

4)( xxfy

)(yB

U={-3,-2,-1,0,1,2}

4)( xxfy

81

5.0

16

6.0

1

1

0

9.0B


1x )1(1 xA 2x )2(2 xA2

21

)2,1(

xx

xxfy

)()()2,1( yy BAAf

-1-10011

0.50.50.10.10.90.9

-22-22-22

0.41.00.41.00.41.0

-13-22-13

max{0.4,0.4}=0.4max{0.5,0.9}=0.9

max{0.1}=0.1max{0.1}=0.1

max{0.4,0.4}=0.4max{0.5,0.9}=0.9

11 11 1/)1(Xx A xxA

22 22 2/)2(

Xx A xxA

3

9.0

2

1.0

1

4.0

2

1.0

B

V={-2,-1,2,3}

4.0)]4.0,9.0min(),4.0,5.0max[min(

)}]2(),1(min{)},2(),1(max[min{)1( 2121

AAAAB

Fuzzy Logic Control---Rules

There are two parts in a fuzzy ruleThere are two parts in a fuzzy rule: an IF part (antecedent) and a THEN part (consequent). We present an illustrative example to explain the procedure of fuzzy inference. Consider an air conditioning system. The intensity of cooling (fuzzy output) might be determined by temperature (fuzzy input) by the following fuzzy rule:

IF the temperature is high THEN the intensity of cooling is strong


Again, we give a general express for fuzzy rules with multi-dimensional inputs as follows:

IF X1 is AND X2 is AND ……Xn is THEN Y is BvpA1

qA2unA


Temperature is moderate

Temp (℃) 22 302618

Cooling is moderate

45 705530 Intensity of cooling (%)

0.50.5.5

28℃ (crisp input )

Fuzzy output (gray area)

IF PartTHEN Part

Figure 5’: An illustrative example of getting fuzzy output from a crisp input

Fuzzy Inference

Cooling is strong

THEN Part

(a) Visual description of fuzzy rule: “IF temperature is high THEN cooling is strong”

Intensity of cooling (%)

75 90

Temperature is high

IF Part

Temp (℃) 28 35

Temperature is low

Temp (℃) 12

Temperature is moderate

Temp (℃) 22 302518 28

(b) Fuzzy expressions of “Temperature is moderate” and “Temperature is low” in temperature input dimension

Fuzzy Logic Control---Inference

The antecedent of a fuzzy rule uses logic “AND” manipulation (conjunction) to combine the fuzzy input variables X1, X2, …, Xn.. Conceptually, we use “minimum” operation to handle the conjunction of the fuzzy input variables. Consider a fuzzy rule Ri. The overall matching degree for this particular fuzzy rule is obtained by the following formula

},,,{min 21, 1 XnXXRX i


μ

0.7

μ

0.7

(a) The clipping method

Clipped to

10 20 30

μ

0.7

μ

0.7

(b) The scaling method

Scaled to

Input Input

0.8

0.3

0.4

0.9

= 0.3

= 0.4

min

min


•Trigger the fuzzy rules•Using clip method


Union

μ

μ

μ

Figure 8: A conceptual diagram to illustrate the procedure of the fuzzy expert system (taken from [/?ref]).

crisp input 1 crisp input 2

min

min

Union

Defuzzification output

The final overall fuzzy output

Rule 1

Rule 2

Fuzzy variable 2Fuzzy variable 1

THEN PartIF Part

Fuzzy output of Rule 1

Fuzzy output of Rule 2

Fuzzy Logic Control---Defuzzification

Two major defuzzification methods are often used: the Mean of Maximum (MOM) method and the Center of Area (COA) method. The MOM computes the average of output values with maximum matching degrees. The formula for the MOM defuzzification is of the form.

P

y

AMOMPy *

*

)(

)(sup*)(|* yyyp Ay

A


The COA method computes the weighted average of an entire set, which is given by

iiA

iiiA

y

yy

y)(

)(

N

i A

N

i A

Axi

xixiC

1

1


μ μ

Defuzzification output Defuzzification output

(a) The COA method (b) The MOM method

Homework

Implementation of Fuzzy Logic System using java, c++, or matlab, and then using an example to test the system.

Using source code from internet to create an example. Afterwards, to explain the source code more detail and the created example.

A brief introduction of Artificial Intelligence Method and Fuzzy Logic Control S. C. Chen 2009/07/17.

Documents

lot of artificial neuron

machine intelligence

degree of intelligence

mycin slide

neural computer language

weight of network foot

machines agent robot

organism neuron