A brief introduction of Artificial Intelligence Method and Fuzzy Logic Control S. C. Chen 2009/07/17.
Post on 19-Dec-2015
216 Views
Preview:
Transcript
Outline
Artificial Intelligence
Artificial Neural Network
Data Mining
Genetic Algorithm
Fuzzy Logic Control
Artificial intelligence
AI (artificial intelligence) is a combination of computer science, physiology, and philosophy.
AI is a broad topic, consisting of different fields, from machine vision to expert systems. The element that the fields of AI have in common is the creation of machines that can "think".
In order to classify machines can thinking, it is necessary to define intelligence for the machines.
Artificial intelligence
What does the degree of intelligence of machines to consist of, for example, solving complex problems, or making generalizations and relationships? And what about perception and comprehension? AI is also call machine intelligence that means people who create a system to solve complex problems by intelligence.Research into the areas of learning, of language, and of sensory perception have aided scientists in building intelligent machines.
Artificial Intelligence---historycentury 1940~ 1950~ 1960~ 1970~ 1980~ 1990~
computer 1945 computer (ENIAC)
1957 Fortran
Research of AI 1953 David Chess
1956 Dartmouth
1977Universal Declaration of Knowledge Engineering
1982
Fifth generation computer
1991
Neural
computer
Language of AI 1960
LISP
1973
PROLOG
Knowledge representation
1973
Production
system
1976
Theoretical framework
Expert System 1965
DENDRAL
1975
MYCIN
Artificial Intelligence---research areas
Natural Language Processing
Knowledge of performance
Intelligent Search
Reasoning
Perception problem
Pattern recognition
The management of imprecise and uncertain
Knowledge acquisition
Artificial Intelligence---research areas
Data mining
ANN (Artificial Neural Network)
Fuzzy Theory
Genetic Algorithm
Machine Learning
Artificial Neural Network---Introduction
ANN (Artificial Neural Network) is also known as:Parallel distributed processors
Adaptive systems
Self-organizing systems
Connectionism
Neurocomputers
NN (neural networks)
Artificial Neural Network---Introduction
The definition of ANN:ANN is a kind of computing system that is created by hardware or software. ANN used a lot of artificial neuron to simulate the ability of an organism neuron. Artificial neuron is a simple simulation of an organism neuron that gathered input data from external environment or other artificial neuron. Afterward, to obtain the result data by the procedure of complex computing, than to output the result to external environment or next artificial neuron.
Artificial Neural Network---Neuron
a Inputs
W Weight
SUM Summation
f Activation function
t Output
Artificial Neural Network---Neuron to Neural Network
Different problems have different combination method of network.
Using usage samples to train the network, and than changing the weight of network foot by foot. Finally, making the value of output Y to close to our purpose value.
Artificial Neural Network---Neural Network Learning
Supervised Learning NetworkPrediction, identify, classify
Unsupervised Learning NetworkClustering
Hybrid Learning Network
Associate Learning NetworkData acquisition, Noise filter
Optimization Application NetworkDesign, Scheduling
Artificial Neural Network---Supervised Learning Network
According to the field of problem, providing the sample include input and output data for training.
The network learning from input and output data to adjust the weights of hidden layer to adaptive input and output.
This module just like mother to watch over children for learning.
Back-propagation neural network is most representative in the supervised learning network.
Artificial Neural Network---Back-propagation neural network
The module of perceptron was be proposed in 1957. This module is lack for hidden layer of neural network. So, the learning ability of NN is much more restricted.
In 1985, P. Werbos, D. Parker, G. E. Hinton proposed the learning algorithm of hidden layer. The proposed learning algorithm makes BP-NN to enter the new generation.
Artificial Neural Network---Back-propagation neural network
Examples:Traveling Salesman Problem
3D Kohonen Feature Map
Data Mining---Introduction
Data explosion problemData explosion problem Automated data collection
tools and mature database
technology lead to
tremendous amounts of data
stored in databasesdatabases, data data
warehouseswarehouses and other other
information repositoriesinformation repositories
We are drowning in datadrowning in data,
but starving for knowledgestarving for knowledge!
SolutionSolution: Data warehousing and data mining
Data warehousing and on-line analytical processing
Extraction of interesting knowledgeknowledge (rulesrules, regularitiesregularities, patterns patterns,
constraintsconstraints) from data in large databases
Data Mining---Introduction
Data Mining ---Introduction, history of database
1960s:
Data collection, database creation, IMS (IP Multimedia Subsystem ) and network DBMS (Database Manage System)
1970s: Hierarchical and network database systemsRelational data model, relational DBMS implementation
1980s: RDBMS, advanced data models (extended-relational, OO, deductive, etc.) Application-oriented DBMS (spatial, temporal, engineering, etc.)
1990s: Data mining, data warehousing, multimedia databases, and Web databases
2000sStream data management and miningData mining and its applicationsWeb technology (XML, data integration) and global information systems
Data Mining ---Introduction
Data mining (knowledge discovery fromknowledge discovery from data data) Extraction of interestinginteresting (non-trivialnon-trivial, implicitimplicit, previously unknownpreviously unknown and potentially usefulpotentially useful) patternspatterns or knowledgeknowledge from huge amount huge amount of dataof data
Data mining: a misnomer?Data mining is not only just data ， but it can finding knowledge!!
Alternative namesKnowledge discoveryKnowledge discovery (mining) in databases (KDD), knowledgknowledge extractione extraction, business intelligencebusiness intelligence, data/pattern analysis, data archeology, data dredging, information harvesting, etc.
Many people treat data mining as a synonymsynonym for another popularly used term, Knowledge Knowledge Discovery from DataDiscovery from Data (KDD) — GeneralizedGeneralized Data Data
miningmining
Alternatively, other view data mining as simply an essential stepan essential step in the process of knowledge discovery — NarrowlyNarrowly Data miningData mining
Data Mining ---Introduction
Data Mining ---Knowledge Discovery (KDD) Process
Data mining—core of knowledge discovery process
Data Cleaning and Data Integration
Databases
Data Warehouse
Task-relevant Data
Selection and Transformation
Data Mining
Evaluation and Presentation
Patterns
Data Mining---KDD Process: Several Key Steps
1.1. Data cleaning Data cleaning Remove noisenoise and inconsistent datainconsistent datamay take 60% of effort!
2.2. Data integration Data integration Where multiple data source may be combined
3.3. Data selection Data selection Where data relevant to the analysis taskrelevant to the analysis task are retrieved from the DB
4.4. Data transformation Data transformation Where data are transformed or consolidated into forms appropriate for mining by performing summarysummary or aggregationaggregation operations, for instancefor instance.
5.5. Data mining Data mining Intelligent methods are applied in order to extract data patterns.Choosing the mining algorithm(s)Choosing the mining algorithm(s) for searching patterns of interest
6.6. Pattern evaluation Pattern evaluation To identify the truly interesting patternsidentify the truly interesting patterns representing knowledge based on some interestingness measures.
7.7. Knowledge presentation Knowledge presentation Where visualization and knowledge representation techniques are used to presepresent the mined knowledge to the usernt the mined knowledge to the user.
We adopt a broad view of data mining functionality:
Data mining is the process of discovering interesting the process of discovering interesting
knowledgeknowledge from large amounts of datalarge amounts of data stored in
databases, data warehouses, or other information
repositories.
Data Mining
Data Mining--- Typical Data Mining System
data cleaning, integration, and selection
Database or Data Warehouse Server
Data Mining Engine
Pattern Evaluation
Graphical User Interface
Knowledge-Base
DatabaseData
WarehouseWorld-Wide
WebOther Info
Repositories
OLAP: OLAP: On line analytical ProcessingOn line analytical Processing
Data Mining and Business Intelligence
Increasing potentialto supportbusiness decisions
End User
Business Analyst
DataAnalyst
DBA
Decision
MakingData Presentation
Visualization Techniques
Data MiningInformation Discovery
Data ExplorationStatistical Summary, Querying, and Reporting
Data Preprocessing/Integration, Data Warehouses
Data SourcesPaper, Files, Web documents, Scientific experiments, Database Systems
Data Mining---What Kind of Data?
Relational databases
Data warehouses
Transactional databases
Advanced DB and information repositoriesObject-oriented and object-relational databases
Spatial and Spatiotemporal Databases
Temporal, Sequence, and Time-Series Databases
Text databases and multimedia databases
Heterogeneous and legacy databases
Data Streams
WWW
Data Mining---What kinds of patterns can be mined?
Data mining functionalities are used to specify the kinds of patterns the kinds of patterns to be foundto be found in data mining tasks.
Data mining tasks can be classified into two categories:
Descriptive Descriptive :
Characterize the general propertiesCharacterize the general properties of the data in the database.
Predictive Predictive :
Perform inferenceinference on the current data in order to make predictions.
In some cases, users may have no idea regarding what kinds of patterns in their data may be interesting interesting, and hence may kind to search for several different kinds of patterns in parallelin parallel.
Thus it is important to have a data mining system that can mine multiple kinds of patternsmultiple kinds of patterns to accommodate different user expectations or applications.
Data mining functionalities, and the kinds of
patterns they can discover, are described below:Concept descriptionConcept description: Characterization and discrimination
Association AnalysisAssociation Analysis
ClassificationClassification and PredictionPrediction
ClusterCluster analysisanalysis
Outlier analysisOutlier analysis
Trend Trend and evolution analysisevolution analysis
Data Mining---What kinds of patterns can be mined?
Data Mining---Association analysis
From transactional databases, RDBMS or other large amounts of data item of storage systems. Finding the interested pattern and frequent pattern to analysis the associationsassociations and the correlationscorrelations between each data item.
this associations doesn’t present directly in data
The best example is to ensure association rules.
Data Mining--- Data Cube Aggregation
Data cubes store multidimensional aggregated inmultidimensional aggregated in
formationformation.
For example:
Data Mining ---Potential Applications
Data analysis and decision supportData analysis and decision support
Market analysis and management
Target marketing, customer relationship management (CRM), market
basket analysis, cross selling, market segmentation
Risk analysis and management
Forecasting, customer retention, improved underwriting, quality
control, competitive analysis
Fraud detection and detection of unusual patterns (outliers)
Other ApplicationsOther Applications
Text mining (news group, email, documents)
Web mining
Bioinformatics and bio-data analysis
Genetic Algorithm---Introduction
Genetic algorithms are a part of evolutionary computing, which is a rapidly growing area of artificial intelligence. Based on Darwinian principles of biological evolution.
First proposed by Prof. John Holland in 1975, and his colleague at Univ. of Michigan.
Genetic Algorithm---Introduction
Chromosomes are strings of DNA and serves as a model for the whole organism.
A chromosome consists of genes.
Each gene encodes a trait.
Complete set of genetic material (all chromosomes) is called genome.
Particular set of genes in genome is called genotype.
Genetic Algorithm ---Encoding
The chromosome should in some way contain information about solution which it represents.
The most used way of encoding is a binary string.
Chromosome 1 1101100100110110Chromosome 2 1101111000011110
Each bit in this string can represent some characteristic of the solution. One can encode directly integer or real numbers.
Genetic Algorithm---Procedure
The initial population generate randomly
Evaluating the fitness function
Is reach the condition?Optimal solution
YES
Reproduction
Crossover
Mutation
NO
Genetic Algorithm---Fitness function
Algorithm is started with a set of solutions (represented by chromosomes) called population. Solutions from one population are taken and used to form a new population. The new population (offspring) will be better than the old one (parent).Solutions which are selected to form new solutions are selected according to their fitness - the more suitable they are the more chances they have to reproduce.
Genetic Algorithm---Reproduction
Reproduction is a kind of computing process that used to decide population elimination by the fitness degree. Fitness degree is calculate by fitness function.
Selection methods:Roulette Wheel Selection
Competitive Selection
Genetic Algorithm ---Crossover
Crossover selects genes from parent chromosomes and creates a new offspring.
Chromosome 111011 | 00100110110Chromosome 211011 | 11000011110Offspring 1 11011 | 11000011110Offspring 2 11011 | 00100110110 “ | “ is the crossover point
Genetic Algorithm ---Crossover
Single point crossover: 11001011+11011111 = 11001111
Two point crossover: 11001011 + 11011111 = 11011111
Uniform crossover: 11001011 + 11011101 = 11011111
Arithmetic crossover: 11001011 + 11011111 = 11001001
Genetic Algorithm---Mutation
Prevent falling all solutions in population into a local optimum of solved problem
Mutation changes randomly the new offspring.
Original offspring 11101111000011110Mutated offspring 11100111000011110Original offspring 21101100100110110Mutated offspring 21101101100110110
Genetic Algorithm---Roulette Wheel Selection
Parents are selected according to their fitness.
The better the chromosomes are, the more chances to be selected they have.
Fuzzy logic control---Part1---Introduction
1965 Fuzzy Set (Prof. Lotfi A.Zadeh,UCB)
1966 Fuzzy logic (Dr. Peter N.Marinos, Bell Lab)
1972 Fuzzy Measure
(Prof.Michio Sugeno)Fuzzy Set
Fuzzy Event
CrispElement
Fuzzy logic control ---Introduction
Knowledge Representation
example: age (Man Old)
traditional
Age (Man Gt 60)
30 60 Ages
1
Membership Function
Fuzzy Logic
AsubsetfuzzytheinxelementtheofmembershipxA :)(Esetreferencetheofelementanx :
EofsubsetFuzzyBA ,,
]1,0[,),(),( baxbxa BA
),( baMINba
),( baMAXba
aa 1
)()( bababa
Fuzzy Logic Control---Introduction
abbaityCommutativ )()(
aa )(
stheoremsDeMorgan '
baba )(
)()( cbacbaeAssociativ
)()()( cabacbavityDistributi
baba )(
Fuzzy Logic Control---Introduction
二值理論推論形式（事實） 麻雀是鳥（規則） 鳥會飛（結論） 麻雀會飛AI Language as LISP,Prolog “Pattern Matching”Fuzzy 推論形式：（事實） 這番茄很紅（規則） 蕃茄若是紅了就熟了（結論） 這蕃茄很熟了
Fuzzy Logic Control---Introduction
(facts) X is
(rule) if X is A then Y is B
希望得到的結論是(result) Y is B
A
1
0
1
0
A
B
A B
Mamdani 法
Fuzzy Logic Control---Crisp set
Set A, whose elements are x1,…. xn , is always written as A={x1,…,xn}
A={x | P(x) } ,x has the property P
Set A is defined by its characteristic function,
AxforAxforA x 1
0)(
A
An example set A={1,2,3,4,5}, U={0,1,2,3,4,5,6,7,8,9}. z(x) isa characteristic function of set A. z(1)=1, z(2)=1, z(3)=1, z(4)=1, z(5)=1, z(6)=0, z(0)=0z(7)=0, z(9)=0
Membership function denoted as ,
There are three stand type of membership
functions, triangle, trapezoid and Gaussian.
A fuzzy set F in X may be represented as
When X is continuous ：
When X is discrete ：
Fuzzy Logic Control---Definition
A 10 A
XxxxF F |))(,( xxF
X
F /)(
xxFX
F /)(
Fuzzy Logic Control---Basic characteristics of fuzzy sets
1. Vertical dimension: height, normalization (maximum form: at least one element with 1.0 membership and one element with membership 0).
2. Horizontal dimension: universal discourse, support sets, alpha cuts.
3. Representation schemes: membership function, ordered set of pairs, polynomial-like (integral-like).
Fuzzy Logic Control---BASIC CONCEPTS
A ofset level thein allfor ,)A(
by defined is function membership whoseN on defined number)(fuzzy set fuzzy a :ycardinalitfuzzy .7
).(A:ycardinalitscalar 6.
.continuous piecewise is function membership whose Ron definedset fuzzy normalized andconvex a :numberfuzzy .5
}.)(|{A :cut-4.
grade. membership possible maximum theattains elements its of oneleast at :setfuzzy Normalized3.
set. that in elementany by attained grade membershiplargest the:Height2.
0}.(x)μ|X{x A supp :Support1.
A~
A
Xx
A
A
x
xXx
).f(x/)f(x/)f(x/
)///f(f(A)
,///A
:principle extension13.
)](),(min[)( :onintersecti12.
)](),(max[)( :union.11
).(1)( :complement10.
B.A )()( :eequivalenc9.
subset.proper B.A )()( :subset.8
n2211
2211
2211
BABA
BABA
AA
BA
BA
n
nn
nn
xxx
xxx
xxx
xxx
xx
xx
xx
─
─
Fuzzy Logic Control---Types of Membership Functions
1. Linear type: linear/step functions. Approximating an unknown or poorly understood concept that is not a fuzzy number. (often expressed as shouldered sets)
2. Triangular type:
often used to model process control systems.
Fuzzy Logic Control---Types of Membership Functions
when you decompose a variable into fuzzy sets, the amount of overlap must vary between 10% to 50%.In modeling dynamic systems, it can approximate their behaviors to nearly any degree of precision.
cx
cxbbcxc
bxaabax
ax
cbaxTri
0
)/()(
)/()(
0
),,;(
Fuzzy Logic Control---Types of Membership Functions
3. Trapezoidal Type:
dx
dxccdxd
cxb
bxaabax
ax
dcbaxTra
0
)/()(
1
)/()(
0
),,,;(
Fuzzy Logic Control---Types of Membership Functions
4. Sigmoid/Logistic Type:
* modeling population dynamics where the sampling of individual values approximates a continuous random variable.
* frequency (proportional) representation: usually, most, always.
cx
cxbaccx
bxaacax
ax
cbaxS
1
))/()((21
)/()(2
0
),,;( 2
2
Fuzzy Logic Control---Types of Membership Functions
1. Fuzzy numbers and around representation: around, close to, few, some.
2. PI, Beta, Gaussian fuzzy sets (bell-shaped): the slope and width of the bell curves indicate the degree of compactness associated with the fuzzy number.
3. PI curves: is not asymptotic. Zero point is at a discrete and specified point.
xbcbccxS
xcbcbcxScbx
),2/,;(1
),2/,;(),;(
Fuzzy Logic Control---Types of Membership Functions
4. Beta curves: more tightly compacted than PI.
5. Similar to beta curves, but the slope goes to zero very quickly with a very short tail.
6. Irregularly shaped and arbitrary fuzzy sets: (domain value, membership) pairs linear interpolation.
2)(
12
),;(
))/)((1(),;(xckeckxG
bcxbcxB
Fuzzy Logic Control---Concept of membership function
μ
a b c
1
(a) Triangular function
μ
a b c
1
d
(b) Trapezoid function
Matching degree μ
1
0
Matching degree μ gradually changing between 0 and 1
Matching degree μ
1
0
0.3
0.8
μ = 1
Membership function 1
Membership function 2
Fuzzy Logic Control ---An example of fuzzy set
)(xA
There three linguistic variables are temp is cold, temp is okay and temp is hot.Each of linguistic variable means a membership function.
1
0
The temp is cold
The temp is okay
The temp is hot
Temperature 。 C 18 23 32
32
0
30
2.0
29
38.0
27
63.0
25
82.0
23
1
20
47.0
19
2.0
18
0)(xAb
)(xokayistempthe Ab
Fuzzy Logic Control---Extension principle
)(0
)(),....,(minsup)(
1
11))(...1( 1
yfif
xxy
rAAyfxrx
B
r
BAyx ff
The extension principle was introduced by Zadeh in 1975 and is an important tool in fuzzy set theory
Ux A xxA /)(
)(/)(/)(/)(
/)()(
2211 yyxyxyx
xxfAfB
BnnAAA
Ux A
Vyxy Ayfx
B
)()( max)(1
Fuzzy Logic Control---Extension principle
x
-3-2-1012
0.50.61.00.90.40.1
8116101
16
max{0.5}=0.5max{0.6,0.1}=0.6max{1,0.4}=1max{0.9}=0.9max={1,0.4}=1max={0.6,0.1}=0.6
)( xA
4)( xxfy
)(yB
U={-3,-2,-1,0,1,2}
4)( xxfy
81
5.0
16
6.0
1
1
0
9.0B
Fuzzy Logic Control---Extension principle
1x )1(1 xA 2x )2(2 xA2
21
)2,1(
xx
xxfy
)()()2,1( yy BAAf
-1-10011
0.50.50.10.10.90.9
-22-22-22
0.41.00.41.00.41.0
-13-22-13
max{0.4,0.4}=0.4max{0.5,0.9}=0.9
max{0.1}=0.1max{0.1}=0.1
max{0.4,0.4}=0.4max{0.5,0.9}=0.9
11 11 1/)1(Xx A xxA
22 22 2/)2(
Xx A xxA
3
9.0
2
1.0
1
4.0
2
1.0
B
V={-2,-1,2,3}
4.0)]4.0,9.0min(),4.0,5.0max[min(
)}]2(),1(min{)},2(),1(max[min{)1( 2121
AAAAB
Fuzzy Logic Control---Rules
There are two parts in a fuzzy ruleThere are two parts in a fuzzy rule: an IF part (antecedent) and a THEN part (consequent). We present an illustrative example to explain the procedure of fuzzy inference. Consider an air conditioning system. The intensity of cooling (fuzzy output) might be determined by temperature (fuzzy input) by the following fuzzy rule:
IF the temperature is high THEN the intensity of cooling is strong
Fuzzy Logic Control---Rules
Again, we give a general express for fuzzy rules with multi-dimensional inputs as follows:
IF X1 is AND X2 is AND ……Xn is THEN Y is BvpA1
qA2unA
Fuzzy Logic Control---Rules
Temperature is moderate
Temp (℃) 22 302618
Cooling is moderate
45 705530 Intensity of cooling (%)
0.50.5.5
28℃ (crisp input )
Fuzzy output (gray area)
IF PartTHEN Part
Figure 5’: An illustrative example of getting fuzzy output from a crisp input
Fuzzy Inference
Cooling is strong
THEN Part
(a) Visual description of fuzzy rule: “IF temperature is high THEN cooling is strong”
Intensity of cooling (%)
75 90
Temperature is high
IF Part
Temp (℃) 28 35
Temperature is low
Temp (℃) 12
Temperature is moderate
Temp (℃) 22 302518 28
(b) Fuzzy expressions of “Temperature is moderate” and “Temperature is low” in temperature input dimension
Fuzzy Logic Control---Inference
The antecedent of a fuzzy rule uses logic “AND” manipulation (conjunction) to combine the fuzzy input variables X1, X2, …, Xn.. Conceptually, we use “minimum” operation to handle the conjunction of the fuzzy input variables. Consider a fuzzy rule Ri. The overall matching degree for this particular fuzzy rule is obtained by the following formula
},,,{min 21, 1 XnXXRX i
Fuzzy Logic Control---Inference
μ
0.7
μ
0.7
(a) The clipping method
Clipped to
10 20 30
μ
0.7
μ
0.7
(b) The scaling method
Scaled to
Figure 8: A conceptual diagram to illustrate the procedure of the fuzzy expert system (taken from [/?ref]).
crisp input 1 crisp input 2
min
min
Union
Defuzzification output
The final overall fuzzy output
Rule 1
Rule 2
Fuzzy variable 2Fuzzy variable 1
THEN PartIF Part
Fuzzy output of Rule 1
Fuzzy output of Rule 2
Fuzzy Logic Control---Defuzzification
Two major defuzzification methods are often used: the Mean of Maximum (MOM) method and the Center of Area (COA) method. The MOM computes the average of output values with maximum matching degrees. The formula for the MOM defuzzification is of the form.
P
y
AMOMPy *
*
)(
)(sup*)(|* yyyp Ay
A
Fuzzy Logic Control---Defuzzification
The COA method computes the weighted average of an entire set, which is given by
iiA
iiiA
y
yy
y)(
)(
N
i A
N
i A
Axi
xixiC
1
1
Fuzzy Logic Control---Defuzzification
μ μ
Defuzzification output Defuzzification output
(a) The COA method (b) The MOM method
top related