-
International Journal of Computational Cognition
(http://www.YangSky.com/yangijcc.htm)Volume 1, Number 4, Pages
7990, December 2003Publisher Item Identifier S
1542-5908(03)10404-6/$20.00Article electronically published on
December 25, 2002 at http://www.YangSky.com/ijcc14.htm. Pleasecite
this paper as: Ching-Hung Lee, Jang-Lee Hong, Yu-Ching Lin, and
Wei-Yu Lai, Type-2 Fuzzy Neural Network Systems and Learning,
International Journal of Computational
Cognition(http://www.YangSky.com/yangijcc.htm), Volume 1, Number 4,
Pages 7990, December 2003.
TYPE-2 FUZZY NEURAL NETWORK SYSTEMS ANDLEARNING
CHING-HUNG LEE, JANG-LEE HONG, YU-CHING LIN, AND WEI-YU LAI
Abstract. This paper presents a type-2 fuzzy neural network
system(type-2 FNN) and its learning using genetic algorithm. The
so-calledtype-1 fuzzy neural network (FNN) has the properties of
parallel com-putation scheme, easy to implement, fuzzy logic
inference system, andparameters convergence. And, the membership
functions (MFs) andthe rules can be designed and trained from
linguistic information andnumeric data. However, there is
uncertainty associated with infor-mation or data. Therefore, the
type-2 fuzzy sets are used to treat it.Type-2 fuzzy sets let us
model and minimizes the effects of uncertain-ties in rule-base
fuzzy logic systems (FLS). In this paper, the previousresults of
type-1 FNN are extended to a type-2 one. In addition,
thecorresponding learning algorithm is derived by real-code genetic
algo-rithm. Copyright c2002 Yangs Scientific Research Institute,
LLC.All rights reserved.
1. Introduction
Recently, intelligent systems including fuzzy logic systems,
neural net-works, and genetic algorithm, have been successfully
used in widely variousapplications. The fuzzy neural network
systems (neuro-fuzzy systems) com-bine the advantages of fuzzy
logic systems and neural networks have becomea very active subject
in many scientific and engineering areas, such as, modelreference
control problems, PID controller tuning, signal processing,
etc.[2,3,6-11]. In our previous results, the FNN has the properties
of parallelcomputation scheme, easy to implement, fuzzy logic
inference system, andparameters convergence. The membership
functions (MFs) and the rulescan be designed and trained from
linguistic information and numeric data.Thus, it is then easy to
design an FNN system to achieve a satisfactory level
Received by the editors December 18, 2002 / final version
received December 23, 2002.Key words and phrases. Fuzzy neural
network, type-2 fuzzy sets, genetic algorithm.This work is
supported by the National Science Council, Taiwan, R.O.C., under
Grant
NSC-91-2213-E155-012.
c2002 Yangs Scientific Research Institute, LLC. All rights
reserved.79
-
80 LEE, HONG, LIN, AND LAI
of accuracy by manipulating the network structure and learning
algorithmof the FNN. However, there is uncertainty associated with
information ordata. Therefore, the type-2 fuzzy sets are used to
treat it.
Recently, Mendel and Karnik [5,12,14,15] have developed a
complete the-ory of type-2 fuzzy logic systems. These systems are
again characterized byIF-THEN rules, but their antecedent or
consequent sets are type-2. A type-2 fuzzy set can represent and
handle uncertain information effectively. Thatis, type-2 fuzzy sets
let us model and minimizes the effects of uncertaintiesin rule-base
fuzzy logic systems (FLS). The purpose of this paper is to de-velop
type-2 fuzzy neural network, i.e., extend our previous results of
theFNN the type-2 one. Indeed, The learning algorithm is derived by
geneticalgorithm.
Genetic algorithm (GA) was first proposed by Holland in 1975
[13,17,18].It is motivated by mechanism of natural selection, a
biological process inwhich stronger individuals are likely be the
winners in a competing envi-ronment. It provides an alternative to
traditional optimization techniquesby using directed random
searches to locate optimal solutions in complexproblems
[1,4,13,17,18]. Recently, GA has emerged as a popular family
ofmethods for global optimization. Through the use of genetic
operations, GAperforms a search by evolving a population of
potential solutions [17,18].
The organization of this paper is as follows. In Section 2, we
briefly intro-duce the preliminaries- type-1 fuzzy neural network,
genetic algorithm andrtype-2 fuzzy set. Section 3 presents the main
result- type-2 FNN systemsand learning algorithm. Finally,
conclusion is summarized in Section 4.
2. Preliminaries
2.1. Fuzzy Neural Network (Type-1 FNN system). The fuzzy
neuralnetwork (FNN) system is one kind of fuzzy inference system in
neural net-work structure [2,3,7,10,11]. A schematic diagram of the
four-layered FNNis shown in Fig. 1. Obviously, it is a static model
of recurrent fuzzy neuralnetwork (RFNN) [7]. The type-1 FNN system
has total four layers. Nodesin layer one are input nodes
representing input linguistic variables. Nodesin layer two are
membership nodes. Here, the Gaussian function is used asthe
membership function (MF). Each membership node is responsible
formapping an input linguistic variable into a possibility
distribution for thatvariable. The rule nodes reside in layer
three. The last layer contains theoutput variable nodes. More
details about FNNs, convergent theorems andthe learning algorithm,
can be found in [6-9]. Also, the FNN used here hasbeen shown to be
a universal approximator. That is, for any given realfunction h :
Rn Rp, continuous on a compact set K Rn, and arbitrary
-
TYPE-2 FUZZY NEURAL NETWORK SYSTEMS 81
> 0, there exists a FNN system F (x,W ), such that F (x,W )
h(x) < for every x in K.
Figure 1. Schematic diagram of fuzzy neural networks.
For generality, we must consider m fuzzy rules which can be
consideredindependently like dealing with the jth fuzzy rule in
Figure 2. Indeed, thesimplified fuzzy reasoning is described as
follows.
Given the training input data xk, k = 1, 2, , , n, and the
desired outputyp, p = 1, 2, , ,m. Thejth control rule has the
following form:
Rj: IF x 1 is Aj1 and . . . . . . xn is A
jn THEN y1 is
j1 and . . . . . . ym is
jm.
where j is the rule number, the Ajqs are membership functions of
the an-tecedent part, and jps are real numbers of the consequent
part. Whenthe inputs are given, the truth value i of the premise of
the jth rule iscalculated by
(1) j = Aj1(x1) Aj2(x2) . . . Ajn(xn).
Among the commonly used defuzzification strategies, the
simplified fuzzyreasoning yields a superior result. The output
where yp of the fuzzy reason-ing can be derived from the following
equation.
(2) yp =i
ipi, p = 1, 2, ,m
-
82 LEE, HONG, LIN, AND LAI
where i is the truth value of the premise of the ith rule.
Figure 2. Construction of the jth component of the FNN.
2.2. Type-2 Fuzzy Sets. The concept of type-2 fuzzy set was
initiallyproposed as an extension of ordinary (type-1) fuzzy sets
by Prof. Zaden [19].And then, the clear definition of type-2 fuzzy
set is proposed by Mizumotoand Tanaka [16]. Recently, Mendel and
Karnik [5,12,14,15] have developeda complete theory of type-2 fuzzy
logic systems (FLSs). These systems areagain characterized by
IF-THEN rules, but their antecedent or consequentsets are type-2. A
type-2 fuzzy set can represent and handle uncertaininformation
effectively. That is, type-2 fuzzy sets let us model and
minimizesthe effects of uncertainties in rule-base FLSs. As
literature [14,15], there areat least four sources of uncertainties
in type-1 FLSs, e.g., antecedents andconsequents of rules,
measurement noise, and training date noisy, etc. Allof these
uncertainties can be translated into uncertainties about fuzzy
MFs.The type-1 fuzzy sets could not treat it because the MFs are
crisp. Thatis, type-1 MFs are of two-dimensional, whereas type-2
MFs are of three-dimensional. It is the new third-dimension of
type-2 MFs that make itpossible to model the uncertainties.
Subsequently, we use the following notation and terminology to
describethe fuzzy sets. Firstly, A is a type-1 fuzzy set and the
membership grade ofx X in A is A(x), which is a crisp number in
[0,1]; a type-2 fuzzy set inX is A and the membership grade of x X
in A is A(x), which is a type-1fuzzy set in [0,1]. The type-2 fuzzy
set A X can be represented as
-
TYPE-2 FUZZY NEURAL NETWORK SYSTEMS 83
(3) A(x) = fx(u1)/u1 + fx(u2)/u2 + + fx(um)/um =i
fx(ui)/ui
The useful type-2 fuzzy set is the footprint of uncertainty
(FOU), e.g.,see Fig. 3 [5,12,14,15]. Figures 3 (a) and 3 (b) show
the Gaussian MFs withuncertain STD and Gaussian MFs with uncertain
mean. These ones areused to develop the type-2 FNN systems in using
on primary and consequentparts. Herein, these MFs with uncertain
mean and STD are described as
A(x) = exp( (xm)
2
2
),m [m1,m2] and
A(x) = exp( (xm)
2
2
), [1, 2],(4)
respectively. Obviously, this type membership can be represented
as boundedinterval by upper MF and lower MF, denote A(x) and A(x).
Details abouttype-2 fuzzy sets can be found in literature
[5,12,14,15].
-1 -0.5 0 0.5 1 1.5 2 2.5 30
0.2
0.4
0.6
0.8
1
-1 -0.5 0 0.5 1 1.5 2 2.5 30
0.2
0.4
0.6
0.8
1
(a)(b)
)(xA
)(xA
)(xA
)(xA
Figure 3. Type-2 fuzzy set- (a) Gaussian MFs with un-certain
mean (b) Gaussian MFs with uncertain STD.
The basics of fuzzy logic do not change from type-1 to type-2
sets. Thedifference between these two systems is output processing.
The type-2 FLSsshould use the type-reducer to reduce the output
fuzzy sets degree. As Fig.4 shows [5,12,14,15], the structure of a
type-2 FLS is similar to the structureof type-1 one. The structure
includes fuzzifier, knowledge base, inferenceengine, type-reducer,
and defuzzifier. Based on the block diagram, we willexplanation the
FLSs of type-2 FNN systems in next section.
-
84 LEE, HONG, LIN, AND LAI
Figure 4. The block diagram of type-2 FLS.
2.3. Genetic Algorithm (GA). GA uses a direct analogy of such
naturalevolution. It presumes that the potential solution of a
problem is an indi-vidual and can be represented by a set of
parameters. These parameterscan be structured by a string of values
and are regarded as the genes of achromosome. Herein, we briefly
introduce it. A population consists of afinite number of
chromosomes (or parameters). The GA evaluates a popu-lation and
generates a new one iteratively, with each successive
populationreferred to as a generation. Fitness value, a positive
value is used to reflectthe degree of goodness of the chromosome
for solving the problem, andthis value is closely related to its
objective value. In operation process, aninitial population P(0) is
given, and then the GA generates a new genera-tion P(t) based on
the previous generation P(t-1). The GA uses three basicoperators to
manipulate the genetic composition of a population: reprod-uct,
crossover, and mutation [1,4, 17,18]. The most common
representationin GA is binary [4,13,18]. The chromosomes consist of
a set of genes, whichare generally characters belonging to an
alphabeta {0,1}. In this paper,the real-coded GA is used to tune
the parameters. It is more natural torepresent the genes directly
as real numbers since the representations of thesolutions are very
close to the natural formulation. Therefore, a chromo-some here is
a vector of floating point numbers. The crossover and
mutationoperators developed for this coding is introduced
below.
-
TYPE-2 FUZZY NEURAL NETWORK SYSTEMS 85
3. Type-2 Fuzzy Neural Network and GA
3.1. Type-2 FNN Systems. Herein, we consider a type-2 FLS
systemwith a rule base of R rules in type-2 FNN system, e.g.,
n-input m-outputwith R rules. The jth control rule is described as
the following form:
Rj: IF x 1 is Aj1 and. . . xn is A
jn THEN y1 is
j1 and . . . ym is
jm.
where j is a rule number, the Ajqs are type-2 MFs of the
antecedent part,and jps are type-1 fuzzy sets of the consequent
part. Herein, the antecedentpart MFs are represented as an upper MF
and a lower MF, denote A(x)and A(x) (see Fig. 3). The consequent
part is a interval set = [, ].The rules let us simultaneously
account for uncertainty about antecedentmembership functions and
consequent parameters values.
When the input are given, the firing strength of the jth rule
is
(5) m = Am1 (x1) Am2 (x2) . . . Amn (xn)where is the meet
operation [5,12,14,15]. Herein, the antecedent operationis product
t-norm. That is, equation (1) can be calculated by
m
= Am1
(x1) Am2 (x2) . . . Amn (xn) andm = Am1 (x1) Am2 (x2) . . . Amn
(xn).(6)
Finally, the type reduction and defuzzification should be
considered. Sim-ilar to the FNN, herein the center of sets
(COS)-type reduction method isused to find
yil =Mi=1
ilil and y
il =
Mi=1
irir(7)
where il denotes the firing strength membership grad (either i
or i).
Hence, the defuzzified output of an interval type-2 FLS is
(8) yi =yil + y
ir
2.
Note that, if rule number R is even M = R2 . On the other hand,
R isodd, M = R12 and
(9) y =yl + yr
2+(i
M+1+ iM+1) (iM+1 +
i
M+1)
4.
Herein, we simplify the computation procedure for computing yr
and ylwhich is difference from literature [14,15]. Details of
comparison can befound in literature [14,15].
-
86 LEE, HONG, LIN, AND LAI
Figure 5 summarizes above discussion and shows a fuzzy inference
system(jth rule) of type-2-FNN system.
Example: Computation of type-2 FNN system with two rulesIf a
type-2 FNN system has two rules as follows:R1: IF x1 is A1 AND x2is
B1 THEN y = w1.R2: IF x1 is A2 AND x2 is B2 THEN y = w2.Figure 6
summaries the computation of type-2 FNN system. In the first
layer, the output values are the input x1 and x2, respectively.
In layer 2, onedetermines the MF grads by type-2 MFs, i.e., MF
grads of upper MF andlower MF. Thus, one obtains [Ai(x1), Ai(x1)]
and [Bi(x2), Bi(x2)], i=1,2.Thus, using the operation in
layer-product, one can have [
i(x1, x2), i(x1, x2)] =
[A1(x1) B1(x2), A1(x1) B2(x2)]. Finally, yr and yl should be
determined.Note that wi = [wi, wi], one has yl = 1w1 + 2w2, yr =
1w1 + 2w2,
and the defuzzified value y =yr + yl
2.
Remark : It is trivial that the type-2 FNN system is a
generalization of theFNN system. That is, the type-2 FNN system can
be reduce to a type-1 oneif the fuzzy sets is type-1. We can find
that details computation of thesesystems are the same.
Figure 5. Fuzzy inference of Type-2 FNN.
3.2. Training of Type-2 FNN- Genetic Algorithm. It is known
thatthe type-1 FNN system is a universal approximator [2,3,6-9].
That is, ingeneral, for function mapping or system identification,
it is easy to designan FNN system to achieve a satisfactory level
of accuracy. By the way, we
-
TYPE-2 FUZZY NEURAL NETWORK SYSTEMS 87
Figure 6. Computation example of a Type-2 FNN.
determine the feature parameters to represent a type-2 fuzzy
set. Usingthese parameters, a type-2 FNN system can be encoded as a
chromosome.Then, the real-code genetic algorithm is used optimize
the type-2 FNNsystem, i.e., antecedent and consequent MFs.
Herein, the training process using real-code genetic algorithm
is describedas follows.Learning Process
Step 1: Constructing and initializing the type-1 FNN systemStep
2: Using the back-propagation algorithm to train the type-1 FNN
and obtain a set of Gaussian functions (mean, variance) and
weighting vec-tor.
Step 3: Using the results of Step 2 and add a uncertainty in
antecedentand consequent part, i.e., m1,m2 = m m,,w1, w2 = w w
orm,1, 2 = ,w1, w2 = w w.
Step 4: Constructing the chromosome (2Rmean+R STD+2R
weight).Step 5: Using the GA to train the type-2 FNN to find the
optimal values.
-
88 LEE, HONG, LIN, AND LAI
The objective of parameters learning is to optimally adjust the
free pa-rameters of the type-2 FNN for each incoming data.
Subsequently, in thisphase the chromosome should be
defined.Chromosome: the genes of each chromosome (denotes xi)
include twoparts. One is the MF and the other is weighting vector.
Each MF containstwo means values (upper MF and lower MF), STD, and
weighting vector(or mean, two STD values, and weight). Therefore,
for a given n-input one-output type-2 FNN with R rules, the number
of genes for each chromosomeis 3R n+2R.Fitness function: Herein,
the fitness function is defined as
(10) fitness (x) =1E, E =
t
i
(di (t) yi (t))2
where di (t) and yi (t) are the desired output and type-2 FNN
system output,respectively.Reproduction: The tournament selection
is used in the reproduction pro-cess [13,18].Crossover: Here, the
real-coded crossover operation is used.
(11) x1i = x1i + (x2i x1i)
(12) x2i = x1i + (x1i x2i)where fitness (x1) fitness (x2), x1i
and x2i are the ith genes of theparents x1 and x2, respectively.
x1i and x
2i are the ith genes of the parents
x1 and x2, is a random number and 0 0.5.
Mutation: The mutation operation is
(13) x1i = x1i +
where i denotes the ith gene and it is randomly chosen; x1i and
x1i are theith genes of the parents x1 and x1 respectively; is a
random number in agiven range.
4. Conclusion
This paper has presented a type-2 FNN system and the
correspondinggenetic learning algorithm. This type-2 FNN will be
used to treat the un-certainty associated with information or data.
That dues to the propertiesof type-2 fuzzy sets, it can represent
and handle uncertain information ef-fectively. Therefore, the
previous results of the FNN have been extended to
-
TYPE-2 FUZZY NEURAL NETWORK SYSTEMS 89
a type-2 one. We determine the feature parameters to represent a
type-2fuzzy set. Using these parameters, a type-2 FNN system can be
encoded asa chromosome. Then, the real-code genetic algorithm is
used optimize thetype-2 FNN system, i.e., antecedent and consequent
MFs.
References
[1] T. N. Bui and B. R. Moon, Genetic Algorithm and Graph
Partitioning, IEEETrans. on Computers, Vol.45, No.7, July 1996.
[2] Y. C. Chen and C. C. Teng, A Model Reference Control
Structure Using A FuzzyNeural Network, Fuzzy Sets and Systems, Vol.
73, pp. 291312,1995.
[3] Y. C. Chen and C. C. Teng, Fuzzy Neural Network Systems in
Model ReferenceControl Systems, in Neural Network Systems:
Technique and Applications, Vol. 6,Edited by C. T. Leondes,
Academic Press, Inc., pp. 285-313, 1998.
[4] D. E. Goldberg, Genetic Algorithms in Search, Optimization
and Machine Learn-ing, Addison-Wesley, Reading, 1989.
[5] N. Karnik, J. Mendel, and Q. Liang, Type-2 Fuzzy Logic
Systems, IEEE Trans.On Fuzzy Systems, Vol. 7, No. 6, pp. 643-658,
1999.
[6] C. H. Lee and C. C. Teng, Fine Tuning of Membership
Functions for Fuzzy NeuralSystems, Asian Journal of Control, Vol.
3, No. 3, pp. 18-25, 2001.
[7] C. H. Lee and C. C. Teng, Identification and Control of
Dynamic Systems UsingRecurrent Fuzzy Neural Networks, IEEE Trans.
on Fuzzy Systems, Vol. 8, No. 4,pp. 349-366, August 2000.
[8] C. H. Lee and C. C. Teng, Approximation of Periodic Function
Using a ModifiedFuzzy Neural Network, International Journal of
Fuzzy Systems, Vol. 2, No. 3, pp.176-182, 2000.
[9] C. H. Lee and C. C. Teng, Tuning PID Controller of Unstable
Processes: A FuzzyNeural Network Approach, Fuzzy Sets and Systems,
Vol. 128, No.1, pp. 95-106,2002.
[10] C. T. Lin and C. S. G. Lee, Neural Fuzzy Systems, Prentice
Hall: Englewood Cliff,1996.
[11] C. T. Lin and C. S. G.Lee, Neural-Network-Based fuzzy logic
control and decisionsystem, IEEE Trans. Computers, vol. C-40,
no.12, pp1320-1336, 1991.
[12] Q. Liang and J. Mendel, Interval Type-2 Fuzzy Logic
Systems: Theory and Design,IEEE Trans. On Fuzzy Systems, Vol. 8,
No.5, pp. 535-550, 2000.
[13] K. F. Man, K. S. Tang, S. Kwong, Genetic Algorithms:
Concepts and Applica-tions, IEEE Transactions on Industrial
Electronics, Vol. 43, No. 5, October 1996.
[14] J. Mendel and R. John, Type-2 Fuzzy Sets Made Simple, IEEE
Trans. On FuzzySystems, Vol. 10, No. 2, pp. 117-127, 2002.
[15] J. M. Mendel, Uncertain Rule-Based Fuzzy Logic Systems:
Introduction and NewDirections, Prentice-Hall: NJ, 2001.
[16] M. Mizumoto and K. Tanaka, Some Properties of Fuzzy Sets of
Type 2, Informa-tion and Control, Vol. 31, pp. 312-340, 1976.
[17] Z. Michalewicz, Genetic Algorithms + Data Structure =
Evolutionary Programs,Springer-Verlag, Berlin, 3rd edition
1997.
[18] M. Srinivas, Lalit M. Patnaik, Genetic Algorithms: A
Survey, Computer, Vol. 27,pp.17-66, 1994.
[19] L A. Zadeh, The Concept of A Linguistic Variable and Its
Application to Approx-imate Reasoning -1, Information Sciences,
Vol. 8, pp. 199-249, 1975.
-
90 LEE, HONG, LIN, AND LAI
Ching-Hung Leea, Jang-Lee Hongb, Yu-Ching Lina, and Wei-Yu
LaiaaDepartment of Electrical Engineering, Yuan Ze University, No.
135, Yuan-Tung Road, Chung-Li, Taoyuan 320, Taiwan,
R.O.C.bDepartment of Electronic Engineering, Van Nung Institute of
Technology,Chung-Li, Taoyuan 320, Taiwan, R.O.C.
E-mail address: [email protected] (C.-H. Lee)