Artificial Neural Network - Back Propagation of Error
Artificial Neural Network - Back Propagation of Error
LI M UT khi ra i, my tnh nhanh chng pht trin v ng mt vai tr rt
quan trng trong nghin cu khoa hc k thut cng nh trong i sng.
Nhng mt my tnh d c mnh n u chng na, cng ch c th lm vic theo mt
chng trnh c hoch nh sn bi lp trnh vin. N vn khng c kh nng lin tng,
kt ni s vic ny vi s vic khc, v quan trng hn ht l kh nng sng to nh
con ngi.
Mng nron nhn to (Artificial neural networks ) ra i xut pht t tng
m phng hot ng ca b no con ngi.
Mng noron nhn to l s ti to bng k thut nhng chc nng ca h thn kinh
con ngi vi v s cc nron c lin kt truyn thng vi nhau qua mng. Ging nh
con ngi, ANN c hc bi kinh nghim, lu nhng kinh nghim v s dng trong
nhng tnh hung ph hp.
Mng nron trong mt vi nm tr li y c nhiu ngi quan tm v p dng thnh
cng trong nhiu lnh vc khc nhau, nh ti chnh, y t, a cht v vt l, d
bo, phn loi... Kt hp cht ch vi logic m, mng nron nhn to to nn cuc
cch mng thc s trong vic thng minh ha v vn nng ha cc b iu khin k
thut cao cho c hin nay v trong tng lai. V d nh ng dng t ng iu khin
h thng li tu, h thng d bo s c,
Mng nron da trn vic m phng cp thp h thng nron sinh hc. Trong tng
lai vi s pht trin m phng nron sinh hc, chng ta c th c loi my tnh
thng minh tht s.
V l do nhm 11 chng em la chn ti Tm hiu k thut mng n ron nhn to.
i su mng Back Propagation of Error. Ci t minh ha. y l mt ti kh ln v
phc tp, song trong phm vi mt bi bo co ngn, chng em xin trnh by nhng
kin thc mc c bn nht.
hon thnh bi bo co ny, chng em nhn c s hng dn nhit tnh ca ThS.
Nguyn Phng Nga ging vin b mn Tnh ton mm lp KHMT2-K6 chng em. Nhng
bi ging v ti liu ca c chnh l c s chng em c th hon thnh tt bo co ca
mnh. Chng em xin chn thnh cm n c!
Nhm thc hinMC LCPHN 1: GII THIU CHUNG V TNH TON MM4
1. Mt s c im ca cng ngh tnh ton mm4
2. Lch s pht trin5
3. ng dng tnh ton mm5PHN 2: GII THIU V ANN6
I. LCH S PHT TRIN CA MNG NRON6
II. MNG NRON SINH HC8
III. MNG NRON NHN TO10
1. Khi nim10
2. Cc kin trc mng12
143. Perceptron
153.1 Lut hc Perceptron
153.2 Gii thut hc Perceptron
164. Mng nhiu tng truyn thng (MLP)
175. Mng Hopfied
6. Xy dng mng nron18
6.1 Chun b d liu18
216.2 Xc nh cc tham s cho mng:
236.3 Vn lng qun (catastrophic forgetting)
246.4 Vn qu khp
257. Hun luyn mng nron:
7.1 Hc c gim st25
267.2 Hc khng gim st
277.3 Hc tng cng
27IV. NG DNG CA MNG NORON
PHN 3: MNG LAN TRUYN NGC SAI S28
28I. TNG QUAN V MNG LAN TRUYN NGC LI
29II.GII THUT HC BP
1. Gii thut hun luyn mng29
312. Thut ton tng qut
323. S gii thut hc BP
324. K thut hc c gim st:
34PHN 4: BI TON MINH HA
341. Gii thiu bi ton
342. Cc bc ci t bi ton bng lnh trong Matlab
373. Ci t bi ton s dng cng c nntool trong Matlab
42KT LUN
43TI LIU THAM KHO
44 KIN NHN XT CA GING VIN HNG DN
PHN 1: GII THIU CHUNG V TNH TON MM
Trong thc t cuc sng, cc bi ton lin quan n hot ng nhn thc, tr tu
ca con ngi u hm cha nhng i lng, thng tin m bn cht l khng chnh xc,
khng chc chn, khng y . V d: s chng bao gi c cc thng tin, d liu cng
nh cc m hnh ton y v chnh xc cho cc bi ton d bo thi tit. Nhn chung
con ngi lun trong bi cnh l khng c thng tin y v chnh xc cho cc hot
ng ra quyt nh ca bn thn mnh.
Trong lnh vc khoa hc k thut cng vy, cc h thng phc tp trn thc t
thng khng th m t y v chnh xc bi cc phng trnh ton hc truyn thng. Kt
qu l nhng cch tip cn kinh in da trn k thut phn tch v cc phng trnh
ton hc nhanh chng t ra khng cn ph hp. V th, cng ngh tnh ton mm chnh
l mt gii php trong lnh vc ny.
Cng ngh tnh ton mm (Soft Computing SC) l mt h thng c t chc cht
ch. H thng ny l s hp nht ca cc phng php c thit k m hnh ha nhm gii
quyt cc bi ton thc khng th hoc qu kh m hnh ha bng ton hc.
ch ca cng ngh tnh ton mm (SC) l pht hin v tm li gii vi bi ton kh
do s m h, khng chc chn, xp x v ch ng mt phn ca d liu, nhm t c s tng
ng vi kh nng tr tu ca con ngi v l gii v hc trong mi trng m h.1. Mt
s c im ca cng ngh tnh ton mm:
- Tnh ton mm cn c trn cc c im, hnh vi ca con ngi v t nhin a ra
cc quyt nh hp l trong iu kin khng chnh xc v khng chc chn to ra cc
my thng minh (th h mi ca TTNT).
- Cc thnh phn ca tnh ton mm c s b sung, h tr ln nhau to h thng c
t chc cht.
- Tnh ton mm l mt hng nghin cu m, bt k mt k thut mi no c to ra t
vic bt trc tr thng minh ca con ngi u c th tr thnh mt thnh phn mi ca
tnh ton mm.
- Cng ngh tnh ton mm bao gm 4 thnh phn chnh: Fuzzy Computing
(FC), Evolutionary Computing (EC), Neural Computing (NC) and
Probabilistic Computing (PC - cn ang pht trin).
2. Lch s pht trin:
SC
=
EC + NN+FL
Soft
Evolutionary Neural Fuzzy
Computing
Computing Netwwork Logic Zadeh
Rechenberg McCulloch Zadeh
1981
1960
1943
1965
EC =
GP + ES
+EP+ GA Evolutionary Genetic Evolution Evolutionary Genetic
Computing
Programming Strategies Progamming Algorithms
Rechenberg Koza Rechenberg Fogel Holland
1960
1992
1965 1962
19703. ng dng ca tnh ton mm:Cc nghin cu gn y v tnh ton mm l bc
pht trin ca cng ngh tnh ton v m ra nhiu ng dng.
Cc m hnh tnh ton mm thng da vo kinh nghim con ngi, s dng dung
sai cho php ca s khng chnh xc, tnh bt nh, gn ng, xp x tm li gii hiu
qu ch n gin, d hiu, d thc hin, chi ph thp.
Cc tng c bn ca tnh ton mm u tin xut hin theo cc bi bo ca Zadeh v
l thuyt tp m vo 1965, sau l bi bo nm 1973 v phn tch h thng phc tp v
qu trnh ra quyt nh, tip theo l bi bo nm 1981 v l thuyt kh nng v phn
tch d kin mm. V sau, mng thn kinh v gii thut di truyn gp phn nng
cao hiu qu ca tnh ton mm.Cc ng dng thnh cng ca tnh ton mm cho thy
tnh ton mm ngy cng pht trin mnh v ng vai tr c bit trong cc lnh vc
khc nhau ca Khoa hc v K thut.
Tnh ton mm biu th mt s chuyn dch, bin ha quan trng trong cc hng
tnh ton. S chuyn dch ny phn nh s kin tr tu con ngi, khng nh my tnh,
c kh nng ng k trong vic lu tr v x l thng tin khng chnh xc v bt nh,
v y mi l nhng thng tin thc t v thng gp.
PHN 2: GII THIU V ANN
(ARTIFICIAL NEURAL NETWORK)I. LCH S PHT TRIN CA MNG NRON
Cc nghin cu v b no con ngi c tin hnh t hng nghn nm nay. Cng vi s
pht trin ca khoa hc k thut c bit l nhng tin b trong ngnh in t hin
i, vic con ngi bt u nghin cu cc nron nhn to l hon ton t nhin. S kin
u tin nh du s ra i ca mng nron nhn to din ra vo nm 1943 khi nh thn
kinh hc Warren McCulloch v nh ton hc Walter Pitts vit bi bo m t cch
thc cc nron hot ng. H cng tin hnh xy dng mt mng nron n gin bng cc
mch in. Cc nron ca h c xem nh l cc thit b nh phn vi ngng c nh. Kt
qu ca cc m hnh ny l cc hm logic n gin chng hn nh a OR b hay a AND
b.
Tip bc cc nghin cu ny, nm 1949 Donald Hebb cho xut bn cun sch
Organization of Behavior. Cun sch ch ra rng cc nron nhn to s tr nn
hiu qu hn sau mi ln chng c s dng.
Nhng tin b ca my tnh u nhng nm 1950 gip cho vic m hnh ha cc
nguyn l ca nhng l thuyt lin quan ti cch thc con ngi suy ngh tr thnh
hin thc. Nathanial Rochester sau nhiu nm lm vic ti cc phng th nghim
nghin cu ca IBM c nhng n lc u tin m phng mt mng nron. Trong thi k
ny tnh ton truyn thng t c nhng thnh cng rc r trong khi nhng nghin
cu v nron cn giai on s khai. Mc d vy nhng ngi ng h trit l thinking
machines (cc my bit suy ngh) vn tip tc bo v cho lp trng ca mnh.
Nm 1956 d n Dartmouth nghin cu v tr tu nhn to (Artificial
Intelligence) m ra thi k pht trin mi c trong lnh vc tr tu nhn to ln
mng nron. Tc ng tch cc ca n l thc y hn na s quan tm ca cc nh khoa
hc v tr tu nhn to v qu trnh x l mc n gin ca mng nron trong b no con
ngi.
Nhng nm tip theo ca d n Dartmouth, John von Neumann xut vic m
phng cc nron n gin bng cch s dng rle in p hoc n chn khng. Nh sinh
hc chuyn nghin cu v nron Frank Rosenblatt cng bt u nghin cu v
Perceptron. Sau thi gian nghin cu ny Perceptron c ci t trong phn
cng my tnh v c xem nh l mng nron lu i nht cn c s dng n ngy nay.
Perceptron mt tng rt hu ch trong vic phn loi mt tp cc u vo c gi tr
lin tc vo mt trong hai lp. Perceptron tnh tng c trng s cc u vo, ri
tr tng ny cho mt ngng v cho ra mt trong hai gi tr mong mun c th.
Tuy nhin Perceptron cn rt nhiu hn ch, nhng hn ch ny c ch ra trong
cun sch v Perceptron ca Marvin Minsky v Seymour Papert vit nm
1969.
Nm 1959, Bernard Widrow v Marcian Hoff thuc trng i hc Stanford
xy dng m hnh ADALINE (ADAptive LINear Elements) v MADALINE.
(Multiple ADAptive LINear Elements). Cc m hnh ny s dng quy tc hc
Least-Mean-Squares (LMS: Ti thiu bnh phng trung bnh). MADALINE l
mng nron u tin c p dng gii quyt mt bi ton thc t. N l mt b lc thch
ng c kh nng loi b tn hiu di li trn ng dy in thoi. Ngy nay mng nron
ny vn c s dng trong cc ng dng thng mi.
Nm 1974 Paul Werbos pht trin v ng dng phng php hc lan truyn ngc
(back-propagation). Tuy nhin phi mt mt vi nm th phng php ny mi tr
nn ph bin. Cc mng lan truyn ngc c bit n nhiu nht v c p dng rng di
nht cho n ngy nay.
Tht khng may, nhng thnh cng ban u ny khin cho con ngi ngh qu ln
v kh nng ca cc mng nron. Chnh s cng iu qu mc c nhng tc ng khng tt n
s pht trin ca khoa hc v k thut thi by gi khi ngi ta lo s rng n lc
my mc c th lm mi vic ca con ngi. Nhng lo lng ny khin ngi ta bt u
phn i cc nghin cu v mng nron. Thi k tm lng ny ko di n nm 1981.
Nm 1982 trong bi bo gi ti vin khoa hc quc gia, John Hopfield bng
s phn tch ton hc r rng, mch lc, ng ch ra cch thc cc mng nron lm vic
v nhng cng vic chng c th thc hin c. Cng hin ca Hopfield khng ch gi
tr ca nhng nghin cu khoa hc m cn s thc y tr li cc nghin cu v mng
nron.
Cng trong thi gian ny, mt hi ngh vi s tham gia ca Hoa K v Nht Bn
bn v vic hp tc/cnh tranh trong lnh vc mng nron c t chc ti Kyoto,
Nht Bn. Sau hi ngh, Nht Bn cng b nhng n lc ca h trong vic to ra my
tnh th h th 5. Tip nhn iu , cc tp ch nh k ca Hoa K by t s lo lng
rng nc nh c th b tt hu trong lnh vc ny. V th, ngay sau , Hoa K
nhanh chng huy ng qu ti tr cho cc nghin cu v ng dng mng n ron.
Nm 1985, vin vt l Hoa K bt u t chc cc cuc hp hng nm v mng nron
ng dng trong tin hc (Neural Networks for Computing).
Nm 1987, hi tho quc t u tin v mng nron ca Vin cc k s in v in t
IEEE (Institute of Electrical and Electronic Engineer) thu ht hn
1800 ngi tham gia.
Ngy nay, khng ch dng li mc nghin cu l thuyt, cc nghin cu ng dng
mng nron gii quyt cc bi ton thc t c din ra khp mi ni. Cc ng dng mng
nron ra i ngy cng nhiu v ngy cng hon thin hn. in hnh l cc ng dng: x
l ngn ng (Language Processing), nhn dng k t (Character
Recognition), nhn dng ting ni (Voice Recognition), nhn dng mu
(Pattern Recognition), x l tn hiu (Signal Processing), Lc d liu
(Data Filtering),..II. MNG NRON SINH HC Khng c mt nh ngha tng qut v
mng n ron, song cc chuyn gia trong lnh vc ny u c chung mt quan im:
Mng n ron bao gm mt tp hp cc phn t x l n gin m chng ta gi l cc n
ron, c lin kt vi nhau v hot ng song song vi nhau. Tnh nng ca mng ph
thuc vo cu trc ca mng, trng s lin kt gia cc n ron v qu trnh x l bn
trong cc n ron. Mng n ron c kh nng hc s liu v tng qut ha t cc s liu
hc.
Cc mng n ron c xy dng m phng chc nng ca mt mng n ron sinh hc ni
chung hay mng n ron sinh hc ca con ngi ni ring.
* M hnh v qu trnh x l trong n ron sinh hc:H thn kinh ca con ngi
c khong 1010 n ron. Mi n ron bao gm thn n ron (cell body), bn trong
c nhn (soma), h thng dy thn kinh vo hnh cy (dendrites) v mt u dy
thn kinh ra (axon).Cc u dy thn kinh vo nhn tn hiu t cc n ron khc,
nhn n ron s sinh ra tn hiu u ra ca n ron v truyn ti cc n ron khc c
ni vi u ra ca chng.
im ni gia cc dy thn kinh vo v ra gi l cc khp. Tn hiu truyn trong
cc dy thn kinh l tn hiu in, dng in l dng chuyn ng ca cc ion.
Ti khp thn kinh, cc tn hiu in u ra s kch thch lm gii phng cc cht
ha hc, c gi l cht truyn n ron. Cc cht truyn n ron s khuch tn qua
khe h khp v xuyn qua mng khp. Cc cht ha hc ny li kch thch to ra cc
tn hiu in u vo ca cc n ron khc.
Khp thn kinh hot ng mt chiu, tc l ch c cc tn hiu in u ra mi to
thnh c cc tn hiu in u vo ca cc n ron khc m khng c chiu ngc li. Hn
na ti cc khp cn c t l bin i thng tin gia u vo v u ra, gi l khuch i
lp.
Hnh 1: M hnh n ron sinh hc
Nh vy nron sinh hc hot ng theo cch thc sau: nhn tn hiu u vo, x l
cc tn hiu ny v cho ra mt tn hiu output. Tn hiu output ny sau c
truyn i lm tn hiu u vo cho cc nron khc.
Tt c cc c im trn u c vn dng mt cch trit trong vic xy dng mt mng
n ron nhn to nhm to ra mt mng n ron ging vi mng n ron sinh hc
nht.
III. MNG NRON NHN TO
1. Khi nim:Mng nron nhn to, Artificial Neural Network (ANN) gi
tt l mng nron l mt m hnh x l thng tin phng theo cch thc x l thng
tin ca cc h nron sinh hc. N c to ln t mt s lng ln cc phn t (gi l
phn t x l hay nron) kt ni vi nhau thng qua cc lin kt (gi l trng s
lin kt) lm vic nh mt th thng nht gii quyt mt vn c th no .
Mt mng nron nhn to c cu hnh cho mt ng dng c th (nhn dng mu, phn
loi d liu, ...) thng qua mt qu trnh hc t tp cc mu hun luyn. V bn
cht hc chnh l qu trnh hiu chnh trng s lin kt gia cc nron.
Mt nron l mt n v x l thng tin v l thnh phn c bn ca mt mng nron.
Cu trc ca mt nron c m t trn hnh di.
Hnh 2: M hnh nron nhn toCc thnh phn c bn ca mt nron nhn to bao
gm:
- Tp cc u vo: L cc tn hiu vo (input signals) ca nron, cc tn hiu
ny thng c a vo di dng mt vector n chiu.
- Tp cc lin kt: Mi lin kt c th hin bi mt trng s (gi l trng s lin
kt Synaptic weight). Trng s lin kt gia tn hiu vo th j vi nron i
thng c k hiu l wij. Thng thng, cc trng s ny c khi to mt cch ngu
nhin thi im khi to mng v c cp nht lin tc trong qu trnh hc mng. Trng
s kt ni: W = [w1, w2, , wn]T - B tng (Summing function) (u): Thng
dng tnh tng ca tch cc u vo vi trng s lin kt ca n. - Ngng (cn gi l
lch - bias): K hiu: Ngng ny thng c a vo nh mt thnh phn ca hm
truyn.
- Hm truyn (Transfer function) : Hm ny c dng gii hn phm vi u ra
ca mi nron. N nhn u vo l kt qu ca hm tng v ngng cho. Thng thng, phm
vi u ra ca mi nron c gii hn trong on [0,1] hoc [-1, 1]. Cc hm truyn
rt a dng, c th l cc hm tuyn tnh hoc phi tuyn. Vic la chn hm truyn
no l tu thuc vo tng bi ton v kinh nghim ca ngi thit k mng.
- u ra (Output): L tn hiu u ra ca mt nron, vi mi nron s c ti a l
mt u ra.
V bn cht mt mng nron c chc nng nh l mt hm nh x F: X Y, trong X l
khng gian trng thi u vo (input state space) v Y l khng gian trng
thi u ra (output state space) ca mng. Cc mng ch n gin l lm nhim v
nh x cc vector u vo x X sang cc vector u ra y Y thng qua b lc
(filter) cc trng s. Tc l y = F(x) = f(W, x), trong W l ma trn trng
s lin kt. Hot ng ca mng thng l cc tnh ton s thc trn cc ma trn.
i vi mi n ron chng ta tnh mt gi tr Net, l hm ca cc tn hiu u vo
xi v trng s wi:
Net = w1x1 + w2x2 + + wnxn
Tn hiu ra ti mi n ron l mt hm s ca Net:
Y =f(Net)Hm truyn f thng s dng trong cc m hnh mng nron nh sau:Hm
truyn thnh ngha
Symmetrical Hard Limit (hardlims)
Linear (purelin) y = f (u) = . u l h s gc ca hm
Saturating Linear (satlin)
Log-Sigmoid (logsig) y=f(u)
Nh vy tng t nh nron sinh hc, nron nhn to cng nhn cc tn hiu u vo,
x l (nhn cc tn hiu ny vi trng s lin kt, tnh tng cc tch thu c ri gi
kt qu ti hm truyn) v cho mt tn hiu u ra (l kt qu ca hm truyn).
2. Cc kin trc mng:Mc d mi nron n l c th thc hin nhng chc nng x l
thng tin nht nh, sc mnh ca tnh ton nron ch yu c c nh s kt hp cc
nron trong mt kin trc thng nht. Mt mng nron l mt m hnh tnh ton c xc
nh qua cc tham s: kiu nron (nh l cc nt nu ta coi c mng nron l mt
th), kin trc kt ni (s t chc kt ni gia cc nron) v thut ton hc (thut
ton dng hc cho mng).
Cch thc kt ni cc nron trong mng xc nh kin trc (topology) ca mng.
Cc nron trong mng c th kt ni y (fully connected) tc l mi nron u c
kt ni vi tt c cc nron khc, hoc kt ni cc b (partially connected)
chng hn ch kt ni gia cc nron trong cc tng khc nhau. Ngi ta chia ra
hai loi kin trc mng chnh:
- T kt hp (autoassociative): l mng c cc nron u vo cng l cc nron
u ra. Mng Hopfield l mt kiu mng t kt hp.
Hnh 3: Mng t kt hp- Kt hp khc kiu (heteroassociative): l mng c
tp nron u vo v u ra ring bit. Perceptron, cc mng Perceptron nhiu
tng (MLP: Multi Layer Perceptron), mng Kohonen, thuc loi ny.
Hnh 4: Mng kt hp khc kiuNgoi ra ty thuc vo mng c cc kt ni ngc
(feedback connections) t cc nron u ra ti cc nron u vo hay khng, ngi
ta chia ra lm 2 loi kin trc mng.
- Kin trc truyn thng (feedforward architechture): l kiu kin trc
mng khng c cc kt ni ngc tr li t cc nron u ra v cc nron u vo; mng
khng lu li cc gi tr output trc v cc trng thi kch hot ca nron. Cc
mng nron truyn thng cho php tn hiu di chuyn theo mt ng duy nht; t u
vo ti u ra, u ra ca mt tng bt k s khng nh hng ti tng . Cc mng kiu
Perceptron l mng truyn thng.
Hnh 5: Mng truyn thng- Kin trc phn hi (Feedback architecture): l
kiu kin trc mng c cc kt ni t nron u ra ti nron u vo. Mng lu li cc
trng thi trc , v trng thi tip theo khng ch ph thuc vo cc tn hiu u
vo m cn ph thuc vo cc trng thi trc ca mng. Mng Hopfield thuc loi
ny.
Hnh 6: Mng phn hi3. Perceptron
Perceptron l mng nron n gin nht, n ch gm mt nron, nhn u vo l
vector c cc thnh phn l cc s thc v u ra l mt trong hai gi tr +1 hoc
-1.
Hnh 7: Perceptronu ra ca mng c xc nh nh sau: mng ly tng c trng s
cc thnh phn ca vector u vo, kt qu ny cng ngng b c a vo hm truyn
(Perceptron dng hm Hard-limit lm hm truyn) v kt qu ca hm truyn s l
u ra ca mng. 3.1 Lut hc Perceptron:
Cho tp mu hc vi vecto vo X v 1 u ra l tng tng ng d(k). i vi nhim
v phn lp th d(k) thng l 1 hoc -1. Lut hc:
- Bt u vi cc trng s ngu nhin.
- Ly mt vecto vo x t tp hun luyn.
- Nu u ra tnh ton yk # d(k) (tc l perceptron p ng sai) th iu
chnh trng mi trng s Wi:
(wi = c(dk yk)xi.
3.2 Gii thut hc Perceptron:
- Vi mi tp cc mu hc D={(x,d)}x l vecto u vo
d l gi tr u ra mong mun (-1 hoc 1)
- Vi mt v d hc x c perceptron phn lp chnh xc, th vecto trng s w
khng thay i.
- Nu d=1 nhng perceptron li sinh ra -1 (Out=-1) th w cn c thay i
sao cho gi tr New(w,x) tng ln.
- Nu d=-1 nhng perceptron li sinh ra 1 (Out=1) th w cn c thay i
sao cho gi tr New(w,x) gim i.
- Perceptron s iu chnh trng s trn thnh phn th I ca vecto u vo mt
lng (wi = c(d f(wixi))xi.
f(wixi)) chnh l gi tr u ra ca perceptron, n c gi tr +1 hoc -1.-
Hiu gia d v f(wixi)) l 0, 2 hoc -2. V vy vi mi thnh phn ca vecto u
vo:
+ Nu gi tr u ra mong mun v gi tr u ra tnh ton t bng nhau th khng
lm g c.
+ Nu gi tr u ra thc l -1 v 1 l gi tr mong mun th tng trng s ca
ng th i ln (2cxi).
+ Nu gi tr u ra thc l 1 v -1 l gi tr mong mun th gim trng s ca
ng th i (-2cxi).
- c c gi l hng s th hin tc hc v nu c ln th cc gi tr iu chnh (wi
s ln, nh vy y nhanh qu trnh wi hi t v gi tr ng ca n.Perceptron cho
php phn loi chnh xc trong trng hp d liu c th phn chia tuyn tnh (cc
mu nm trn hai mt i din ca mt siu phng). N cng phn loi ng u ra cc hm
AND, OR v cc hm c dng ng khi n trong m u vo ca n ng (n m). N khng
th phn loi c u ra ca hm XOR.
4. Mng nhiu tng truyn thng (MLP)
M hnh mng nron c s dng rng ri nht l m hnh mng nhiu tng truyn
thng (MLP: Multi Layer Perceptron). Mt mng MLP tng qut l mng c n
(n2) tng (thng thng tng u vo khng c tnh n): trong gm mt tng u ra
(tng th n) v (n-1) tng n.
Hnh 8: Mng MLP tng qutKin trc ca mt mng MLP tng qut c th m t nh
sau:
- u vo l cc vector (x1, x2, ..., xn) trong khng gian n chiu, u
ra l cc vector (y1, y2, ..., ym) trong khng gian m chiu. i vi cc bi
ton phn loi, n chnh l kch thc ca mu u vo, m chnh l s lp cn phn loi.
Mi nron thuc tng sau lin kt vi tt c cc nron thuc tng lin trc n.
- u ra ca nron tng trc l u vo ca nron thuc tng lin sau n.
Hot ng ca mng MLP nh sau: ti tng u vo cc nron nhn tn hiu vo x l
(tnh tng trng s, gi ti hm truyn) ri cho ra kt qu (l kt qu ca hm
truyn); kt qu ny s c truyn ti cc nron thuc tng n th nht; cc nron ti
y tip nhn nh l tn hiu u vo, x l v gi kt qu n tng n th 2; qu trnh
tip tc cho n khi cc nron thuc tng ra cho kt qu.
* Mt s kt qu c chng minh:
- Bt k mt hm Boolean no cng c th biu din c bi mt mng MLP 2 tng
trong cc nron s dng hm truyn sigmoid.
- Tt c cc hm lin tc u c th xp x bi mt mng MLP 2 tng s dng hm
truyn sigmoid cho cc nron tng n v hm truyn tuyn tnh cho cc nron tng
ra vi sai s nh ty .
- Mi hm bt k u c th xp x bi mt mng MLP 3 tng s dng hm truyn
sigmoid cho cc nron tng n v hm truyn tuyn tnh cho cc nron tng ra.
khc phc nhng kh khn i vi nhng bi ton c mu phn chia khng tuyn tnh,
mng nron nhiu lp c s dng. C rt nhiu cng trnh nghin cu vmng MLP v
cho thy nhiu u im ca mng ny.
Mng MLP l mt gii php hu hiu cho vic m hnh ho, c bit vi qu trnh
phc tp hoc c ch cha r rng. N khng i hi phi bit trc dng hoc tham s.
Mng MLP l c s cho thut ton lan truyn ngc v kh nng xp x lin tc.
5. Mng Hopfield:
Mng Hopfield l mt dng mng nron nhn to truyn ngc do John Hopfield
a ra. Mng Hopfield ng vai tr nh cc h thng b nh c th nh a ch ni dung
vi cc nt ngng dng nh phn.- Bo m s hi t v mt cc tiu cc b, nhng khng
m bo s hi t v mt trong cc mu c lu tr.- Gm 1 lp n cc nron kt ni y
.
- Cc nron lin kt vi mi nron khc nhng khng t kt ni.
6. Xy dng mng nron:
6.1 Chun b d liu
a. Kch thc mu Khng c nguyn tc no hng dn kch thc mu phi l bao
nhiu i vi mt bi ton cho trc. Hai yu t quan trng nh hng n kch thc
mu:
- Dng hm ch: khi hm ch cng phc tp th kch thc mu cn tng.
- Nhiu: khi d liu b nhiu (thng tin sai hoc thiu thng tin) kch
thc mu cn tng.
i vi mng truyn thng (feedforward), cho hm ch c phc tp nht nh, km
mt lng nhiu nht nh th chnh xc ca m hnh lun c mt gii hn nht nh. C th
cn tp mu v hn t n gii hn chnh xc. Ni cch khc chnh xc ca m hnh l hm
theo kch thc tp mu. Khi kch thc mu tng, chnh xc s c ci thin - lc u
nhanh, nhng chm dn khi tin n gii hn.
Dng tng qut ca mi lin h gia sai s v kch thc mu nh sau:
Hnh 12: Mi lin h gia sai s v kch thc muTrong thc hnh thng gp phi
2 vn sau :
- i vi hu ht bi ton thc t, mu b rng buc cht ch vi d liu c sn. Ta
thng khng c c s lng mu mong mun.
- Kch thc mu cng c th b gii hn bi b nh hoc kh nng lu tr ca my
tnh. Nu tt c cc d liu ng thi c gi trong b nh sut thi gian luyn, kch
thc b nh my tnh s b chim dng nghim trng.
Nu lu tr trn a s cho php dng mu ln hn nhng thao tc c a t th h ny
sang th h khc khin cho tin trnh chm i rt nhiu.
Ch : vic tng kch thc mu khng lm tng thi gian luyn. Nhng tp mu ln
hn s yu cu t th h luyn hn. Nu ta tng gp i kch thc ca mu, mi th h
luyn s tn thi gian khong gp i, nhng s th h cn luyn s gim i mt na.
iu ny c ngha l kch thc mu (cng c ngha l chnh xc ca m hnh) khng b
gii hn bi thi gian luyn.
Lut c bn l: S dng mu ln nht c th sao cho kh nng lu tr trong b nh
trong (nu lu tr ng thi) hoc trn a t (nu thi gian c t a).
b. Mu con Trong xy dng m hnh cn chia tp mu thnh 2 tp con: mt xy
dng m hnh gi l tp hun luyn (training set), v mt kim nghim m hnh gi
l tp kim tra (test set). Thng thng dng 2/3 mu cho hun luyn v 1/3
cho kim tra. iu ny l trnh tnh trng qu khp (overfitting).
c. S phn tng mu Nu ta t chc mu sao cho mi mu trong qun th u c c
hi nh nhau th tp mu c gi l tp mu i din. Tuy nhin khi ta xy dng mt
mng xc nh xem mt mu thuc mt lp hay thuc mt loi no th iu ta mong mun
l cc lp c cng nh hng ln mng, t c iu ny ta c th s dng mu phn tng. Xt
v d sau[1]:
Gi s ta xy dng m hnh nhn dng ch ci vit tay ting Anh, v ngun d
liu ca ta c 100.000 k t m mi k t c km theo mt m cho bit n l ch ci
no. Ch ci xut hin thng xuyn nht l e, n xut hin 11.668 ln chim khong
12%; ch ci xut hin t nht l ch z, ch c 50 ln chim 0,05%.
Trc ht do gii hn ca b nh my tnh, gi s b nh ch c th x l c 1300
mu. Ta to hai dng tp mu: tp mu i din v tp mu phn tng. Vi tp mu i
din, ch e s xut hin 152 ln (11,67% ca 1300) trong khi ch z ch xut
hin mt ln (0,05% ca 1300). Ngc li ta c th to tp mu phn tng mi ch c
50 mu. Ta thy rng nu ch c th dng 1300 mu th tp mu phn tng s to ra m
hnh tt hn. Vic tng s mu ca z t 1 ln 50 s ci thin rt nhiu chnh xc ca
z, trong khi nu gim s mu ca e t 152 xung 50 s ch gim cht t chnh xc
ca e.
By gi gi s ta dng my tnh khc c b nh x l mt lng mu gp 10 ln, nh
vy s mu s tng ln 13000. R rng vic tng kch thc mu s gip cho m hnh
chnh xc hn. Tuy nhin ta khng th dng tp mu phn tng nh trn na v lc ny
ta s cn ti 500 mu cho ch z trong khi ta ch c 50 mu trong ngun d
liu. gii quyt iu ny ta to tp mu nh sau: tp mu gm tt c cc ch him vi
s ln xut hin ca n v km thm thng tin v ch c nhiu mu nht. Chng hn ta
to tp mu c 50 mu ca ch z ( l tt c) v 700 mu ca ch e (ch m ta c nhiu
mu nht).
Nh vy trong tp mu ca ta, ch e c nhiu hn ch z 14 ln. Nu ta mun cc
ch z cng c nhiu nh hng nh cc ch e, khi hc ch z ta cho chng trng s
ln hn 14 ln. lm c iu ny ta c th can thip cht t vo qu trnh lan truyn
ngc trn mng. Khi mu hc l ch z, ta thm vo 14 ln o hm, nhng khi mu l
ch e ta ch thm vo 1 ln o hm. cui th h, khi cp nht cc trng s, mi ch
z s c nh hng hn mi ch e l 14 ln, v tt c cc ch z gp li s c nh hng
bng tt c cc ch e.
d. Chn bin Khi to mu cn chn cc bin s dng trong m hnh. C 2 vn cn
quan tm:
- Cn tm hiu cch bin i thng tin sao cho c li cho mng hn: thng tin
trc khi a vo mng cn c bin i dng thch hp nht, mng t c hiu xut cao
nht. Xt v d v bi ton d on mt ngi c mc bnh ung th hay khng. Khi ta c
trng thng tin v ngi ny l ngy thng nm sinh. Mng s t c hiu qu cao hn
khi ta bin i trng thng tin ny sang thnh tui. Thm ch ta c th quy tui
v mt trong cc gi tr: 1 = tr em (di 18), 2 = thanh nin (t 18 n di
30), 3 = trung nin (t 30 n di 60) v 4 = gi (t 60 tr ln).
- Chn trong s cc bin c bin i, bin no s c a vo m hnh: khng phi bt
k thng tin no v mu cng c li cho mng. Trong v d d on ngi c b ung th
hay khng trn, nhng thuc tnh nh ngh nghip, ni sinh sng, tiu s gia
nh, l nhng thng tin c ch. Tuy nhin nhng thng tin nh thu nhp, s con
ci, l nhng thng tin khng cn thit.
6.2 Xc nh cc tham s cho mng:
a. Chn hm truyn Khng phi bt k hm truyn no cng cho kt qu nh mong
mun. tr li cho cu hi hm truyn nh th no c coi l tt ? l iu khng h n
gin. C mt s quy tc khi chn hm truyn nh sau:
- Khng dng hm truyn tuyn tnh tng n. V nu dng hm truyn tuyn tnh
tng n th s lm mt vai tr ca tng n : Xt tng n th i:
Tng trng s ni = wiai-1 + bi
ai = f(ni) = wf ni +bf (hm truyn tuyn tnh)
Khi : tng trng s ti tng th (i + 1)
ni+1 = wi+1ai + bi+1
= wi+1[wf ni +bf] + bi+1
= wi+1 [wf(wiai-1 + bi) + bf] + bi+1
= Wai-1 + b
Nh vy ni+1 = Wai-1 + b, v tng i khng cn gi tr na.
- Chn cc hm truyn sao cho kin trc mng nron l i xng (tc l vi u vo
ngu nhin th u ra c phn b i xng). Nu mt mng nron khng i xng th gi tr
u ra s lch sang mt bn, khng phn tn ln ton b min gi tr ca output. iu
ny c th lm cho mng ri vo trng thi bo ha, khng thot ra c.
Trong thc t ngi ta thng s dng cc hm truyn dng S. Mt hm s(u) c gi
l hm truyn dng S nu n tha mn 3 tnh cht sau:
s(u) l hm b chn: tc l tn ti cc hng s C1 C2 sao cho: C1 s(u) C2
vi mi u.
s(u) l hm n iu tng: gi tr ca s(u) lun tng khi u tng. Do tnh cht
th nht, s(u) b chn, nn s(u) s tim cn ti gi tr cn trn khi u dn ti
dng v cng, v tim cn gi tr cn di khi u dn ti m v cng.
s(u) l hm kh vi: tc l s(u) lin tc v c o hm trn ton trc s.
Mt hm truyn dng - S in hnh v c p dng rng ri l hm Sigmoid.
b. Xc nh s nron tng n Cu hi chn s lng noron trong tng n ca mt
mng MLP th no l kh, n ph thuc vo bi ton c th v vo kinh nghim ca nh
thit k mng. Nu tp d liu hun luyn c chia thnh cc nhm vi cc c tnh tng
t nhau th s lng cc nhm ny c th c s dng chn s lng nron n. Trong trng
hp d liu hun luyn nm ri rc v khng cha cc c tnh chung, s lng kt ni c
th gn bng vi s lng cc mu hun luyn mng c th hi t. C nhiu ngh cho vic
chn s lng nron tng n h trong mt mng MLP. Chng hn h phi tha mn
h>(p-1)/(n+2), trong p l s lng mu hun luyn v n l s lng u vo ca
mng. Cng nhiu nt n trong mng, th cng nhiu c tnh ca d liu hun luyn s
c mng nm bt, nhng thi gian hc s cng tng.
Mt kinh nghim khc cho vic chn s lng nt n l s lng nt n bng vi s
ti u cc cm m (fuzzy clusters). Pht biu ny c chng minh bng thc
nghim. Vic chn s tng n cng l mt nhim v kh. Rt nhiu bi ton i hi nhiu
hn mt tng n c th gii quyt tt.
tm ra m hnh mng nron tt nht, Ishikawa and Moriyama (1995) s dng
hc cu trc c qun (structural leanrning with forgetting), tc l trong
thi gian hc ct b i cc lin kt c trng s nh. Sau khi hun luyn, ch cc
noron c ng gp vo gii quyt bi ton mi c gi li, chng s to nn b xng cho
m hnh mng nron.
c. Khi to trng s:Trng s thng c khi to bng phng php th sai, n
mang tnh cht kinh nghim v ph thuc vo tng bi ton. Vic nh ngh th no l
mt b trng stt cng khng h n gin. Mt s quy tc khi khi to trng s:- Khi
to trng s sao cho mng nron thu c l cn bng (vi u vo ngu nhin th sai
s lan truyn ngc cho cc ma trn trng s l xp x bng nhau):
|W1/W1| = |W2/W2| = |W3/W3|
Nu mng nron khng cn bng th qu trnh thay i trng s mt s ma trn l
rt nhanh trong khi mt s ma trn khc li rt chm, thm ch khng ng k. Do
cc ma trn ny t ti gi tr ti u s mt rt nhiu thi gian.
- To trng s sao cho gi tr kt xut ca cc nt c gi tr trung gian.
(0.5 nu hm truyn l hm Sigmoid). R rng nu ta khng bit g v gi tr kt
xut th gi tr gia l hp l. iu ny cng gip ta trnh c cc gi tr thi
qu.
Th tc khi to trng s thng c p dng:
B1: Khi to cc trng s nt n v cc trng s ca cc cung lin kt trc tip
gia nt nhp v nt xut, nu c gi tr ngu nhin, nh, phn b u quanh 0.
B2: Khi to mt na s trng s ca nt xut gi tr 1, v na kia gi tr
-1.
6.3 Vn lng qun (catastrophic forgetting)
Catastrophic forgetting l vn mt mng qun nhng g n hc c trong cc
mu trc khi ang hc cc mu mi. Nguyn nhn l do s thay i cc trng s theo
cc mu mi, nu nh cc mu c trong mt thi gian khng c a vo hun luyn.
trnh iu ny, ta thng thc hin vic hun luyn lun phin gia mu c v mu
mi.
Hnh 13: Hun luyn lun phin trn hai tp muXt v d mng c hun luyn lun
phin vi hai tp mu A v B (hnh 13). Ti mi chu k mng s hc tp mu A sau
hc tp mu B. Khi bc vo chu k th hai, li lc bt u hc tp mu A cao hn l
chu k th nht khi va hc xong tp A. iu ny l do gia hai ln hc tp mu A
mng hc tp mu B. Tuy nhin nu xt trn c chu k th li hun luyn s gim
xung. Tc l li lc bc vo chu k th ba s nh hn lc bc vo chu k th
hai.
C nhiu phng php hun luyn d liu mi. Chng hn sau khi mt s mu mi c
hc, mt vi mu c c chn ngu nhin trong s cc mu trc a vo hc. Vn s kh
khn hn khi cc mu c khng cn na. Khi cc mu gi (pseudoexamples) c th c
s dng lu gi cc trng s cng gn cc trng s trc cng tt.
6.4 Vn qu khp
a. Khi nim qu khp Vn qu khp xy ra khi mng c luyn qu khp (qu st)
vi d liu hun luyn (k c nhiu), nn n s tr li chnh xc nhng g c hc, cn
nhng g khng c hc th n khng quan tm. Nh vy mng s khng c c kh nng tng
qut ha.
V mt ton hc, mt gi thuyt (m hnh) h c gi l qu khp nu tn ti gi
thuyt h' sao cho:
1. Error train (h) < Error train (h')
2. Error test (h) > Error test (h')
b. Gii quyt qu khp Vn qu khp xy ra v mng c nng lc qu ln. C 3 cch
hn ch bt nng lc ca mng:
Hn ch s nt n
Ngn khng cho mng s dng cc trng s ln
Gii hn s bc luyn
Khi mng c luyn, n chuyn t cc hm nh x tng i n gin n cc hm nh x
tng i phc tp. N s t c mt cu hnh tng qut ha tt nht ti mt im no . Sau
im mng s hc m hnh ha nhiu, nhng g mng hc c s tr thnh qu khp. Nu ta
pht hin ra thi im mng t n trng thi tt nht ny, ta c th ngng tin trnh
luyn trc khi hin tng qu khp xy ra.
Ta bit rng, ch c th nh gi mc tng qut ha ca mng bng cch kim tra
mng trn cc mu n khng c hc. Ta thc hin nh sau: chia mu thnh tp mu
hun luyn v tp mu kim tra. Luyn mng vi tp mu hun luyn nhng nh k dng
li v nh gi sai s trn tp mu kim tra. Khi sai s trn tp mu kim tra tng
ln th qu khp bt u v ta dng tin trnh luyn.
Ch rng, nu sai s kim tra khng h tng ln, tc l mng khng c s nt n
qu khp. Khi mng s khng c s nt cn thit thc hin tt nht. Do vy nu hin
tng qu khp khng h xy ra th ta cn bt u li nhng s dng nhiu nt n hn.7.
Hun luyn mng nron:* Cc k thut hc:
Khi nim: Hc l qu trnh thay i hnh vi ca cc vt theo mt cch no lm
cho chng c th thc hin tt hn trong tng lai.
Mt mng nron c huyn luyn sao cho vi mt tp cc vector u vo X, mng c
kh nng to ra tp cc vector u ra mong mun Y ca n. Tp X c s dng cho
hun luyn mng c gi l tp hun luyn (training set). Cc phn t x thuc X c
gi l cc mu hun luyn (training example). Qu trnh hun luyn bn cht l s
thay i cc trng s lin kt ca mng. Trong qu trnh ny, cc trng s ca mng
s hi t dn ti cc gi tr sao cho vi mi vector u vo x t tp hun luyn,
mng s cho ra vector u ra y nh mong mun.
C ba phng php hc ph bin l hc c gim st (supervised learning), hc
khng gim st (unsupervised learning) v hc tng cng (Reinforcement
learning):
7.1 Hc c gim st:L qu trnh hc c s tham gia gim st ca mt thy gio.
Cng ging nh vic ta dy mt em nh cc ch ci. Ta a ra mt ch a v bo vi em
rng y l ch a. Vic ny c thc hin trn tt c cc mu ch ci. Sau khi kim
tra ta s a ra mt ch ci bt k (c th vit hi khc i) v hi em y l ch
g?
Vi hc c gim st, tp mu hun luyn c cho di dng D = {(x,t) | (x,t)
[IRn x Rm]}, trong : x = (x1, x2, ..., xn) l vector c trng n chiu
ca mu hun luyn v t = (t1, t2, ..., tm) l vector mc tiu m chiu tng
ng, nhim v ca thut ton l phi thit lp c mt cch tnh ton trn mng nh th
no sao cho vi mi vector c trng u vo th sai s gia gi tr u ra thc s
ca mng v gi tr mc tiu tng ng l nh nht. Chng hn mng c th hc xp x mt
hm t = f(x) biu din mi quan h trn tp cc mu hun luyn (x, t).
Xs
Ytnh Ys
Hnh 9: S k thut hc c gim st
Nh vy vi hc c gim st, s lp cn phn loi c bit trc. Nhim v ca thut
ton l phi xc nh c mt cch thc phn lp sao cho vi mi vector u vo s c
phn loi chnh xc vo lp ca n.
7.2 Hc khng gim st: L vic hc khng cn c bt k mt s gim st no.
Xs
Out
Tn hiu ra
Hnh 10: S k thut hc khng gim st
Trong bi ton hc khng gim st, tp d liu hun luyn c cho di dng: D =
{(x1, x2, ..., xn)}, vi (x1, x2, ..., xn) l vector c trng ca mu hun
luyn. Nhim v ca thut ton l phi phn chia tp d liu D thnh cc nhm con,
mi nhm cha cc vector u vo c c trng ging nhau.
Nh vy vi hc khng gim st, s lp phn loi cha c bit trc, v ty theo
tiu chun nh gi tng t gia cc mu m ta c th c cc lp phn loi khc
nhau.
7.3 Hc tng cng: i khi cn c gi l hc thng - pht (reward-penalty
learning), l s t hp ca c hai m hnh trn. Phng php ny c th nh sau: vi
vector u vo, quan st vector u ra do mng tnh c. Nu kt qu c xem l tt
th mng s c thng theo ngha tng cc trng s kt ni ln; ngc li mng s b
pht, cc trng s kt ni khng thch hp s c gim xung. Do hc tng cng l hc
theo nh ph bnh (critic), ngc vi hc c gim st l hc theo thy gio
(teacher).
Xs
Ytnh
Tn hiu
Y/N
Hnh 11: S k thut hc tng cngIV. NG DNG CA MNG NORONNgy nay, mng n
ron ngy cng c ng dng nhiu trong thc t. c bit l cc bi ton nhn dng
mu, x l, lc d liu, v iu khin. ng dng ca mng nron c chia thnh cc loi
sau:
- X l ngn ng : X l ngn ng t nhin
- Nhn dng mu: Nhn dng nh, Nhn ging ni, Nhn dng ch vit
- X l tn hiu: iu khin t ng
- Lc v phn loi d liu: Chun on bnh, Tm kimPHN 3: MNG LAN TRUYN
NGC SAI S
(BACK PROPAGATION OF ERROR BP)I. TNG QUAN V MNG LAN TRUYN NGC
LI:
Th gii thc xy ra tnh hung l d liu b thiu hoc b nhiu. a ra c d on
thch hp da trn nhng thng tin b thiu ny l rt kh (cha c mt l thuyt no
c th gip ti to li d liu b mt). Mng BP c th a ra c mt s cu tr li
thch hp.
Cu trc BP gm t nht 3 lp:
- Mt lp vo (Input Layer)
- t nht mt lp n gia (Hidden Layer)
- Mt lp ra (Output Layer)
Thng thng cc node u vo c kt ni y ti cc node lp n v cc node lp n
c kt ni y vi cc node trong lp u ra.
u ra ca mng lan truyn ngc c xem nh mt b phn lp quyt nh.
Vi mng lan truyn ngc, qu trnh hc xy ra trong sut mt chu k hun
luyn. Gm cc bc:
- Mi mu u vo trong tp mu hc c p dng cho cc node lp vo v sau c
lan truyn tin.
- Mu sau khi c lan truyn n lp u ra c so snh vi mu ra (u ra l
tng) tnh ton li u ra.
- Li ng vi mi mu u ra sau c lan truyn ngc t cc u ra ti cc u vo
nhm iu chnh cc trng s mt cch thch hp trong mi lp ca mng.
- Sau khi mng lan truyn ngc c hun luyn phn loi chnh xc cho cc tp
mu hc, n c th c kim tra trn mt tp mu cha qua hun luyn (kim tra nng
lc d bo ca mng). Nu nng lc d bo tt, mng c th c dng d bo.
Hnh 12: M hnh mng nron nhiu lp lan truyn ngcII. GII THUT HC
BP
1. Gii thut hun luyn mng:
Xt mng nron c 3 lp, lp input c 1 node, lp n c m node, lp output
c n node. Xt gii thut lan truyn ngc vi mng nron nhn to ny:
* Kin trc c s ca gii thut:
Khi to cc trng s
Repeat
For each mu hc
Hc vi mu ny
End
Until li mc nh chp nhn c.
- Bc 1: a cc gi tr vo (input), ra (output) v dng ma trn gi tr-
Bc 2: Gi nh s neural m ca lp hidden tha 1 p=[4.7 6.1 5.6 5.8
6.5;
3.2 2.8 3.0 2.7 3.2;
1.3 4.7 4.1 5.1 5.1;
0.2 1.2 1.3 1.9 2.0];
>> t=[0 1 1 0 0];
- To mng n-ron l net vi u vo l p v hm mc tiu t.
- To mng vi 2 n ron lp n v s dng hm truyn logsig cho c 2 lp:
>> net=newff(p,t,2,{'logsig' 'logsig'});Bc 2: Hun luyn
mng:- Truyn mu A vo mng:>> p1=[4.7;3.2;1.3;0.2];>>
t1=[0];a tp hun luyn vo mng bng lnh train():
>> net=train(net,p1,t1);
Bc 3: Tnh vecto u ra y ca mng:Sau khi hun luyn mng ta c th s dng
chng bng cch gi hm sim().V d s dng hm sim() vi vect u vo p ta s thu
c kt qu y l m phng ca t.>> y=sim(net,p1)y =
0.5000Bc 4: nh gi li:Hm nh gi li mc nh y l MSE (mean-square
error).>> mse(t1-y)
ans =
0.2500
Bc 5: Hiu chnh trng s:
Sau khi hun luyn, mng nron cho u ra y cha ging vi vecto mc tiu
t, mng tin hnh cp nht li trng s lin kt v ngng theo cng thc:wk+1 =
wk kgk
vi: wk l vecto trng s v ngng hin thi
k l h s hc hin thi
gk l gi tr gradient hin thi.Bc 6: Lp i lp li cc bc t bc 2 n bc 5
cho n khi mng t trng thi hi t.- Tip tc truyn mu B vo mng:
>> p2=[6.1;2.8;4.7;1.2];>> t2=[1];
>> net=train(net,p2,t2);
>> y=sim(net,p2)
y =
1.0000
>> mse(t2-y)
ans =
1.7515e-011- Truyn mu C vo mng:
>> p3=[5.6;3.0;4.1;1.3];
>> t3=[1];>> net=train(net,p3,t3);
>> y=sim(net,p3)
y =
1.0000
>> mse(t3-y)
ans =
2.1198e-011Chng ta c th thay i cc gi tr khi to ca mng v hun luyn
li mng c kt qu chnh xc nh mong i hn.C th thc hin o to mng hng lot
vi tp mu p, t a vo:
>> net=train(net,p,t);
>> y=sim(net,p)
y =
0.9954 0.6744 0.9414 0.5000 0.5001
>> mse(t-y)
ans =
0.3201
>> net=init(net);
>> net=train(net,p,t);
>> y=sim(net,p)
y =
0.5000 0.5000 0.5000 0.5000 0.5000
>> mse(t-y)
ans =
0.2500..3. Ci t bi ton s dng cng c nntool trong Matlab:
- Trong ca s Command Window g lnh: nntool
Ca s Network/ Data Manager hin ra:
- Tp tp d liu u vo v tp mc tiu, chn New Data:
Tp d liu vo t Name l Input, Data Type l inputs. Nhp d liu di dng
ma trn khung Value -> bm Create.
Tp mc tiu t Name l Target, Data Type l Targets. Nhp d liu di dng
ma trn vo khung Value.
- To mng: Trong ca s Network/Data Manager chn New Network:
t Network Name l CayIris, Get from input chn Input, Number of
layers l 2, number of neurons cho layer1 l 2, layer2 l 1 v Transfer
Function cho c 2 layer l logsig ->Bm Create.
- Bm View xem kin trc mng va c to:
- o to mng: bm vo CayIris trong khung Network ca ca s
Network/Data Manager -> chn Train -> Ca s Network CayIris hin
ra.
- Thit lp cc trng s lin kt v ngng cho mng trong tab Weights bng
Set Weight hoc s dng cc trng s do mng t ng khi to.
- Trong tab Train chn Training Data vi Inputs l Input, Targets l
Target -> Bm Train Network.
- Sau khi o to mng, Output v Errors s c a ra trn ca s
Network/Data Manager. xem c, bm chn vo CayIris_outputs hoc
CayIris_errors -> chn View:
- Ta thy u ra output ca mng rt gn vi u ra mc tiu mong mun v li
gim dn.
mng t trng thi hi t, ngha l gi tr outputs ging vi tp mc tiu nht,
tin hnh o to li mng vi cc ma trn trng s v ngng c cp nht sau nhng ln
o to trc .KT LUN ti ny trnh by cc vn v cng ngh tnh ton mm v mng
nron, gm: tm hiu v tnh ton mm, khi nim ca mng n ron nhn to, lch s
pht trin, cc m hnh mng v phng php xy dng cng nh hun luyn mng. Trong
i su vo vic xy dng mt mng nron truyn thng MLP, v thut tun hun luyn
Lan truyn ngc.
ti ny cng trnh by mt thc nghim cho cc l thuyt v mng MLP v thut
ton Lan truyn ngc nu ra, ci t minh ha bi ton bng ngn ng Matlab.
Hng pht trin ca ti: Tm hiu v cc thut ton hun luyn mng n ron khc,
c th a ra cc so snh, cng nh chn m hnh thch hp cho cc bi ton c
th.
Pht trin chng trnh thc nghim thnh mt chng trnh c ngha thc t h nh
phn loi mu, nhn dng ch vit tay, nhn dng nh, da trn nn tng mng xy
dng.
Chng em rt mong nhn c s ng gp kin ca cc thy c gio trong khoa v
cc bn sinh vin ti ca chng em c hon thin hn na trong tng lai.
Chng em xin chn thnh cm n!
TI LIU THAM KHO
[1]. Genevieve Orr, Nici Schraudolph and Fred Cummins
http://www.willamette.edu/~gorr/classes/cs449/intro.html [2].
Christos Stergiou and Dimitrios Siganos. Neural Networks.
http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html
[3]. Nikola K. Kasabov. Foundations of Neural Networks, Fuzzy
Systems, and Knowledge Engineering. Massachusetts Institute of
Technology.[4]. H m, mng nron v ng dng - Bi Cng Cng Nguyn Don
Phc.[5].http://www4.hcmut.edu.vn/~huynhqlinh/TinhocDC/WebLQNguyen/noron%20nhan%20tao/feedforward_1.html
KIN NHN XT CA GING VIN HNG DNN1
N2
N3
N4
N1>N4
N1>N3
N3>N1
N3>N2
N3>N4
N4>N3
N4>N2
N4>N1
N2>N4
N2>N3
N1>N2
N2>N1
Mng nron N
Hiu chnh W
Sai s
Mng n ron N
Mng nron N
Hiu chnh W
Tn hiu tng cng
Bias
Hidden Layer
Input Layer
Output Layer
Lan truyn tng mu hc
Hiu chnh trng s
Ht mu hc
s vng
WHide &Wlast & RMS
Ht
Cha
Cha
3.2
1.3
0.2
T0 = 0
4.7
iNhm 11 LTKHMT2K6 1 Ging vin: Nguyn Phng Nga