Ageing Adaptation for Multimodal Biometrics using Adaptive Feature Set Update Algorithm

2009 IEEE International Advance Computing Conference (IACC 2009)Patiala, India, 6-7 March 2009

Ageing Adaptation for Multimodal Biometrics usingAdaptive Feature Set Update Algorithm

H B Kekre, V A BharadiNMIMS University, Mumbai, Maharashtra, India - 400056

Abstract-- Multimodal Biometrics is an emerging domain in claimed identity, identification involve comparing thebiometric technology where more than one biometric trait is acquired biometric information against templatescombined to improve the performance. The biometric system corresponding to all users in the database [3].take Face, Fingerprint, Voice, Handwritten Signatures, Retina, Here we discuss a different issue for biometric systems. TheIris, Gait, Palm print, Ear & Hand geometry as common enrollment of a user for biometric authentication requiresfeatures. Human is identified by correct matching of thesefeatures. However, features like face, voice, and signature have acquisition of biometric trait and feature extraction. Forlow permanence and they change with time. Ageing of human, as example we take photographs for face recognition, fingerprintwell as other psychological & environmental conditions cause scan for fingerprint identification, depending on differentgradual change in these features. While enrolling feature set we system different feature set needs to be enrolled. A lot ofdon't consider this factor. research work is done on feature extraction & representation.

Here we propose a new concept that can be used in designing As biometric systems use feature related to human body thesefuture multimodal biometrics systems which can adapt to the features are dependent physical condition of human bodychange in the biometrics features like face, voice, signature, and which change with age. One more thing is that, not all thegait over the time or any other factor without compromising the features change, features like fingerprint, Iris, retina have verysecurity. Regression based technique can be used to detectchange. This algorithm requires use of at least one biometric hegrese ofapermance,san thaedono vary wit tiebfeature which has very low variance or high degree of features like face, voice, signature have high variance and theypermanence, like Fingerprint, Iris, Retina etc. This algorithm can change with time. This change is due to the Ageing of humanaddress the problem of false rejection caused by sustained body, psychological, environmental or other factors. Therechange in biometric features due to Ageing or any other factor changes may be sustained or for a short span of time.without the need of re-enrollment of feature set. Current biometric system has user enrollment procedures

which take these features from a dataset collected over a veryI. INTRODUCTION short period of time; this may be a time span of few days to

Multimodal Biometrics is a combination of more than one few weeks. It is also not practical to collect a feature set for abiometric feature for human identification. A biometric is a time span of years. We present here a mechanism for enrollingbiological measurement of any human physiological or feature set for multimodal biometric system so that it can

behavior characteristics that can be used to verify the identity adapt to change in biometric trait due to Ageing or any otherof an individual. Unimodal Biometric systems suffer from factor. This mechanism is applicable for only multimodalseveral problems like noisy sensor data, non-universality, lack biometric system using more than one biometric trait.of individuality, non-availability of invariant representations,etc, [1]. These problems are responsible for an increase inerror rates and decrease in system reliability for high securityneeds. Multimodal biometric systems overcome some of the We consider here some standard databases used inproblems associated with unimodal biometric systems by dbio e.combining the decisions from different biometrics using an database.effective fusion rule, thus achieving higher accuracy and A. AR Face Databasebetter performance [2]. The AR face [4], [5] contains over 4,000 color face imagesA biometric-based authentication system operates in two of 126 people (70menand 56 women), including frontal views

modes: Enrollment and Authentication. In the enrollment of faces with different facial expressions, lighting conditions,mode, a user's biometric data is acquired using a biometric and occlusions. Tread and stored in a database. The stored biometric template is 55 w omn) weetakenrin tof sionsu(seaaedbtwlabeled with a user identity to facilitate authentication. In the .wek )and eaetincontai orimages. Twetyauthentication mode, a user's biometric data is once again fee iag each session containi 10c o thes.en12acquired and the system uses this to either verify the claimed individuals are selected and used in our experiment. The faceidentity of the user or identify who the user is. While portion of each image is manually cropped and thenverification involves comparing the acquired biometric normalized to 50 X 40 pixels. The sample images of oneinformation with only those templates corresponding to the person are shown in Fig. 1. These images vary as follows:

978-1-4244-2928-8/09/$25.00 ( 2009 IEEE 535

each database to be used in the FVC2002 test, however, isestablished as 110 fingers, 8 impressions per finger (880impressions) (Fig. 2). Collecting some additional data gave usa margin in case of collection errors, and also allowed us tosystematically choose from the collected impressions those toinclude in the test databases.

Fig. 1. Example images from the AR Face database.

Another example is Facie database is FERET database [6],using similar data collection mechanism the example is asshown below,

Fig. 3. Example Fingerprints from the FVC 2002

C. Signature DatabaseWe consider a database collected for signature recognition

in [8]; this database was collected from 100 people, in moreFigure 2. Example images from the FERET Face database. than one session. The database was collected over a time span

of average two weeks. A typical record is shown below in Fig.4. There are various biometrics features databases available

Organizers of Fingerprint Verification Competition 2002 over the internet for research purpose. All these databases arehave designed their own database [7]. Four databases collected over a time span of few days to months. Theseconstitute the FVC2002 benchmark. databases do not consider the variations in the feature set dueThree different scanners and the SFinGE synthetic to Ageing. Once the person is enrolled to the system thegenerator was used to collect fingerprints. Fig. 3 shows an biometric feature set is static. In the next section we discussimage for each database. This database mainly concerns the change in biometric features due to various factors.hardware variations, different scanners were used forgenerating database. The details for scanners are as follows.* DBI Optical Identix TouchView II 388374 - 500 dpia a* DB2 Optical Biometrika FX2000 296 560 - 569 dpi _* DB3 Capacitive Precise Biometrics 100 SC 300 300 - 500 dpi* DB4 Synthetic SFinGE v2.51 288 384 - 500 dpiEA total of ninety students (20 years old on the average)

enrolled in the first two years of the Computer Science degreeprogram at the University of Bologna kindly agreed to act asvolunteers for providing fingerprints. Each volunteer wasinvited to present him/herself at the collection place in threedistinct sessions, with at least two weeks time separating each Xsession. At the end of the data collection, we had collected foreach database a total of 120 fingers and 12 impressions perfinger (1440 impressions) using 30 volunteers. The size of Fig. 4. Example Fingerprints from the Signature database used in [8].

536 2009 IEEE Internlationlal Advance Computing Conference (IACC 2009)

III. CHANGE IN BIOMETRIC FEATURE DUE TO AGEING A. Facial FeaturesAs we grow up our physical appearance change as well Fig. 5 shows example of such phenomenon. Changes in

change occurs in behavioral characteristics. We know that as facial geometry of a person over a time span of 5 & 10 yeardefined previously biometrics is derived from measurable shown and we can see gradual changes occurring. Fig. 5 (b)physical and behavioral characteristics, the feature set of shows such changes in a female candidate. If a person isbiometrics changes with time. The features that undergo facing a biometric identification system for such a long timegradual changes are Facial features, Voice, Gait, and we have to design the system so that it can identify theseSignature. The features like fingerprint, Iris, Retina, DNA changes and update feature set.have high permanence and they do not change drastically. B. Signature, Voice & Gate

Similar changes can be observed in signature, voice tone, gaitof human being; these features have intra class variations aswell as variations due to Ageing and other conditions.

IV. NEED FOR ADAPTIVE FEATURE SET

From the previous discussion we can conclude tow points1. Biometric eatures like face, voice, signature, and

2004 2005 2006 psychological change or other environmental factor.2. Current feature set enrollment mechanisms are static

and the feature set need to be updated as thebiometric features change with time.

Here we consider extended application of multimodalbiometric systems, we can use this multimodal approach inmachines having Al and interacting with humans. We canimplement the multimodal biometrics system for humanidentification. We also consider that such human machine

2007 2008 Next What? interaction will be there for longer time (for years) over whichsome of the biometric features change gradually, this period

(a) and amount of variations is different for different feature anddifferent individual.We now present an algorithm for multimodal biometric

System for adaptive feature set updating process. Thisalgorithm divides the biometric feature sets in two categoriesas follows:1. Features with low permanence over time: Signature, Voice,Gait, Facial features.2. Features with high permanence over time: Fingerprints,Palm prin, Iri Reti, (DNA

( CaeififarfF en0 yearsThough the biometric feature as listed above change, they2002 2EEE Interrlatiorlal Advance are still unique for the individual and still can be used foridentification of the individual. We need to develop amechanism that can handle this. We must also consider thesecurity aspects, as the forgers should not be allowed tochange the feature set& accietludin shldbprohibited.

V. CONSIDERATIONs FOR ADAPTIVE FEATURE SET UPDATING

This algorithm is mainly aimed at multimodal biometric2006 208systems, based on more than one biometric feature. As the

(b) algorithm is adaptive, here we try to design a mechanism thatwill be able to track the change and update the feature set

Fig. 5. Effect of Agcing on Facial Feature scrl.Frti ehv odcd h uhniiyo h(b)Chagesinfacal eaureforFemlecanidae 10 ear) ,eaturelv os t.We definvefolown terms:e ubntct o h

A. Independent Feature Set (SI) Fused Srre S il I

While developing a multimodal biometric system we haveA G

to include at least one feature which will remain constant over (g hard-swi nwCnr and SVMtime. This feature set will be used to confirm the dependentfeature set update procedure. Independent feature set includesFingerprint, Palm Print, Iris, and Retinal Scan. S S(xm)

C(dlaf[rfr s erfirx[pndent 1ndedent

B. Dependent Feature Set (SD) . .a.This is a feature set which changes over time and need to I

be updated, this include voice, facial features, face, Signature xlk 1Vl T(Static & Dynamic) , Gait etc.

Fig. 6. Fusion network in a Mixture of Expert Architecture (MOE) [1 ].C. Allowed Degree of VarianceAllowed degree of variance in the biometric features for F. Interval of Update or Needfor Update

which updating is not needed, or the performance metrics are This is the time after which a biometric feature set is to bewithin required limits. For Signature, Voice this variance will updated or a condition indicating need for update. Thebe calculated based on a training mechanism as discussed in condition will be decided by classifier of the dependent[9] which is based on Euclidian distance model, and for facial feature. As shown in Fig. 6, we can see two classifiers,feature the facial regression technique [12] will be used. corresponding to an independent biometric feature and a

dependent biometric feature. The update will be triggered in

D. Scale Invariant Feature Transform following conditions

The SIFT features represent a compact representation of (a) Time limit is reachedthe local gray level structure, invariant to image scaling, We set a time period for checking need for update. Thistranslation, and rotation, and partially invariant to illumination time limit depends on the rate of change of dependentchanges and affine or 3D projections. SIFT has emerged as a biometric trait. At this time we do not have any fixed numbervery powerful image descriptor and its employment for face for but this can be decided by actual implementation, for aanalysis and recognition was systematically investigated in start we can set this to a time period of 1 Month, or even more[10] where the matching was performed using three than that. This is actually depends on Ageing of human beingtechniques: and corresponding change in the biometric feature.

(a) Minimum pair distance,(b) Matching eyes and mouth, and (b) IncreasedFRR in dependent biometric trait(c) Matching on a regular grid. Consider case of facial features. This can be due to sudden

change in facial geometry, arising from illness, facial makeup,The present system considers spatial, orientation and surgery or any other factor. Voices tone my change due to

keypoint descriptor information of each extracted SIFT point. illness or Ageing effect. This will lead to increased vectorThus for example the input to the feature extraction system is distance for feature set used while training and input vectorthe face image and the output is the set of extracted SIFT may be rejected. This change is to be observed over time andfeatures s = (s 1, S2........ Sm) where each feature point then only update should be triggered. Intelligent module issi=(x, y, 0, k) consist of the (x, y) spatial location, the local required to detect this. Considering all these factors we noworientation 0 and k is the keydescriptor [10] of size Ix128. put a formal description of the steps in Adaptive Feature SetCurrently we use the SIFT for facial feature and use of this for Algorithm.other biometric features is being studied and is an open issuefor research. G. Facial Regression

In [12] , Zou, Chellappa et al. have used a technique calledE. Fusion ofMultibiometric Features Image Based Regression (IBR) for Age estimation and Facial

Rather than performing fusion on sensor level or feature Recognition across Ageing Progression. They proposed alevel .

use heredecisionlevelfuo. We d nBayesian age difference classifier that is built on probabilisticlevel we use here decision level fusion. We designspcsfa

independent classifier for each biometric attribute and fuse the eigenspaces framework.decisions of the classifier.This allows us to update the feature In current algorithm we can use this classifier to detect thedecisions of the classifier. This allowss to update the Fure change in facial feature due to Ageing and can be used to

set. We use the scheme as dscussed Udate feature set dnamicall. The classifier need to benetwork in MOE (mixture-of-expert) architecture. Each vector mdfe o nerto ncretagrtmsequence iS compressed into a local score. The local scores arethen fused by a gating network as shown in Fig. 6.

538 2009 IEEE Internlationla/Advance Computing Conference (IACC 2009)

VI. ADAPTIVE FEATURE SET UPDATING ALGORITHM One Classifier which belongs to independent feature playsThis algorithm is working after the decision level of a an important role. Here we have multibiometric system and

biometric system; it takes input for different classifiers from fusion algorithms working for combining the decisions fromthe fusion network and monitors the results. The feature these classifiers are used for authenticity of user and if thevectors are extracted using Scale Invariant Feature Transform classifier giver input feature vector SI as correct then only the(SIFT) (For facial feature). We consider two distinct set of dependent feature vector SD can be updated if updatefeature vectors condition is triggered. Here we consider a case of multimodal(a) Set ofdependentfeature vectors biometrics system having two classifiers where one belongs to

SD = Isdl, sd2, sd3....sdm where sdi is a feature point as Fingerprint Verification (Operating on SI) and other belongsdiscussed in [9]. to Facial Feature Classifier (Operating on SD).With each feature vector we add a counter df corresponding A AF

to defaults, i.e. Number of failures while the independent A tVUAlgorithmefeature vector gives a correct classification. 1. Reset the time interval ti=O, default counter for dependent

feature df=O;(b) Set ofindependentfeature vectors 2. Start the multimodal biometric system. Read the input test

SI = {sil, si2, si2.....sim} vector ST={SDT,SIT} for user UnWhere ST= Test feature vector set

For a dependent feature set we define tu as time interval for SDT = Test Dependent Feature vectorupdate in days. We use a classifier based on a model as (Dependent Feature vector Subset)described in [9], which was initially used for classifying SIT = Test Independent Feature vector

1 1 * * * r r (~~~~~~~~~~~Independent Feature vector Subset)handwritten signatures. Here we use a variance of feature (I pe)vector points and Euclidian distance of input test vector for the 3. If Adaptive Gating Network Classifier REJECTS the test

median feature vector point for classification. We calculate the vector ST {SIT+SDT} then GO TO step 4. Else go to step 2variance (cGi), threshold (thi) for distance of input feature (Next Test).vector SDi for the dependent biometric feature. The threshold 4. If Independent Feature Classifier REJECTS the test feature

(thi) is specific for each dependent biometric feature used. vector (SIT) then go to Step 2 else GO TO Step 5.This is calculated for the feature vector set used for enrollment (REJECTED TEST VECTOR). NO NEED TO UPDATE.of a person and this indicates intra-class variation of the 5. If Dependent Feature Vector Classifier REJECTS thefeature. Any feature vector having distance from median feature vector ST{SDT} (While dependent feature vector isfeature point less than threshold is accepted and else rejected. ACCEPTED) Mark as default condition df=df+1 for user Un.This condition is monitored used for decision of update.

Dependent BDomei1c Feaiture Feature V&Ktar Extrant nEfloikmneflt (Scale Invariant Feature Classer for Dependent

Senwi & PreprocessingI NnsT ror) ntioFeati ure Set(Faca, Vbke Sionatura w.) Daabsli E nrblasn1TIm|l

Central Nature 1vat Fetue 0k1ID1 Ik ~ Uerr~fi

~~~~~~~~~~~~~~~~~~~~~~~nl Deiso sI trk

VKlor Vector~Update -4,

~I ddpendnt Bl3ometn. FeaureEiriniIk;ti Feature Vedctr EAracioIn Classifier idr Ii1 mentidri

Senso & PreprocssiAng Database EnroIIment Biometntr Fealure Seta(Fingprnt, Iris. Reinaet.)

tEtWFig. 7. Multmoda Bimeri Syte usn Adptv Fetr Veto Upat mechanism.iit

209IEEItentiona Advnc CoptpCinXg Cofrec (IC 209 53

6. If default count df > permitted defaults (DP) for user Un, Currently unimodal biometric systems are ubiquitous andINVOKE Feature vector UPDATE MODULE. Reset Default multimodal biometric systems are having low penetrationcounter df =0, for user Un. GO TO step 2 (This (df=O) will owing to high cost and complex structure [13]. But where weavoid unnecessary repeated update). need high accuracy and sustainability, this algorithm can7. If Age difference classifier detects valid change in surely help to improve performance of the system. Thebiometric feature, INVOKE Feature vector UPDATE application of this algorithm is in all multimodal biometricMODULE. Reset Default counter df=0, for user Un. GO TO systems, with very large time span for operation over astep 2. specific population like Schools, Offices, Hospitals, Military,8. If time interval to update ti > permitted limit (TP) of user Research Laboratory Access control equipments. AnotherUn then INVOKE the Feature Vector UPDATE MODULE. application is development of Machines with Al and highReset ti=0,df=0 for user Un. GO TO step 2. degree ofhuman interaction.

Implementation of this algorithm does not require change inB. Feature Vector Update Module any of the hardware (Sensors). Change in coding architecture

This is very important module, this works on the same lines is required and this can be integrated with existing work.as the enrollment module for each of the classifier. We operate Current multimodal biometric system designers shouldthe module on the FIFO basis, oldest feature vector is deleted consider this algorithm for implementation. This algorithm isand the latest Feature vector will be added. The new feature proposed for improving performance of multimodalvector will be added to the set and we calculate the variance biometrics systems and imparts adaptability towards change in(oi), threshold (th.) for distance of input feature vector SDi for biometric features due to Ageing by making these systemsthe dependent biometric feature with the included feature aware of gradual changes in human biometric trait over thevector. This forms the updated feature vector set (SDi) for time. Performance of such system should be tested for longeruser Un. time over a real time application scenario.The steps are as follows. This algorithm is not final and may need revisions as many1. Prompt user Un for UPDATE of Feature Vector stating the of the parameters are still unknown, like permitted number ofreason defaults (DP) and time interval to update (TP) & design of age2. If allowed by user, read again the feature vector for difference classifier need further research. Better featurecorresponding dependent feature vector. Find SIFT [10] as vector extraction, training & enrollment mechanisms can bediscussed earlier. Tag the feature vector with current date & combined with this algorithm for improving performance.time3. Delete the oldest feature vector from the set SD = {Sdl,sd2.. ..Sdn}. REFERENCES4. Using the training mechanism in [9] find out variance (i) [] K. Nandkumar, "Integration of Multiple Cues in biometric Systems", PHDthreshold (thi) for the updated set, Update the corresponding Thesis, Michigan State University, 2005.record. [2] Teddy Ko, "Multimodal Biometric Identification for Large User5. Prompt the user of the Completion of Update Process. Population Using Fingerprint, Face and Iris Recognition", Proceedings of

34th Applied Imagery and Pattern Recognition Workshop (AIPR), 2005.

C. Security in Updating Process [3] A. Ross and A.K. Jain, "Information Fusion in Biometrics", PatternRecognition Letter 24, pp.2115- 2125, 2003.As the feature vector update is triggered only when the [4] A.M. Martinez and R. Benavente, "The AR Face Database," CVCIndependent Feature Vector successfully ACCEPTED, this Technical Report #24, June 1998.adds security against false updating of feature set. We take [5] A.M. Martinez and R. Benavente, "The AR Face Database"fingerprint, Palm print, Retina, and Iris as Independent feature http://rvll.ecn.purdue.edu/aleix/-aleix-face-DB.html, 2006.

[6] P.J. Phillips, "The Facial Recognition Technology (FERET) Database,"vector, there biometrics features have high CCR (Correct http://www.itl.nist.gov/iad/humanid/feret/feret_master.html, 2006.Classification Rate) and very difficult to forge as compared to [7] D. Maio, D. Maltoni, R. Cappelli, J.L. Wayman, A.K. Jain, "FVC2002:the dependent features. As no other entry point for triggering Second Fingerprint Verification Competition", 16th ICPRthe updating (INVOKE UPDATE MODULE Event) process, [8] H B Kekre, V A Bharadi, A Ambardekar, "Signature Recognition by PixelVariance Analysis Using Multiple Morphological Dilations", Internationalwe can say that the update process is secure Journal of Information Retrieval (IJIR), International Sciences Press,

Vol. 1 no. 1 June 2008, pp 5-9, ISSN: 0974-6285VII. CONCLUSION AND FUTURE WORK [9] B. Majhi, Y S Reddy, D Prasanna Babu, "Novel Features for Off-line

Signature Verification", International Journal of Computers,The Adaptive feature vector algorithm is proposed for Communications & Control Vol. I, 2006development of complex multimodal biometrics systems with [10] M. Bicego, A. Lagorio, E. Grosso and M. Tistarelli, "On the use of SIFTlarge operating time span. The authors are in process of features for face authentication", Proceedings of CVPR Workshop, Newlarge operating ~~~~~~~~~~~~~York,2006.development of classifier and multimodal biometric system. [11] s. Y. Kung, Man-Wai Mak, "On Consistent Fusion Of MultimodalAt this moment we present the algorithm conceptually. This Biometrics", ICASSP 2006, June 2006.algorithm is very crucial as we will have more and more [12] S K Zhou, R. Chellappa, W. Zhao, "Unconstrained Face Recognition",multimodal biometric systems coming in human interaction. [1]Springer Science, 2006, ISBN-1O: 0-387-26407-8, pp 111-127

Fig.7. Showsarchitectue for proposed biometri A. K. Jamn, K. Nandakumar and A. Ross and, "Handbook ofFlg..Shws ach1tctur forpropsedmultimodal boerc Multibiometrics", Springer, ISBN - 978-0- 387-22296-7, 2006.system with Adaptive Feature Vector Update algorithm.

540 2009 IEEE Internlationlal Advance Computing Conference (IACC 2009)