Critical Analysis of Adaptive Biometric Systems Norman Poh a , Ajita Rattani * and Fabio Roli Department of Computing, FEPS, University of Surrey, Guildford, UK a Department of Electrical and Electronic Engineering, University of Cagliari Piazza d’Armi, Cagliari, Italy [email protected],{ajita.rattani,roli}@diee.unica.it December 18, 2012 Abstract Biometric based person recognition poses a challenging problem because of large variability in biometric sample quality encountered during testing and a restricted number of enrollment samples for training. Solutions in the form of adaptive biometrics have been introduced to address this issue. These adaptive biometric systems aim to adapt enrolled templates to variations in samples observed during operations. However, despite numerous advantages, few commercial vendors have adopted auto-update procedures in their products. This is due in part to the limited under- standing and limitations associated with existing adaptation schemes. In view that the topic of adaptive biometrics has not been systematically investigated, this paper works toward filling this gap by surveying the topic from a growing body of the recent literature and by providing a coherent view (critical analysis) of the limitations of the existing systems. In addition, we have also identified novel research directions and proposed a novel framework. The overall aim is to advance the state-of-the-art and improve the quality of discourse in this field. 1 Introduction While the biometric technology continues to improve, an intrinsic characteristic of this technology is that a sys- tem’s error rate, e.g., the false accept rate (FAR), false reject rate (FRR) and equal error rate (EER) (the rate at which FAR is equal to FRR), cannot attain the absolute zero. A major cause of these errors is the compound effect of the scarcity of training samples during the enrollment phase as well as the presence of substantial sample variations due to human-sensor interaction and the acquisition environment during operations [1]. Apart from this, being biolog- ical tissues in nature, biometric traits can be altered either temporarily or permanently, due to ageing [2], diseases or treatment to diseases. An important consequence of these factors is that a biometric reference 1 (obtained during 1 A template refers to the biometric sample used for enrolment. The term model refers to a statistical representation derived from one or more biometric samples. In order for our discussion to cover both types of method, we shall adapt the standard vocabulary that is biometric reference or 1
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Critical Analysis of Adaptive Biometric Systems
Norman Poha, Ajita Rattani∗ and Fabio Roli
Department of Computing, FEPS, University of Surrey, Guildford, UKa
Department of Electrical and Electronic Engineering, University of Cagliari
Biometric based person recognition poses a challenging problem because of large variability in biometric sample
quality encountered during testing and a restricted number of enrollment samples for training. Solutions in the form
of adaptive biometrics have been introduced to address this issue. These adaptive biometric systems aim to adapt
enrolled templates to variations in samples observed during operations. However, despite numerous advantages, few
commercial vendors have adopted auto-update procedures in their products. This is due in part to the limited under-
standing and limitations associated with existing adaptation schemes. In view that the topic of adaptive biometrics has
not been systematically investigated, this paper works toward filling this gap by surveying the topic from a growing
body of the recent literature and by providing a coherent view (critical analysis) of the limitations of the existing
systems. In addition, we have also identified novel research directions and proposed a novel framework. The overall
aim is to advance the state-of-the-art and improve the quality of discourse in this field.
1 Introduction
While the biometric technology continues to improve, an intrinsic characteristic of this technology is that a sys-
tem’s error rate, e.g., the false accept rate (FAR), false reject rate (FRR) and equal error rate (EER) (the rate at which
FAR is equal to FRR), cannot attain the absolute zero. A major cause of these errors is the compound effect of the
scarcity of training samples during the enrollment phase as well as the presence of substantial sample variations due
to human-sensor interaction and the acquisition environment during operations [1]. Apart from this, being biolog-
ical tissues in nature, biometric traits can be altered either temporarily or permanently, due to ageing [2], diseases
or treatment to diseases. An important consequence of these factors is that a biometric reference 1 (obtained during1A template refers to the biometric sample used for enrolment. The term model refers to a statistical representation derived from one or more
biometric samples. In order for our discussion to cover both types of method, we shall adapt the standard vocabulary that is biometric reference or
1
enrollment) cannot be expected to fully represent a person’s identity.
Solutions in the form of adaptive biometrics have been introduced to address this issue of reference representa-
tiveness [3, 4]. These adaptive biometric systems attempt to update reference galleries by integrating information
captured in input operational samples. The two-fold aim is to continuously adapt the biometric system to the intra-
class variation of the input data as a result of (1) changing acquisition conditions that may have adverse impact on
the system, e.g., pose and illumination changes for face biometrics, and (2) age and life-style related changes that can
cause permanent changes to the biometric trait.
Most of the existing automated adaptive biometric systems have adopted semi-supervised learning [11, 4] for
the purpose of adaptation. Semi-supervised learning is a machine learning scheme based on the joint use of labeled
and unlabeled samples. In other words, input samples are assigned identity labels using enrolled references and the
positively classified samples are used to adapt the references. A commonly adopted adaptation procedure is to augment
the reference set with the newly classified input samples. The efficacy of the system can be gauged by comparing
the obtained performance gain with a traditional biometric system which does not have any adaptation mechanism.
The expected performance gain is dependent on the effective labeling (classification) of the input samples. This is
because misclassification errors will introduce impostor samples into the updated reference set, the result of which
can be counterproductive. An adaptive biometric system may also operate in supervised mode in which biometric
samples are manually labeled [3]. The supervised method represents the best case performance as all the available
positive (genuine) samples are used for adaptation. However, manual intervention may be time consuming and costly.
Therefore, it is generally infeasible to manually update references regularly.
In contrast, an adaptive biometric system has numerous advantages. First, with this system, one no longer needs
to collect a large number of biometric samples during enrollment. Second, it is no longer necessary to re-enrol or
re-train the system (classifier) from scratch in order to cope up with the changing environment [3]. This convenience
can significantly reduce the cost of maintaining a biometric system. Third, the actual observed variations can be in-
corporated into the references. Despite these advantages, to our knowledge, few biometric vendors such as BIOsingle
(fingerprint) and Recogsys (hand geometry) have incorporated automated adaptation mechanism into their technolo-
gies at the time of this writing. This is due in part to the limited understanding and limitations associated with existing
adaptive biometric systems.
The goal of this manuscript is to advance the state-of-the-art in adaptive biometrics by improving the understanding
and drawing on the limitations of the existing adaptive biometric systems. To this aim, critical analysis of the existing
literature is conducted. Based on the findings of the critical analysis, we propose a novel framework that aims to
mitigate some of the limitations and investigate possible future research avenues.
Specific contributions of this manuscript are as follows:
simply reference. A reference is subsequently used for comparing a biometric test/query sample to obtain a similarity score.
2
1. a taxonomy of adaptive biometric systems through a number of key attributes,
2. use of a meta-analysis technique to objectively compare the effectiveness of key attributes across various systems
reported in the literature, and
3. identification of novel research directions based on the findings of the above meta-analysis.
A preliminary version of this manuscript appeared in [3] in the form of critical survey. The current manuscript
substantially differs from [3] in the following ways. First, novel attributes that distinguish an adaptive system from one
another are introduced. Second, meta-analysis is utilized to aid analysis of various state-of-the-art adaptive systems.
Last but not least, a novel framework, as well as research directions, is proposed.
The paper is organized as follows: section 2 formulates the key attributes and conducts the meta-analysis. Section
3 provides the novel framework and research directions. Conclusions are drawn in section 4.
2 Attributes and Critical analysis
2.1 Attributes of the existing adaptive biometric systems
In an attempt to categorize adaptive biometric systems, the most logical way to proceed is to define a number of
key attributes. On surveying the current state-of-the-art, we find that the following attributes are relevant to distinguish
one adaptive biometric system from another:
1. Supervised against Semi-supervised: The foremost attribute in classifying adaptive systems is indisputably on
the basis of whether the data labeling process is supervised [3, 4, 5, 9] or unsupervised [12, 13, 4]. While
in supervised adaptation, samples are manually labeled, in the unsupervised case, they are inferred by the
system. The latter approach is generally referred to as semi-supervised learning because the enrolment biometric
reference (template) is effectively labeled but the potential operational biometric samples that are used for
adaptation are unlabeled. As mentioned before, supervised adaptation represents the best case scenario, i.e.,
resulting in the best possible performance because all available genuine samples are used for the process of
adaptation. Therefore, it is generally useful to report both strategies when comparing different adaptive methods.
2. Self- against Co-train: For an automated adaptive systems based on semi-supervised learning, self- [12, 13, 14]
and co-training [17, 4] are the commonly adopted schemes for adaptation. In self-training, the system updates
itself by adding only highly confidently classified input samples as additional data for training. A sample is
said to be highly confidently classified if its matching score on comparison with the enrolled templates is above
a stringent operating threshold. The reason to adopt highly confidently classified samples for adaptation is to
avoid impostor intrusion into the updated template set.
3
On the other hand, a co-training based scheme utilizes the mutual and complementary help of the two biometrics
to update the references. Intuitively, one system is expected to assign correct labels to biometric query samples
that are difficult for another system. Consider an example of face and fingerprint co-training system. While on its
own, the face sub-system may have difficulty in labeling a query sample in difficult conditions, the fingerprint
sub-system may classify the associated fingerprint sample with very high confidence. In this case, the face
system can benefit from the high confidence of the fingerprint by incorporating the additional face samples for
training. Therefore, two systems operating at high threshold can still help each other to identify difficult samples
exhibiting large intra-class variations.
3. Verification against Identification: These adaptive biometric system can also be differentiated on the basis of the
system’s basic mode of operation i.e., verification (input sample is compared to the references of the claimed
identity) or identification (input samples are matched to the references of all the users in the database and then
the correct identity is determined among the top most retrievals) [1]. Accordingly, the performance gain will be
measured using EER or rank-one performance metrics, respectively.
4. Level of adaptation: In addition to the adaptation at the reference level, the process of adaptation can also take
place at score or decision level where the matching score or decision functions are adapted to the variations of
the input biometric samples. For instance, Reference [5] uses biometric sample quality to adapt the matching
score so as to render the final accept/reject decision independent of the input sample quality.
5. Online against offline: Online adaptive systems [12, 13] adapt themselves as soon as the input data is available
after the recognition process. On the other hand, offline methods [15, 16, 4] adapt themselves after a batch of
input samples have been accumulated over a period of time. Another fine distinction between the two is that
while an online method follows the chronological ordering of the availability of the samples during adaptation,
the offline one may not adhere to such an ordering.
6. Quality against non-quality based: Recent advancement in the biometric community 2 shows that biometric
sample quality has considerable impact on the system performance for various traits like fingerprint, iris, face,
and etc, as well as for fusion [7]. Quality measures quantify the degree of excellence or conformance of bio-
metric samples to some predefined criteria known to influence the system performance.
However, it is only recently that biometric sample quality has been considered for adaptive biometric systems [5,
6]. Quality based adaptation requires maintaining a different set of updated references for each type of condition.
Since a query sample is always acquired under a particular condition, the inference (matching task) requires
identification of its quality condition and matching with the set of references of the same quality type [6]. In
this manuscript, the resultant system is termed condition-adaptive system.
Since each paper reported several experiments (with a median of 4), a total of 103 experiments are available for
meta-analysis.
We characterize and summarize the outcome of these 103 experiments 3, based on pre-identified attributes, using
meta-analysis. A generalized linear model (GLM) with linear output is trained with a data table containing one
experiment per line. This model takes a set of attributes (as binary variables) in order to predict the performance
gain as its output. In the following sections, we first explain how a generalized linear model (GLM) can be used
to characterize the outcome of one of the 103 experiments and how the attributes are encoded. We then present the
experimental protocols. The final subsection presents the findings of the meta-analysis.
2.2.1 Standardizing the Performance Statistics
Proceeding to meta-analysis is not straightforward since the performance quoted by each paper is not consistent.
In particular, there are two types of performance metric that are systematically quoted: Equal Error Rate (EER) and
rank-one recognition performance. EER quantifies the probability of error at an operating threshold where the rate of
false acceptance is equal to that of false rejection. It is often quoted in a biometric verification scenario. The rank-
one recognition performance, on the other hand, is quoted in a biometric identification scenario. It is defined as the
probability of a target user is indeed ranked the top from a gallery of registered users.
In order to handle the different metrics used, we opted to derive a secondary metric called performance gain. It
is defined as the amount of improvement with respect to the baseline system as well as the primary target metric one
would like to achieve.
For EER, it is defined as:
Perf. gain =EERb − EERa
EERb − 0(1)
where EERa is the EER of the adaptive system, EERb is the EER of the baseline system, and the zero value in the
denominator is the target EER value which one would like to achieve. For the rank-one recognition performance, we
3The data used for meta-analysis is available in the following link: https://sites.google.com/site/ajitarattaniitaly/resources
6
used the following performance gain definition, instead:
Perf. gain =Perfa − Perfb1− Perfb
(2)
where Perfa denotes the performance of the adaptive system whereas Perfb is the performance of the baseline system,
and the unit value in the denominator is the target rank-one recognition performance.
Despite the differences in definition of the EER and the rank-one recognition metrics, the performance gain metric
has the following properties in both cases. First, a positive performance gain implies improvement over the baseline
system. Second, the maximum performance gain can be almost equal to one. This can be simply verified by the fact
that Perfa ≤ 1 and EERa >= 0. Therefore, given that an adaptive biometric system is always reported to be better
than its baseline counterpart, the performance gain will be bounded between 0 and 1. If a value of 1 is registered,
then the target metric is achieved (that is zero for EER and one for rank-one recognition performance). Therefore, the
performance gain we introduced is a viable means to handle the differences in the two primary metrics used (due to
the different mode of operation i.e., verification or identification) by the researchers, allowing the performance gain of
different systems to be compared on equal ground.
2.2.2 Encoding the Attributes of an Experimental Outcome
An experiment is assigned a code of 4 bits in order to represent the following binary attributes, namely:
1. Presence of quality (quality): 1 means yes; and 0, otherwise
2. Use of co-training (co-train): 1 means yes; and 0, otherwise, which implies either self-training or supervised
adaptation
3. Use of supervised adaptation (supervised): 1 means yes; and 0, otherwise, which implies semi-supervised
adaptation (co-training or self-training)
4. Presence of non-match samples in the data set reserved for adaptation (impostor attack): 1 means pres-
ence; and 0, otherwise.
For instance, an experiment coded as 1101 implies that the experiment involves an adaptive biometric system that
uses biometric sample quality, relies on co-training hence, cannot be supervised and contains non-match samples
(impostor attacks) in the adaptation data set.
In order to compare different attributes, two types of meta-analysis experiments are performed, namely single-
factor and multi-factor analysis. In the former, only one of the four attributes (as explained earlier) is considered,
whereas in the latter, all four attributes are considered at the same time. In order to carry out the two types of
experiments, we used a generalized linear model with linear output that estimates performance gain using eq.(1) or (2).
7
Apart from these binary attributes, we also collected other contextual variables that may impact on the generalization
performance of our analysis. These variables are the database size, number of samples used for adaptation, modality
involved etc. Collected data for the contextual variables 4 demonstrate that face is the commonly adopted modality
followed by fingerprint in the existing studies. The adopted adaptation procedure is the same irrespective of the
modality involved. Existing adaptive studies have handled short-to-medium term temporal variations without explicitly
considering the ageing effect over long time span. This is evident by the fact that the adopted databases are collected
over 14-15 weeks in most of the studies.
However, since these contextual variables are not used in inference, they are not considered in fitting the general-
ized linear model (GLM). By inference, we mean that the model will be used to predict a novel but valid combination
of attributes not necessarily represented by the data table. Therefore, our primary goal is to study the attributes that
are likely to dictate the performance gain of an adaptive biometric system. The influence of contextual variables are
not of interest here but this can be a subject of future investigation. Next, we explain how the generalized linear model
is trained.
2.2.3 Training with Generalized Linear Model (GLM)
The generalized linear model [31] takes a set of four attributes as independent variables in order to predict the
performance gain as its output. If a ≡ [a1, . . . aN ] is a vector of binary attributes encoded as a binary string, and
w ≡ [w1, . . . wN ] is the weight vector of real numbers whose elements are associated with those in a, then, GLM
produces
y = waT + w0 (3)
as output. The training process involves estimating the vector of coefficients w including a bias term, w0 ∈ R.
After training, the weights {w0, . . . , wN} are obtained. The GLM is inferred by enumerating a subset of valid
attributes {a}. In the single-factor analysis, eq. (3) is then invoked to consider only an attribute (N = 1) which
can take either a 1 or a 0. The performance gain inferred by both cases, along with their respective upper and lower
confidence intervals, are then compared. In the multi-factor analysis scenario, the multi-dimensional attribute a is
enumerated but invalid combinations are excluded. The performance gain of each valid attributes in {a} is then
compared.
Next, we shall report the findings of single-factor meta-analysis followed by that of multi-factor one.
2.2.4 Findings of Single-factor Meta-analysis
The result for single-factor analysis is shown in Figure 1.
4Collected data is available in the tabulated form (excel file) in the following link: https://sites.google.com/site/ajitarattaniitaly/resources
8
• Supervised adaptation is likely to outperform (about 22.2% more performance gain) semi-supervised method to
adaptation such as self-training and co-training. As mentioned before, performance of the supervised adaptation
can be considered as best case as the references are adapted to all the available genuine samples [4]. This is
in contrary to methods based on semi-supervised learning in which only selective (mostly highly confidently
classified) samples are used for adaptation. Our meta-analysis findings show large variance in the performance
gain of the supervised method such that it overlaps significantly with that of semi-supervised one, indicating
also the effectiveness of the latter. However, for the real time deployment of automated methods (based on
semi-supervised learning), their performance should be equal to their supervised (manual) counterparts. Thus,
further indicating the need of effective adaptation schemes for automated systems.
• Co-training is likely to boost the performance gain by about 25.3 % in comparison to its self-training counterpart.
• The use of biometric sample quality appears to be much better than not using this information. According to our
findings, adaptive biometric systems considering quality measurements resulted in about 47% more performance
gain.
• Including impostor samples in the adaptation set can result in lesser performance gain (16% lesser in our ex-
periments) than if the samples were not present. Since an automated adaptive system deployed in operational
environment is vulnerable to impostor attack, it is unrealistic not to include impostor samples in the adapta-
tion set. As a consequence, our exercise here shows that not including impostor samples in adaptation set can
over-estimate the performance gain.
2.2.5 Findings of Joint-factor Meta-analysis
Figure 2 summarizes performance gain for the joint-factor meta-analysis scenario spanned by four binary at-
tributes: quality, co-train, supervised, and impostor attack. For instance, 0001 implies that an
adaptive system that does not use biometric sample quality, that is based on self-training (hence, not supervised),
and the system has been tested with non-match samples (impostor attack) in the adaptation data set. The attribute
impostor attack is always true as this strategy reports a less biased performance gain, as explained before.
The first three attributes are then enumerated, excluding invalid combinations. For instance, it is not possible that
co-training and supervised adaptation to be present at the same time, as co-training is a semi-supervised
learning strategy; hence, cannot be supervised. Note that the adaptive systems considering supervised adaptation and
quality at the same time (quoted as 1011) are managing the updated references on the basis of quality type. The query
biometric samples are matched to the references of the same quality type [5]. Adaptive systems based on co-training
exploit mutual and complementary information of the bi-modal system for template adaptation as well as testing. On
the other hand, existing studies on supervised adaptation have been reported only for single biometric modality. This
explains the superiority of co-training over supervised adaptation.
9
20 25 30 35 40 45 50 55 60 65
quality
co−train
supervised
impostor
Performance gain (%)
Figure 1: The performance gain for a given attribute obtained by the trained generalized linear model. A blue (red)bar denotes a 95% confidence interval around the expected performance gain, denoted as circle (square), when a givenattribute is present (absent).
These findings suggest that there is a natural increase in performance as one exploits co-training, supervised
adaptation and biometric sample quality systematically. To sum-up, our meta-analysis findings (both single and joint
factor analysis), support the conjecture that quality and co-train are important attributes for the design of automated
adaptive biometric systems.
3 Novel Framework and Research Directions
In this section, we propose a novel framework and set some future research directions that are motivated by the
findings of the meta-analysis.
The results of both the single and joint factor analysis indicate that quality and co-training are important ingredients
when designing an adaptive biometric system. Furthermore, when analyzing the contextual variables of the reported
systems, such as the adopted database size and the number of samples, we found that these systems did not consider
the notion of time in order to account for the ageing effects which may induce temporal performance variation over a
long time span. Motivated by this, we shall propose a novel system that can make use of quality and further include
the notion of time in a single framework. This proposed novel framework is termed as condition-and age-adaptive
system.
10
0 10 20 30 40 50 60 70 80
1101
1011
1001
0101
0011
0001
Performance gain (%)
Figure 2: The performance gain along with the confidence intervals of various configurations spanned by four binaryattributes: quality, co-train, supervised, and impostor (see text).
3.1 Framework for condition and age adaptive system
Existing adaptive biometric systems have not considered the ageing effect explicitly. A possible reason for this is
that, the effect of ageing is often considered to be very different from that caused by biometric sample quality. As a
consequence, methods that aim to address ageing often assume that the biometric sample is free from noise, that is,
images are often well aligned and acquired in controlled conditions.
In practice, however, an adaptive biometric system has to deal with both the aspects (i.e., adaptation to ageing
and quality conditions) for the life-long learning and coping under non-stationary conditions caused by changes in
biometric sample quality. Two separate strategies are needed in order to handle variations caused by biometric sample
quality and those caused by ageing because while the former can cause dramatic changes to the captured biometric
features almost instantaneously, age-related changes are, in comparison, a much more slower and irreversible process.
However, beyond a certain limit of time, the variation due to age-related factors will dominate over that due to the
quality-related ones. This is illustrated in Figure 3, where one image is taken under a somewhat controlled condition,
another with a significantly different quality (head pose) but taken at the same time, and then another taken after two
years.
Thus in order to cope with the changes in the quality as well as temporal variations in the input sample, we propose
a possible framework called a condition and age adaptive system.
We have adopted a Bayesian approach for the formulation of the system. This choice is appropriate because
biometric features are generated by a stochastic process. As a result, no two consecutive samples obtained from a
biometric trait are exactly the same. Thus the uncertainty at the feature space can be characterized using a distribution
11
Figure 3: Illustrating an example of face images taken at different quality conditions (left versus middle) and over time(left versus right).
defined over the feature space. Indeed, a number of state-of-the-art face and speaker recognition classifiers are based
on Bayesian formulation, e.g., [32] and [33]. Furthermore, the state-of-the-art online template update method used in
the fingerprint literature [12, 13, 14] can be interpreted using a Bayesian framework [6].
Next, we introduce the bayesian framework and explain the proposed system.
3.1.1 Bayesian Framework and notations
The recursive formulation of Bayesian estimation allows one to update the parameters of an “old” or initial model
with a new ones given only the latest sample. Thus, given a sequence of observations collected over time, (x1, . . . , xT )
or (x1 : xT ), one can estimate a statistical model parameterized by θ, p(x|θ), in the following way (ignoring the
normalizing factor in each step since we are only interested in maximizing the function with respect to θ):
p(θ|x1 : xT ) ∝T∏
i=1
p(xi|θ)p(θ)
∝T∏
i=2
p(xi|θ)p(θ|x1)
∝T∏
i=3
p(xi|θ)p(θ|x1, x2)
∝...
∝ p(xT |θ)p(θ|x1 : xT−1) (4)
This recursive formulation implies that in order to calculate the optimal value of θ given all previously observed
T samples, one only needs to use the parameter calculated up to T − 1 to do so. The above recursive formulation
shows the benefit of learning for density-based classifiers as an example, leading to finding the optimal value of θ.
This recursive formulation is known as true recursive Bayesian learning. The right- hand term, p(θ|x1 : xT ), is a
reproducing density and the term p(θ) is a conjugate prior [34]. Although the above adaptation is well established and
appears to be sound, it does not consider biometric sample quality nor the ageing effect.
A theoretical framework for model (reference) adaptation using biometric sample quality has been proposed in
12
[6] but did not consider the time effect. The model is henceforth referred to a condition adaptive system. In the
subsequent section, we shall propose a theoretical formulation of condition and age adaptive system that considers
both aspects, hence will be capable of life-long learning (age adaptive) and learning under non-stationary environment
causing concept drift (condition adaptive).
Let x be a biometric feature vector; j ∈ N, the user’s identity; Qu ∈ {Q0, Q1, . . . , QQ}. The condition in
which a biometric sample is captured, with Q0 being the enrolment (controlled) condition and Qu|u ̸= 0 being
other uncontrolled conditions. Each condition Qu is due to a number of factors. Let these factors be enumerated by
(f (1), . . . , f (F)). For instance, for the face biometrics, f (1) is lighting; f (2) corresponds to facial expression types;
f (3) indicates the presence of glasses, f (4) estimates the head pose, and etc. Then, each Qu is a compound effect of
these factors, i.e., Qu = (f (1), . . . , f (F)). It is arguable that, in practice, the condition Q is countable but the total
number of conditions, i.e., Q+1 (including the enrolment condition Q0), cannot be determined exactly. This number,
however, is not impossible to estimate. For instance, it can be estimated by clustering quality measures, or by manual
annotation [6].
Let t ∈ N, the “time” at which a sample is captured. This notion of time is discrete; it is loosely defined such that
two samples that are close in time (say a few seconds apart) will have the same t value. The rationale for using this
definition of t is that the appearance of each biometric trait does not change, as a result of ageing, at the same rate.
Using the above notation, the feature distribution of person j can be completely specified by p(x|j,Q, t).
Let x(j,Q, t) be a sample drawn from p(x|j,Q, t). We shall refer to x(j,Q0, t0) as a reference or model where t0 is
the time at which this sample is obtained; and p̂(x|θ(j,Q0, t0)), a model with parameter θ(j,Q0, t0) that approximates
the true density p(x|j,Q0, t0). Q0 implies that the sample is taken under controlled conditions, that is one in which
all the quality-related factors have been carefully controlled, i.e., F (1) = F(1)0 , . . . , F (F) = F
(F)0 . The notation also
allows us to describe non-ideal samples, for instance, non-frontal head poses, presence of glasses, as may be captured
during enrollment i.e, {x(j,Qu, t0)} for u ̸= 0. The distribution defined over these samples is written as p(x|j,Qu, t0)
for each u; and their corresponding approximated model, as p̂(x|θ(j,Qu, t0)).
Let y ∈ R be a matching score. Furthermore, let j∗ be the claimed identity and x ≡ x(j,Q, t∗) be a query sample
taken at time t∗ from an unknown person j under an unknown condition state Q. For simplicity and without loss of
generality, we also write x ≡ x(j,Q, t) but write in full in order to emphasize a particular state, e.g., x(j,Q, t∗) to
emphasize a given time t∗ and x(j,Q∗, t) to emphasize a given state Q∗.
13
We can then define the following modes of operation: