CONFIDENCE MEASURE FOR AUTOMATIC FACE RECOGNITION

Ladislav Lenc and Pavel Kr´al

Department of Computer Science and Engineering, University of West Bohemia, Plzeˇn, Czech Republic

Keywords:

Automatic face recognition, Conﬁdence measure, Gabor wavelets, Czech news agency.

Abstract:

This paper deals with the use of conﬁdence measure for Automatic Face Recognition (AFR). AFR is realized

by the adapted Kepenecki face recognition approach based on the Gabor wavelet transform. This work is

motivated by the fact that obtained recognition rate on the real-world corpus is only about 50% which is not

sufﬁcient for our application, a system for automatic labelling of the photographs in a large database. The

main goal of this work is thus the proposition of the post-processing of the classiﬁcation result in order to

remove the “incorrectly” classiﬁed face images. We show that the use of conﬁdence measure to ﬁlter out

incorrectly recognized faces is beneﬁcial. Two conﬁdence measures are proposed and evaluated on the Czech

News Agency (

CTK) corpus. Experimental results conﬁrm the beneﬁt of the use of conﬁdence measure for

the automatic face recognition task.

1 INTRODUCTION

Automatic Face Recognition (AFR) consists of auto-

matic identiﬁcation of a person from a digital image

or from a video frame by a computer. A huge amount

of methods for face recognition was proposed in the

last two decades. As many researchers agree (Wiskott

et al., 1999; Bolme, 2003; Shen and Bai, 2006), some

of the most successful methods are based on Gabor

wavelet transform.

Particularly, the method proposed by Kepenekci

in (Kepenekci, 2001) gives a very good face recog-

nition accuracy and outperforms the majority of the

other approaches. In our previous work, we thus

adapted the Kepenekci method in order to increase

signiﬁcantly the recognition accuracy and robustness

of the algorithm and decrease the computation time.

We experimentally conﬁrmed that the proposed mod-

iﬁcations signiﬁcantly improve recognition rate and

robustness of the algorithm, while the computation

time doesn’t change. When the parameters are set

correctly, the use of more training examples increases

the recognition rate on the ORL and Czech News

Agency (

CTK) corpora by 17% and by 35% in ab-

solute value, respectively. The ﬁnal obtained recogni-

tion rate on the ORL database is 100%, however for

the

CTK corpus it is only 50%.

The results of this work will be used by the

CTK.

CTK owns a large database (about 2 millions) of pho-

tographs. A signiﬁcant number of photos is manually

annotated (i.e. the photo identity is known). However,

other photos are unlabelled; the identities are thus un-

known. The main task of our application consists in

automatic labelling of the unlabelled photos. This ap-

plication must also handle cases when a new photo-

graph is added into the database (automatic labelling

of this picture). The system must also guarantee the

cases when the image is not well aligned, when its

pose varies and when the lighting conditions differ.

Note that about ten labelled images of each person

are available.

The above presented recognition accuracy is not

sufﬁcient for our application. The main goal of this

work thus consists of the proposition of the post-

processing of the classiﬁcation result in order to re-

move the “incorrectly” classiﬁed face images. We

proposed the conﬁdence measure technique for this

task. Note that to the best of our knowledge, there is

very little existing work on conﬁdence measure in the

automatic face recognition domain. Alternatively, a

number of studies have been published for other do-

mains, and particularly for automatic speech recogni-

tion.

Section 2 presents a short review of face recogni-

tion approaches. Section 3 describes conﬁdence mea-

sure methods that we used in order to remove the in-

correctly recognized examples. Section 4 deals with

the evaluation of the results. In the last section, we

discuss the research results and we propose some fu-

ture research directions.

365

Lenc L. and Král P..

CONFIDENCE MEASURE FOR AUTOMATIC FACE RECOGNITION.

DOI: 10.5220/0003690403570360

In Proceedings of the International Conference on Knowledge Discovery and Information Retrieval (KDIR-2011), pages 357-360

ISBN: 978-989-8425-79-9

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

2 RELATED WORK

Early face recognition approaches are based on

normalized error measures between signiﬁcant face

points, as proposed for instance in (Bledsoe, 1966).

The main drawback of such methods consists in the

necessity of manual labelling of the images. Another

method that uses similar measurements was designed

by Kanade (Kanade, 1977). In this case, the labelling

of important face points is automatic.

One of the ﬁrst successful approaches is Prin-

cipal Component Analysis (PCA), so called Eigen-

faces (Turk and Pentland, 1991). Eigenfaces is a sta-

tistical method that takes into account the whole im-

age as a vector. Image vectors are put together and

create a matrix. Eigenvectors of this matrix are cal-

culated. Face images can then be expressed as a lin-

ear combination of these vectors. Each image is rep-

resented as a set of weights for corresponding vec-

tors. This method performs very well when images

are well aligned and have approximately the same

pose. Changing lighting conditions, pose variations,

scale variations and other dissimilarities between im-

ages decrease the recognition rate rapidly (Sirovich

and Kirby, 1987).

Further methods using Neural networks or Hidden

Markov Models (HMMs) are introduced. One of the

best performing methods based on neural networks is

presented in (Lawrenceet al., 1997), while HMMs are

successfully used in (Neﬁan and Hayes, 1998).

In the last couple of years, several successfull

approaches based on Gabor wavelets were intro-

duced (Shen and Bai, 2006). The ﬁrst method for face

recognitionbased on Gabor wavelets was proposedby

Lades (Lades et al., 1993). He used Gabor wavelets

for image preprocessing. Some approaches (Shen,

2005) also combine the preprocessing with Gabor

wavelets with well established methods such as Eige-

faces, Fisherfaces etc.

One successful approach described in (Wiskott

et al., 1999; Bolme, 2003) is Elastic Bunch Graph

Matching (EBGM). Gabor wavelet convolutions

(Jets) in speciﬁed positions (Landmarks) are used for

image representation. Another Gabor wavelet based

method was proposed in (Kepenekci, 2001). This

method differs from the previous one, that it deter-

mines the landmarks automatically and their number

is thus not constant.

3 CONFIDENCE MEASURE FOR

AUTOMATIC FACE

RECOGNITION

As in many other works (Lleida and Rose, 1996;

Jiang, 2005), our ﬁrst conﬁdence measure for auto-

matic face recognition is an estimate of the a poste-

riori class probability. The output of our classiﬁer is

P(F|C), where C is the recognized face class and F

represents the face features. The likelihoods P(F|C)

are normalized to compute the a posteriori class prob-

abilities as follows:

P(C|F) =

P(F|C).P(C)

∑

I∈F I M

P(F|I).P(I)

(1)

F I M represents the set of all faces and P(C) denotes

the prior probability of the face class C.

In the ﬁrst version of our algorithm, called Abso-

lute conﬁdence value method, only the faces

C so that

C = argmax

(P(C|F)) (2)

C|F) > T (3)

are considered as recognized correctly.

In the second version of our approach, called

Relative conﬁdence value method, the difference be-

tween the best hypothesis and the second best one is

computed by the following equation:

P∆ = P(

C|F) − max

C6=

(P(C|F)) (4)

Only the faces with P∆ > T are accepted. This

second approach aims at identifying the faces that

“dominate” all the other candidates, which is not al-

ways well captured by the ﬁrst method.

T is in the both cases an acceptation threshold and

its optimal value is found experimentally.

4 EXPERIMENTS

4.1

CTK Corpus

All experiments are evaluated on the previously cre-

ated

CTK corpus. This corpus is composed of the im-

ages of individuals in uncontrolled environment that

were randomly selected from the large

CTK database.

All images were taken during a long time period (20

years or more). They were automatically resized to

the size 92 × 120 pixel and transformed to grayscale.

The resulting corpus contains pictures of 70 individu-

als, at least 8 images for each person. Note that orien-

tation, lighting conditions and background of images

KDIR 2011 - International Conference on Knowledge Discovery and Information Retrieval

366

differ signiﬁcantly. A correct face recognition using

this dataset is thus very difﬁcult.

Figure 1 shows one example from this corpus.

This corpus is available for free for research purpose

upon request to the authors.

Figure 1: Examples of the

CTK face corpus.

4.2 Experimental Setup

One modiﬁcation of the Kepenekci algorithm consists

of the use more trainingexamples instead of one train-

ing image per person during the face modeling step.

Therefore, our conﬁdence measure methods are eval-

uated with the different number of the training ex-

amples (between 1 and 7). Only the most important

“border” cases (with 1 and 7 training examples) are

reported in this paper.

We used the threshold values T ∈ [0;1] when 0 is

the case without any conﬁdence measure (all recog-

nized faces are accepted). The cases when the face

recognition rate remains constant are not reported in

any ﬁgure.

CTK requires a guarantee that a high face recogni-

tion rate (it means 95% or higher) is achieved. How-

ever, it is not necessary to identify all individuals.

4.3 Absolute Conﬁdence Value

Figure 2 plots the face recognition rate and number

of the classiﬁed faces when one training example is

used. The result obtained without any conﬁdence

measure (or equivalently for T = 0) is only 15%.

Hoverer, when the conﬁdence measure is used, the

best obtainedscore is 100%. This result is constrained

by the fact, that only few examples is accepted (about

2%) which is unfortunately not acceptable by the tar-

get application (see Section 1).

Figure 3 shows the face recognition rate and num-

ber of the classiﬁed faces when all seven training ex-

amples are used. The result obtained without any con-

ﬁdence measure is 52%. This value conﬁrms the good

performance of our modiﬁcations of the Kepenekci

approach. However this score is still not high enough

100

0.56 0.58 0.6 0.62 0.64 0.66 0.68

Number of classified faces [in %]

Recognition rate [in %]

Threshold

Classified

Recognition rate

Figure 2: Absolute Conﬁdence Value method: Face recog-

nition rates and numbers of the classiﬁed faces when one

training example is used (T ∈ [0;1]).

for our target application. When obtained recognition

rate is 95%, we ﬁlter out about 90% faces.

100

0.58 0.59 0.6 0.61 0.62 0.63

Number of classified faces [in %]

Recognition rate [in %]

Threshold

Classified

Recognition rate

Figure 3: Absolute Conﬁdence Value method: Face recog-

nition rates and numbers of the identiﬁed faces when all

seven training examples are used (T ∈ [0;1]).

4.4 Relative Conﬁdence Value

Figure 4 shows the face recognition accuracy and

number of the classiﬁed faces in relation to the thresh-

old T (one training example is used). This ﬁgure

shows that this method performs better than the pre-

vious one. However, the number (about 5% with the

95% recognition rate) of the accepted examples is still

too low.

Figure 5 plots the face recognition accuracy and

number of the classiﬁed faces in relation to the thresh-

old T when seven training examples are used. When

obtained recognition rate is 95%, we accept about

35% faces. This score is high enough for our applica-

tion.

CONFIDENCE MEASURE FOR AUTOMATIC FACE RECOGNITION

367

100

0 0.01 0.02 0.03 0.04 0.05 0.06 0.07

Number of classified faces [in %]

Recognition rate [in %]

Threshold

Classified

Recognition rate

Figure 4: Relative Conﬁdence Value method: Face recog-

nition rates and numbers of the classiﬁed faces when one

training example is used (T ∈ [0;1]).

100

0 0.001 0.002 0.003 0.004 0.005 0.006 0.007 0.008 0.009

Number of classified faces [in %]

Recognition rate [in %]

Threshold

Classified

Recognition rate

Figure 5: Relative Conﬁdence Value method: Face recogni-

tion rates and numbers of the classiﬁed faces when all seven

training examples are used (T ∈ [0;1]).

5 CONCLUSIONS AND

PERSPECTIVES

The main contribution of this work is the proposition

and evaluation of two conﬁdence measure techniques

as a post-processing of the automatic face recognition

task. We suggest using a conﬁdence measure due to

the relatively low face recognition rate on the

CTK

corpus. The experiments show that around 10% of

the images is classiﬁed with nearly 100% accuracy

by the Absolute conﬁdence value approach. We fur-

ther show, that the second proposed approach, Rel-

ative conﬁdence value method, is more suitable for

the practical use. Around 30% of images is classiﬁed

with an accuracy close to 100% by this method.

The perspectives of this work are numerous, in-

cluding evaluation of the methods on the larger real-

word corpora (the faces are not well aligned and the

lighting conditions differ), the development of more

sophisticated conﬁdence measures or in adding the

pre-processing step, i.e. eye localization and the sub-

sequent image rotation and aligning.

ACKNOWLEDGEMENTS

This work has been partly supported by

CTK and by

the UWB grant SGS-2010-028 Advanced Computer

and InformationSystems. We also would like to thank

CTK for providing the photographic data.

REFERENCES

Bledsoe, W. W. (1966). Man-machine facial recognition.

Technical report, Panoramic Research Inc., Palo Alto,

CA.

Bolme, D. S. (2003). Elastic Bunch Graph Matching. PhD

thesis, Colorado State University.

Jiang, H. (2005). Conﬁdence measures for speech recog-

nition: a survey. Speech Communication, 45(4):455–

470.

Kanade, T. (1977). Computer recognition of human faces.

Birkhauser Verlag.

Kepenekci, B. (2001). Face Recognition Using Gabor

Wavelet Transform. PhD thesis, The Middle East

Technical University.

Lades, M., Vorbr¨uggen, J. C., Buhmann, J., Lange, J., and

von der Malsburg, C. (1993). Distortion invariant

object recognition in the dynamic link architecture.

IEEE Transactions On Computers.

Lawrence, S., Giles, S., Tsoi, A., and Back, A. (1997).

Face recognition: A convolutional neural network ap-

proach. IEEE Trans. on Neural Networks.

Lleida, E. and Rose, R. C. (1996). Likelihood Ratio Decod-

ing and Conﬁdence Measures for Continuous Speech

Recognition. In ICSLP’96, volume 1, pages 478–481,

Philadelphia, USA.

Neﬁan, A. V. and Hayes, M. H. (1998). Hidden markov

models for face recognition. In IEEE International

Conference on Acoustics, Speech, and Signal Process-

ing.

Shen, L. (2005). Recognizing Faces - An Approach Based

on Gabor Wavelets. PhD thesis, University of Not-

tingham.

Shen, L. and Bai, L. (2006). A review on gabor wavelets

for face recognition. Pattern Analysis & Applications.

Sirovich, L. and Kirby, M. (1987). Low-dimensional proce-

dure for the characterization of human faces. Journal

of the Optical Society of America, 4.

Turk, M. A. and Pentland, A. P. (1991). Face recognition

using eigenfaces. In IEEE Computer Society Confer-

ence on In Computer Vision and Pattern Recognition.

Computer Vision and Pattern Recognition.

Wiskott, L., Fellous, J.-M., Kr¨uger, N., and von der Mals-

burg, C. (1999). Face recognition by elastic bunch

graph matching. Intelligent Biometric Techniques in

Fingerprint and Face Recognition.

KDIR 2011 - International Conference on Knowledge Discovery and Information Retrieval

368