FACE VERIFICATION IN UNCONTROLLED LIGHT CONDITIONS

OF STREET

Mariusz Leszczy

´

nski and Władysław Skarbek

Department of Electronics and Information Technology, Warsaw University of Technology

Nowowiejska 15/19, 00-665 Warsaw, Poland

Keywords:

Face veriﬁcation, Linear Discriminant Analysis, face metrics, Receiver Operating Characteristics.

Abstract:

Impact of light conditions on face veriﬁcation are considered for three linear discriminant feature extraction

schemes. Two veriﬁcation scenarios, the single image query and multi image query, were compared. The

extraction algorithms are based on compositions of feature projections on global, intra and inter-class error

subspaces: Linear Discriminant Analysis LDA, Dual Linear Discriminant Analysis DLDA, and their combi-

nation LDA+DLDA. The metrics for evaluation of the veriﬁcation error is the Mahalanobis distance between

normalized feature vectors. The normalization of feature vectors is justiﬁed with the upper bound by Fisher

separation index for feature vectors. Experiments conducted on facial databases with complex background

show the high performance of DLDA and DLDA+LDA veriﬁers with Equal Error Rate EER less than one

percent. The degradation of results, when controlled light conditions are replaced by uncontrolled ones, is of

factor two.

1 INTRODUCTION

Nowadays, due to society demand, face recognition is

of signiﬁcant interest in scientiﬁc research. Besides

facial image searching and indexing, face identiﬁca-

tion and face veriﬁcation are the main tasks imple-

mented in face recognition systems.

Compact representation of extracted facial fea-

tures is one of the requirements for facial image in-

dexing (cf. contributions in (MPEG-7, 2004)) of

which fulﬁllment is necessary to get real time re-

sponse for queries in large databases. In case of hu-

man id veriﬁcation systems the size of facial descrip-

tion is not important because the access to features

of relevance is direct on the basis of delivered iden-

tiﬁer. However, there are several advantages to rep-

resent them in computer storage as short as possible

within acceptable increase (if any) of veriﬁcation er-

ror (Jain et al., 1999):

• veriﬁcation can be supported by searching of most

similar face images;

• shorter facial descriptions reduce bandwidth re-

quirements in network distributed applications

(like intelligent cash machines);

• mass production of biometric passports is easier if

their memory requirements are moderate;

• in global security systems archiving billions of

biometric passports could be more tractable.

It follows from the experience of MPEG-7 core

experiments (MPEG-7, 2004) that scalar uniform

quantization from 64 bits of double ﬂoating point pre-

cision to ﬁve bits of ﬁxed point precision for vector

components of facial descriptions based on PCA and

LDA analysis improves the searching performance.

In this research we investigate whether the same con-

clusion can be drawn in case of veriﬁcation in the sys-

tem based on DLDA (Dual Linear Discriminant Anal-

ysis).

The algorithms for LDA and DLDA based on

two SVDs is described in (Skarbek, W., Kucharski,

K. and Bober M., 2004). Recently we have elabo-

rated common conceptual framework for both meth-

ods which avoids traditional formal solution through

generalized eigenvalue problem with respect to within

and between-class scatter matrices by representa-

tion of discrimination process as a sequence of pro-

jection and scaling operations desribed by Disc-

rimanants Analysis Diagram (DAD) (Skarbek W. and

Leszczy

´

nski, M., 2007). The above references give

a complete mathematical and algorithmic discussion

of LDA and DLDA concepts. However, an engineer-

ing point of view on linear discriminant modeling of

data, given in Section II, can help readers to get right

intuitions for using such tools.

In case of cash machine applications not always

427

Leszczy

´

nski M. and Skarbek W. (2007).

FACE VERIFICATION IN UNCONTROLLED LIGHT CONDITIONS OF STREET.

In Proceedings of the Second International Conference on Signal Processing and Multimedia Applications, pages 417-421

DOI: 10.5220/0002142004170421

Copyright

c

SciTePress

Figure 1: Uniform versus complex background: face im-

ages for a person from Altkom database and two persons

from Vision database.

light conditions can be controlled. In this research

we check experimentally the impact of lighting onto

results of veriﬁcation measured via ROC graphs.

2 DISCRIMINANT FEATURES

EXTRACTION

In the presented research the veriﬁcation is based on

thresholding of weighted Euclidean distance of nor-

malized query feature vector to average feature vector

of facial images for the person of claimed identity.

The feature vector is obtained from input face im-

age in the following chain of operations:

1. source (spectral) feature extraction: window face

detection, face window normalization to ﬁxed

eye center positions, transformation of normal-

ized face image into spectral domain 2D DFT, ex-

traction of signiﬁcant Fourier coefﬁcients, source

spectral data centering by grand mean;

2. discriminant feature extraction to get DLDA

(LDA) vector: projection onto inter-class (intra)

singular subspace, inter-class (intra) component-

wise scaling, projection into intra-class (inter) sin-

gular subspace;

3. ﬁnal processing to get target features ready for

matching: projection of DLDA (LDA) vector onto

the unit sphere, and intra-class (inter) component-

wise scaling.

Figure 2: Face images from Yale, MPEG, and WUT

databases.

Traditionally LDA model W = [w

1

, . . . , w

M

] is ob-

tained from the solutions of the following generalized

eigenvalue problem:

S

b

w

m

= λ

i

S

w

w

m

w

m

∈ R

N

, m = 1, . . . , M, M ≤ N

(1)

by selecting independent eigenvectors of highest

eigenvalues (Fukunaga, 1992) of the adjoint classical

eigenvalue problem. Here S

b

– between-class scat-

ter matrix, S

w

– within-class scatter matrix. In or-

der to get the adjoint eigenvalue problem Fukunaga

used the Cholesky lower triangular decomposition of

S

w

= C

w

C

t

w

which is faster and more accurate than

matrix inversion (Golub and Loan, 1989):

C

−1

w

S

b

C

−t

w

(C

t

w

w

m

) = λ

w

(C

t

w

w

m

)

SIGMAP 2007 - International Conference on Signal Processing and Multimedia Applications

428

Figure 3: First 15 eigenfaces (ﬁsherfaces) from U

(b)

in LDA (upper part) and U

(w)

in DLDA (lower part).

In case of DLDA the model is obtained from the

solutions of the following dual generalized eigenvalue

problem:

S

w

w

m

= λ

i

S

b

w

m

w

m

∈ R

N

, m = 1, . . . , M, M ≤ N

(2)

by selecting independent eigenvectors of least posi-

tive eigenvalues (Fukunaga, 1992) of the adjoint clas-

sical eigenvalue problem obtained also by use of

Cholesky matrix decomposition.

Despite its computational advantages the above

solution requires full rank property of the matrix S

w

in case of LDA and the same property for S

b

in case

of DLDA.

If rank(S

w

) < N then data regularization is per-

formed by PCA (Principal Component Analysis). Ac-

tually the change of coordinates in global error space

is performed, followed by rejection of least variance

components till the resulting principal subspace has

trivial intersection {0

N

} with kernel space of S

w

.

Since there is unknown direct formula for the num-

ber of rejected PCA component a trial-error process

is applied and its result strongly depends on the ac-

tual training data set X := [x

1

, . . . , x

L

]. Mathemati-

cally the result depends on mutual relationship be-

tween the kernel of total (global error) scatter matrix

S

t

= S

w

+ S

b

and the kernel of S

w

.

Introducing in (Skarbek W., Kucharski K. and

Bober M., 2006) our LDA and DLDA approach based

on orthogonal projections onto two singular sub-

spaces we intended to replace the relative analysis

of those kernels by the single kernel analysis for S

w

in case of LDA and kernel analysis for S

b

in case of

DLDA.

In order to give intuitive interpretation for our dis-

criminant modelers (LDA and DLDA) we use the fol-

lowing concepts:

1. change of Cartesian coordinate system in selected

error space by rotation and scaling of axis vectors;

2. projection onto subspace of maximum energy of

inter-class error (LDA case);

3. projection onto subspace of minimum energy

intra-class error (DLDA case).

In terms of the above operations the DLDA ex-

traction process can be described as follows:

1. Compute global error x

1

by source data x

0

center-

ing:

x

1

:= x

0

− ¯x;

2. Get major PCA inter representation, i.e. represent

the global error in major PCA base U

(b)

found at

training time for inter-class errors:

x

2

:= (U

(b)

)

t

x

1

3. Scale PCA inter representation using diagonal

matrix Σ

(b)

which normalized training PCA inter

coefﬁcient to unit values:

x

3

:= Σ

(b)

x

2

Note: For training data the variance of x

3

variable

equals to the dimensionality of x

3

.

4. Get minor PCA intra representation, i.e. repre-

sent scaled PCA inter feature vector in minor PCA

base U

(w)

found at training time for intra-class er-

rors:

x

4

:= (U

(w)

)

t

x

3

Hence the aggregated DLDA matrixW has the fol-

lowing effect:

y = W

t

(x

0

− ¯x) = (U

(w)

)Σ

(b)

(U

(b)

)

t

(x− ¯x) (3)

It is interesting that in matching stage the best re-

sults are achieved by Mahalanobis distance for unit

length feature vectors:

δ(y

A

, y

B

) :=

y

A

ky

A

k

−

y

B

ky

B

k

t

Λ

y

A

ky

A

k

−

y

B

ky

B

k

(4)

where Λ := (Σ

(b)

)

2

in case of LDA and Λ := (Σ

(w)

)

2

in case of DLDA.

The normalization of feature vectors is closely re-

lated to the Fisher separation index between LDA or

DLDA feature vectors y

A

and y

B

of face images A and

B :

1

2

y

A

ky

A

k

−

y

B

ky

B

k

= 1−

2y

t

A

y

B

2ky

A

kky

B

k

≤

≤ 1−

2y

t

A

y

B

ky

A

k

2

+ky

B

k

2

=

ky

A

−y

B

k

2

ky

A

k

2

+ky

B

k

2

Since discriminant analysis indirectly reduces the

Fisher separation index then it also reduces the nor-

malized intra-class error.

FACE VERIFICATION IN UNCONTROLLED LIGHT CONDITIONS OF STREET

429

Figure 4: ROC results for face veriﬁers in uncontrolled (upper row) and neutral (lower row) lighting conditions. Face images

from Altkom, Vision, Yale, MPEG, and WUT databases.

3 EXPERIMENTAL RESULTS

We selected the normalized luminance facial images

(46 × 56 resolution with the same position of eyes)

from the following databases (cf. Fig. 1,2): Altkom

(80 persons with 15 images each), Vision (26 persons

with varying number of images per person), Yale (15

persons with 11 images each), MPEG (110 persons

with 5 images each), WUT (54 persons with 3 images

each).

From the previous works described in (Skarbek,

W., Kucharski, K. and Bober M., 2004) it is al-

ready known that in case of face veriﬁcation the op-

timization of inverse Fisher ratio (DLDA) leads to

better results than the optimization of Fisher ratio

(LDA). Figure 3 gives more insight for this phe-

nomenon. Namely, DLDA eigenfaces (ﬁsherfaces)

are more contrasted and more focused on particular

facial parts.

It is interesting that aggregation of DLDA and

LDA veriﬁers by the maximum, the arithmetic mean,

and the harmonic mean of distances give intermediate

results (in ROC sense) between the best DLDA re-

sults and and signiﬁcantly worse LDA results. How-

ever, the geometric mean of both distances leads to

slight improvements of EER and ROC over DLDA.

SIGMAP 2007 - International Conference on Signal Processing and Multimedia Applications

430

In Fig. 4 we compare this results in two query scenar-

ios: single image (Image ROC) and multi-image (Per-

son ROC) and two lighting scenarios: unconstrained

(upper part ROCs) and neutral (lower part ROCs).

In case of Image ROC, we observe the improve-

ment of Equal Error Rate EER for the neutral lighting

for about two times. The insigniﬁcant improvement

in case of Person ROC is justiﬁed by small number of

neutral photos for most of persons engaged in training

and testing.

4 DISCUSSION AND

CONCLUSIONS

Impact of light conditions on face veriﬁcation are con-

sidered for three linear discriminant feature extraction

schemes. Two veriﬁcation scenarios, the single image

query and multi image query, were compared. The ex-

traction algorithms are based on compositions of fea-

ture projections on global, intra and inter-class error

subspaces: Linear Discriminant Analysis LDA, Dual

Linear Discriminant Analysis DLDA, and their com-

bination LDA+DLDA.

The metrics for evaluation of the veriﬁcation er-

ror is the Mahalanobis distance between normalized

feature vectors. The normalization of feature vectors

is justiﬁed with the upper bound by Fisher separation

index for feature vectors.

Experiments conducted on facial databases with

complex background show the high performance of

DLDA and DLDA+LDA veriﬁers with Equal Error

Rate EER less than one percent. The degradation of

results when controlled light conditions are replaced

by uncontrolled ones is of factor two.

In cash machine application the input of veriﬁca-

tion system is given as temporal sequence of images.

On the basis of the previous works we recommend

the design of face veriﬁer for this application by the

following six steps:

1. Detect frontal pose of face by Discrete Gabor Jet

DGJ algorithm w.r.t. inner eye and nose corners

(Skarbek W. and Naruniec J., 2007).

2. Compensate lighting by Quotient Illumination

Relighting algorithm (QIR) (Cao B., Shan S., Gao

W. and D. Zhao, 2003).

3. Compensate pose by inner eye and mouth corners

to ﬁnd the homographic mapping.

4. Align the compensated image by alignment of the

line segment joining outer eye corners.

5. Design the feature extraction scheme by optimiz-

ing DAD diagram.

6. If the veriﬁer of 5 is not satisfactory then optimize

DLDA cascade.

ACKNOWLEDGEMENTS

The work presented was developed within VIS-

NET 2, a European Network of Excellence

(http://www.visnet-noe.org), funded under the

European Commission IST FP6 Programme.

REFERENCES

Fukunaga, K. (1992). Introduction to Statistical Pattern

Recognition. Academic Press.

Golub, G. and Loan, C. (1989). Matrix Computations. The

Johns Hopkins University Press.

Jain, A. K., Bolle, R., and Pankanti, S., editors (1999). Bio-

metrics: Personal Identiﬁcation in Networked Society.

Kluwer Academic Publishers.

Cao B., Shan S., Gao W. and D. Zhao, (2003). Illumina-

tion normalization for robust face recognition against

varying lighting conditions in . IEEE ICCV-2003,

157:164, 2003

MPEG-7, editor (2004). Multimedia content description in-

terface. Visual Descriptor Extensions., volume 15938-

3:2002/Amd.1:2004. ISO/IEC.

Skarbek W., Kucharski K. and M. Bober, (2004). Dual LDA

for Face Recognition in . Fundamenta Informaticae

303:334:2004

Skarbek W., Kucharski K. and Bober M. , (2006). Cas-

cade of dual LDA operators for face recognition in .

Geometric properties from incomplete data, Kluwer

Academic Publishers 199:219, 2006

Leszczy

´

nski M. and Skarbek W., (2007). Biometric Veriﬁ-

cation by Projections in Error Subspaces in . Rough

Set and Knowledge Technology, Springer LNAI 4481,

166:174, 2007

Naruniec J. and Skarbek W., (2007). Face Detection by

Discrete Gabor Jets and Reference Graph of Fiducial

Points in . Rough Set and Knowledge Technology,

Springer LNAI 4481, 187:195, 2007

FACE VERIFICATION IN UNCONTROLLED LIGHT CONDITIONS OF STREET

431