The Face Recognition Processes - Neurofuzzy Approach

Wojciech Biniek, Edward Puchała and Maria Bujnowska-Fedak

Nokia, Strzegomska 36, 53-611 Wroclaw, Poland

Department of Systems and Computer Networks, Faculty of Electronics, Wroclaw University of Science and Technology,

Janiszewskiego 11/17, 50-372 Wroclaw, Poland

Department of Family Medicine, Wroclaw Medical University, Syrokomli 1, 51-141 Wroclaw, Poland

Keywords:

Face Recognition, Face Landmarks Detection, Biometrics, Fuzzy Logic, Neurofuzzy Systems.

Abstract:

The paper deals with the novel neuro-fuzzy approach for face recognition problem. A proposed method

consists of two steps. The ﬁrst one means image preprocessing (face detection and landmarks extraction).

In particular, this concerns points on the face such as the corners of the mouth, points along the eyebrows,

on the eyes, nose and jaw. In the second step, based on extracted features, neurofuzzy system recognizes to

whom detected face belongs. Classical fuzzy controllers need an expert knowledge to deﬁne set of rules and/or

defuzziﬁcation process. Main concept of neurofuzzy approach is to replace expert with neural networks. This

paper shows that, neurofuzzy system can suit face recognition process and provide better results than other

popular techniques.

1 INTRODUCTION

Facial appearance is deﬁnitely very important biomet-

ric characteristic as the most decisive way to recog-

nize and distinguish people. Since photography was

invented it is used in passports or identity cards as

it guarantees unambiguous identiﬁcation. Nowadays,

there exist a lot of archival photos databases, which

can be automatically searched. Therefore, large vari-

ety of applications for face recognition process, opens

up, e.g. looking for suspects in police database, phys-

ical and logical access to protected resources and hu-

manoid robots. Face recognition process is intuitive

and obvious for humans, whereas it can be very chal-

lenging for computers and artiﬁcial intelligence , be-

cause it is impossible to create simple rules leading

to mathematical model of this complicated process.

Usually face recognition systems begin with image

processing in order to detect if there is a face and

where is it localized. There are many methods of face

detection, based on face anatomy. Using standard im-

age processing operations like shifts, scaling and ro-

tations, face image can be normalized to simplify next

steps of recognition. There are deﬁned two categories

of methods which can be applied for image process-

ing (Bolle et al., 2004): based on facial appearance,

face geometry or hybrids. Approach presented in

this paper deals with face geometry method, which

closely depends on position and geometrical relations

between face details, like eyes, mouth, nose etc. In

this case, recognition is a matter of comparison with

layouts of details stored in database of known faces.

Feature-by-feature comparison may be inefﬁcient and

lacks generalization and there is a need for more opti-

mized solution. Another promising concept that could

be apply for face recognition is so called “computing

with words” introduced by (Zadeh, 1996). Its origin

dates back to his famous article ,,Fuzzy Sets” (Zadeh,

1965, p. 338-353) where he introduced new approach

for describing not precise and many-meanings con-

cepts as opposed to well-known mathematical meth-

ods, which use classical divalent. Inspiration for cre-

ating fuzzy logic was human’s brain, which use not

precise terms of natural language and can create very

complicated models of complex reality, making good

decisions and deal with many complicated tasks with-

out any measures and calculations. Computing with

words is particularly useful when: i) available infor-

mation has low accuracy level, ii) there is a toleration

for inaccuracy and task can be done with low cost, iii)

problem cannot be solved using classical methods or

it is just too complicated to deﬁne it numerically. In

theory, it suits very well face recognition problem, be-

cause people usually describe others using not precise

terms like ,,big/small eyes/nose” etc. The main draw-

back of this approach is that systems based on fuzzy

logic have no ability to learn and everything must be

deﬁned explicitly by creating set of rules manually.

Biniek, W., Puchała, E. and Bujnowska-Fedak, M.

The Face Recognition Processes - Neurofuzzy Approach.

DOI: 10.5220/0006538300830088

In Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2018) - Volume 2: BIOIMAGING, pages 83-88

ISBN: 978-989-758-278-3

Figure 1: Steps of recognition process.

Face recognition process requires self-learning tool

like neural network. Therefore, combining com-

plementary solutions of fuzzy logic and neural net-

work appears to be a very promising path (Dhava-

likar and Kulkarni, 2014). However, in general fuzzy

logic and neural networks approach the design of in-

telligent systems from quite different perspectives.

Fuzzy logic allows making deﬁnite decisions based

on imprecise or ambiguous data, whereas neural net-

works tries to incorporate human thinking process

to solve problems without mathematically modeling

them (Yuan et al., 2004). Even though both methods

can be used to solve nonlinear problems, and prob-

lems that are not properly deﬁned, they are fully in-

dependent (Vyas and Garg, 2012). Both neural net-

works and fuzzy logic are powerful design techniques

that have their strengths and weaknesses summed up

in Table 1. This paper presents neurofuzzy approach

in face recognition process. Presented solution con-

sist of two main step: image processing and face

recognition. Figure 1 shows ﬂow chart of presented

process in details.

Table 1: Properties of neural networks and fuzzy logic

(Makhsoos et al., 2009).

Neural Networks Fuzzy Logic

Knowledge

Representation

Implicit, the system

can not be easily

interpreted or modiﬁed

Explicit,veriﬁcation and

optimization are very

easy and efﬁcient

Trainability

Trains itself by

learning from

data sets

None, everything must

be deﬁned explicitly

2 METHOD DESCRIPTION

In this section all elements of presented process will

be explained. In particular this concerns: image pre-

processing, face detection and neuro-fuzzy system.

2.1 Image Preprocessing

Image preprocessing is responsible for ﬁnding human

faces in an image, estimating landmarks and normal-

izing face image to selected size. Face detector and

landmark estimator is already implemented in dlib

http://dlib.net/. Face detector is made using the stan-

dard Histogram of Oriented Gradients (HOG) feature

combined with a linear classiﬁer, an image pyramid

and sliding window detection scheme. Blue frame in

ﬁgure 2a marks the result of the detection step, which

is just estimation where the face is located. Based

on this estimation, landmarks positions are detected.

Landmark Detector was created by using dlib’s imple-

mentation after One Millisecond Face Alignment with

an Ensemble of Regression Trees (Kazemi and Sulli-

van, 2014). Green frame in ﬁgure 2a is a region which

contains all detected landmarks and a role model for

normalization shown in ﬁgure 2b. This process is very

important, because it automatically creates reference

system for landmarks coordinates, so it is possible to

analyze every case in the same way.

(a) Face detection and

landmarks

(b) Normalization

Figure 2: Steps of image preprocessing.

2.2 Face Geometry

Landmarks detector provide positions of 68 land-

marks arranged on the face. Based on detected land-

marks it is possible to separate selected regions such

BIOIMAGING 2018 - 5th International Conference on Bioimaging

as the mouth, left and right eye, nose and face shape,

which consists of jaw and eyebrows. This approach

makes the fuzzy representation of features more use-

ful and understandable and as a consequence the com-

puting with words strategy. It is also possible to mea-

sure distance between selected points and regions.

This solution should give more precision in the whole

process, because it includes region arrangement rela-

tive to each other. Features can be combined in many

different ways.

(a) Face shape features (b) Eyes features

(e) Distances between re-

gions

Figure 3: Example of possible features set.

Figure 3 shows exemplary combination of geo-

metrical features for selected regions (ﬁgures: 3a, 3b,

3c, 3d) and spatial relations between regions (ﬁgure

3e).

2.3 Neurofuzzy System

Classical fuzzy controllers need predeﬁned member-

ship functions and set of rules linked with them. This

is made usually by human expert.

Face recognition process is a problem that re-

quires specially deﬁned solution, that will ﬁt spec-

iﬁed cases, with no need for human expert. The

Figure 4: Structure of neurofuzzy network (MF - Member-

ship function, NE - Neuron).

best way to create neurofuzzy system is to represent

it as multilayer network with one output node (neu-

ron). Figure 4 shows a scheme of an exemplary con-

troller. Presented solution uses singleton as fuzziﬁ-

cation method and neural network as defuzziﬁcation

block (Rutkowska et al., 1999). This scheme can

be mathematically described using following expres-

sion:

y = ¯w

∏

i=1

exp

−



¯x

− ¯x



, (1)

where:

• ¯w is a vector of neural network weights,

• i is a number of input variables,

• k is a number of membership functions for each

variable universe,

• n is a number of all possible rules, which is a prod-

uct of number of input variables with number of

membership function,

• ¯x

is a mean value of normal distribution, which

is used here as membership function,

• σ

is a sigma parameter (standard deviation) for a

normal distribution.

In the structure of neurofuzzy controller shown

in ﬁgure 4 we can separate two speciﬁc functional

modules. First one consists of layers L1 and L2,

which represent fuzziﬁcation using singleton method

and fuzzy inference blocks in classical fuzzy con-

troller. Layer L3 is a neural network which realizes

defuzziﬁcation part. Output of L1 consists of values

of membership functions existing in input variables

universe. One of the most popular membership func-

tion is Gaussian function, which appears in many con-

texts in the natural sciences.

Of course, it is possible to use another member-

ship functions - this can be a subject of further re-

search. Shape of Gaussian function depends on two

parameters: x is a mean value (central value)and σ

responsible for width of the distribution. Figure 5

The Face Recognition Processes - Neurofuzzy Approach

0 1 2 3 4

5 6

7 8 9 10

0.2

0.4

0.6

0.8

Small

Average

Big

Universe

Membership Function

Figure 5: Universe of fuzzy variable with standard distribu-

tion membership functions.

shows exemplary universe of fuzzy variable. These

parameters can be set precisely, depending on uni-

verse of each variable. Variable universe can be de-

ﬁned as difference between its largest and smallest

value observed in training set. The easiest way to

start is to convert universe of input and output vari-

ables into linguistic domain like ,,small/average/big

nose”, set them evenly into variable space and learn-

ing in the iterative way. Result of this step is set of

parameters pairs (x

, σ

) which describe all member-

ship functions for each fuzzy set in the whole variable

space. Layer L2 is a representation of fuzzy infer-

ence. Each element in this layer is speciﬁc combi-

nation of L1 outputs. Size of this layer depends on

number of input variables and number of membership

functions for each variable universe. Layer L3 repre-

sents defuzziﬁcation block of the system using neural

network. Number of neurons in input layer is equal to

outputs from layer L2. It is also possible to deﬁne spe-

ciﬁc parameters like number and size of hidden lay-

ers, number of learning epochs, activation function.

3 EXPERIMENTAL RESULTS

AND DISCUSSION

The experimental investigation of proposed approach

was divided into three stages. The ﬁrst one covers

the aspect of deﬁning how input features determine

results of the whole system. Reducing the number

of input features can be beneﬁcial for system ability

to generalize, because in the case of too many fea-

tures the classiﬁer will overﬁt. The second one was

performed in order to show how size of the data set

inﬂuences classiﬁcation process. The last one is a

comparison with other classiﬁcation methods. Neuro-

fuzzy system will be compared with k-nearest neigh-

bors, naive Bayes, decision tree and neural network

approaches which are standard tools used for face

recognition.

Training set used in this case, comes from FERET

(Phillips, 2016) database. In total 248 people which

have at least three frontal photos of their faces, were

selected. This means that the number of elements (L)

of the training set ranges from 248 to 744. As a train-

ing algorithm, back propagation method was used.

Test set consists of one photo of each person cho-

sen randomly from available samples (248 elements).

Presented solution using input vector of features with

structure:

X = (x

, x

, · · · , x

, x

)

, (2)

where x

, x

, · · · , x

are single features read from

face image. The result of the operation of the system

is the number of the class to which the object belongs.

There are 248 classes (one for each person).

3.1 Features Selection

Way to check how input features inﬂuence ﬁnal result

is to increase number of used features in each itera-

tion. In this case there are about 48 features. While

iterating from 1 to 48 selected number of features we

need to check ﬁnal correct recognition of whole sys-

tem. Result of this experiment is presented in Figure

6. The biggest difference in the correct recognition

value can be seen between 1 and 10 selected features.

The correct recognition increases strongly with the

number of features for low values, much slower in the

range of 10-48 features (20% increase) and saturates

for larger number of features. Therefore, face recog-

nition process based on neurofuzzy system needs at

0 10 20 30 40

Number of features

Correct recognition [%]

Correct recognition depending on number of features

Neurofuzzy Mamdani

Figure 6: Correct recognition depending on number of fea-

tures.

BIOIMAGING 2018 - 5th International Conference on Bioimaging

least 10 input features to get about 80% correct recog-

nition. It is also possible to determine which features

are dominant. Figure 7 shows most dominant features

listed below marked on exemplary face. It is notewor-

thy that the most decisive features are those related to

face geometry (width and height) and distances be-

tween separated regions - for example distance be-

tween corners of the eyes. The dominant features, in

decreasing order of use, are as follows:

1. face9 (points 8 - 27),

2. general2 (points 36 - 45),

3. face14 (points 21 - 22),

4. face6 (points 5 - 11),

5. face11 (points 18 - 25),

6. general5 (points 16 - 35),

7. general7 (points 0 - 31),

8. face4 (points 3 - 13),

9. mouth1 (points 48 - 54),

10. general8 (points 0 - 48).

Figure 7: Face with selected 10 most dominant features.

3.2 Inﬂuence of Size of the Input Data

This experiment studies how number of people in the

input set inﬂuences the ﬁnal result. Figure 8 shows the

correct recognition value depending on size of input

set for learning. Each iteration of this experiment has

been performed for 1 to 248 faces.

In Figure 8 results for different size of samples

per person, corresponding to 1, 2 or 3 photos, are pre-

sented. In all cases the correct recognition values de-

creases with the size of the input set by 10%-15%. It

is obvious that set with three photos per person guar-

antees the best correct recognition, which decreases

when the number of photos per person for learning is

decreased.

100

150

200

250

100

Size of input set

Correct recognition [%]

Correct recognition depending on size of input set

One photo per person

Two photos per person

Three photos per person

Figure 8: Correct recognition depending on size of input

set.

3.3 Comparison with other

Classiﬁcation Methods

There are many methods of solving classiﬁcation

problem. Comparison with other classiﬁcation meth-

ods is a way to verify if method works correctly. It

is not possible to easily assign classiﬁer to speciﬁc

problem. The best way to select suitable method

is by comparison. This paper compares neurofuzzy

approach with k-nearest neighbors (for k = 5 neigh-

bors), naive Bayes using normal distribution func-

tion, decision tree and neural network. As a com-

parative criterion, the value of the correct classiﬁca-

tion was used. Figure 9 shows result of this com-

parison. Developed and presented in this work neu-

rofuzzy approach gives the best correct recognition

with the value over 85%. This is probably caused by

fuzziﬁcation process, which normalizes input values

and naive Bayes is based on normal distribution. Pre-

sented result depends on initialization of input param-

eters and can change in some speciﬁc cases, which

can be investigated in future work.

0 10 20 30 40

50 60

70 80

90 100

Knn

Decision tree

Neural network

Neurofuzzy Mamdani

Naive Bayes

62.3

74.5

69.9

88.8

78.1

Correct recognition [%]

Figure 9: Neurofuzzy correct recognition comparison with

other classiﬁcation methods.

The Face Recognition Processes - Neurofuzzy Approach

4 CONCLUSIONS

Obtained results conﬁrms that neurofuzzy system can

suit face recognition process. The approach was ver-

iﬁed by features selection experiment and inﬂuence

of size of the input data. Finally, neurofuzzy ap-

proach was compared with other classiﬁcation meth-

ods, which shows presented system works better. The

proposed work can be further extended by adding dif-

ferent types of membership functions or checking oth-

ers parameters of the system. Try to use different

features reduction techniques is also worth attention.

A comparison of the presented approach with deep

learning methodology will be analyzed in the future

too. Presented system has a wide range of appli-

cations in research and human-computer interfaces.

Based on the obtained results, we can say that neu-

rofuzzy approach ﬁts better face recognition process

than other most popular classiﬁcation techniques, es-

pecially neural network, which is also a part of neu-

rofuzzy system.

REFERENCES

R. Bolle, J. Connell, S. Pankanti, N. K. Ratha, and A. W.

Senior. Guide to Biometrics. Springer, 1st edition, 2004.

A. S. Dhavalikar and R. K. Kulkarni. Face detection

and facial expression recognition system. International

Conference on Electronics and Communication System,

2014.

V. Kazemi and J. Sullivan. One millisecond face alignment

with an ensemble of regression trees. The Conference on

Computer Vision and Pattern Recognition, 2014.

N. T. Makhsoos, R. Ebrahimpour, and A. Hajiany. Face

recognition based on neuro-fuzzy system. JCSNS Inter-

national Journal of Computer Science and Network Se-

curity, 9(4), 2009.

P. J. Phillips. Color feret database, 2016. URL http://

www.itl.nist.gov/iad/humanid/feret/feret master.html.

[Online; accessed 26-July-2017].

D. Rutkowska, M. Pilinski, and L. Rutkowski. Sieci

neuronowe, algorytymy genetyczne i systemy rozmyte.

Wydawnictwo Naukowe PWN, Warszawa, Ł

z, 1st edi-

tion, 1999.

R. Vyas and G. Garg. Face recognition using feature extrac-

tion and neuro-fuzzy techniques. International Journal

of Electronics and Computer Science Engineering, 1(4),

2012.

X. Yuan, J. Lu, and T. Yahagi. Face recognition based on

neuron fuzzy systems. Circus and Systems, 2004.

L. A. Zadeh. Fuzzy sets. Information and Control, 1965.

L. A. Zadeh. Fuzzy logic = computing with words. IEEE

Transactions On Fuzzy Systems, 4(2), 1996.

BIOIMAGING 2018 - 5th International Conference on Bioimaging