Multimodal Biometric Identification System based on Cascaded
Advanced of Fingerprint and Finger Vein Images and AND Rule at
Decision Level Fusion
El Mehdi Cherrat
1
, Rachid Alaoui
2
and Hassane Bouzahir
1
1
ISTI Laboratory, National School of Applied Sciences, Ibn Zohr University, Agadir, Morocco
2
ASTIMI Laboratory, Higher School of Technology- Sale-, Mohammed V University, Rabat, Morocco
Keywords:
Fingerprint Recognition; Finger Vein Recognition, Minutiae points, histogram of oriented gradients Feature,
Fusion.
Abstract:
The multimodal identification system can integrate a variety of biometric characteristics. The main advan-
tage of multibiometric system against traditional single biometric is achieving the recognition process more
accurate and safe. In this paper, we will present a multimodal biometric recognition system that combines
fingerprint and finger vein. The features in theses biometric traits are extracted to identify that individual is
genuine or impostor using minutia points for fingerprint and Histogram of Oriented Gradient for finger vein.In
the first stage, cascade multimodal biometric system based on the both biometric modalities is applied. In
the second stage, the fusion is accomplished at decision level method based on AND rule using multimodal
biometric recognition system. The simulation results have demonstrated that the proposed fusion algorithm
performs increase probability the accuracy to 99,85 than the other system based on unimodal characteristics.
1 INTRODUCTION
In recent years, the biometric system necessity has
been rapidly increased. The biometric recognition is
required reliable to distinguish one individual from
another using measurable morphological (such as fin-
gerprint, face, iris, etc.) or behavioral (for example
voice, signature, etc.) features. With these charac-
teristics including being less susceptible to verifica-
tion being stolen or forgotten. It is used for criminal
identification, immigration and naturalization service,
securing access to buildings or personal objects, sup-
porting anonymous transactions, etc.(Cherrat et al.,
2017).
The most common biometric system is fingerprint
recognition. It is considered an excellent biometric
modality for identification or verification the person,
especially in the latest smart phones and consumer de-
vices. Compared to other biometric traits, the finger
vein modality has achieved popularity in biometric
recognition because of the variety advantages given
by these systems for example, 1) the vein of each
person are completely unique and different 2) it is
identified as being less prone to modify with age and
growth (3 the finger veins biometric is easily acquired
using sensor capable of capturing or the NIR (Near-
Infrared) light source 4) the vein structure is hidden
inside the skin. Thus, the possibility of spoof the
human recognition system is very complex (Khellat-
Kihel et al., 2016).
The general structure of biometric recognition
system consists of four main steps. In the first one,
the acquisition of biometric image is process of get-
ting a digitalized image of a person using specific cap-
turing device. In the second step, the pre-processing
is allowed to improve overall quality of the captured
image and to correct its orientation . After that, the
region of interest is localized. It is the process of ob-
tenting all important data needed for recognition. In
the next step, the features information are extracted
using different algorithms. In the last step, generally,
the matching of the extracted characteristics is applied
in order to perform the recognition of the person.
The multimodal biometrics combines two or more
different biometric modalities and reduces certain
limitations of systems based on one modality such
as spoof attacks,non-universality, noise in sensed
data,inter-class similarities and intra-class variations.
Thus, the recognition system based on fusion of
multibiometric is most recommended for significantly
Cherrat, E., Alaoui, R. and Bouzahir, H.
Multimodal Biometric Identification System based on Cascaded Advanced of Fingerprint and Finger Vein Images and AND Rule at Decision Level Fusion.
DOI: 10.5220/0009773701610166
In Proceedings of the 1st International Conference of Computer Science and Renewable Energies (ICCSRE 2018), pages 161-166
ISBN: 978-989-758-431-2
Copyright
c
 2020 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved
161

improving the system performance and reducing the
error rate the identification or verification of the in-
dividual. This fusion can be applied at the sen-
sor level, the feature extraction level, the matching-
score level, rank level and at the decision level. The
multimodal biometric systems are classified as multi-
instance, multi-sensor, multi-algorithm, multi-modal
and hybrid systems (Khellat-Kihel et al., 2016).
The rest of the paper is separated into four sec-
tions. In the section 2, the related works in the field
are reviewed. Section 3 discusses the proposed algo-
rithm. Experimental results have been analysed and
discussed in Section 4. Finally, the conclusion is pre-
sented in the last section.
2 RELATED WORKS
Many techniques have been proposed of the multi-
modal biometrics system. Ross et al (Ross, 2003)
presented different levels of fusion and score level fu-
sion on the multimodal biometric system. Singh.al
(Singh et al., 2004) proposed biometric recognition
system based on face combining visible and thermal
Infrared (IR) images at sensor level. Son (Son and
Lee, 2005) have been subjected a fusion of face and
iris at feature level. Ross.al (Ross and Govindara-
jan, 2005) presented hand and face combined at fea-
ture level. Moreover, the experiments were applied
in three different scenarios. At the fusion of match
scores, Jain.al (Jain and Huang, 2004) proposed lin-
ear discriminant function and the decision trees. Dif-
ferent fusion techniques and normalization methods
of fingerprint, hand geometry and palm-print biomet-
ric sources are achieved by Yang.al (Yang and Ma,
2007). Another multimodal biometric system based
on multi-instance iris recognition system using a fu-
sion of right iris and left iris for the same individ-
ual is studied by Wu.al. (Wu et al., 2007). Jain.al
(Jain et al., 1999) introduced a multimodal biomet-
ric system using face, fingerprint, and voice. Yang.al
(Yang, 2018) presented a multi-biometric system can-
celable using fingerprint and finger-vein, which com-
bines the minutia points of fingerprint and finger-vein
image feature based on a feature-level of three fusion
techniques. The fusion multimodal biometric sys-
tem based on fingerprint and finger-vein at score level
using four score fusion approaches (min score, max
score, simple sum, user weighting) and three score
normalization techniques (min-max, z-score, hyper-
bolic tangent) is developed by Vishi.al (Vishi and
Jøsang, 2017).
3 PROPOSED METHOD
In first level of our algorithm, fingerprint image is
enhanced using gabor filter technique, binarized and
passed to thinning algorithm. Then, the features
points are extracted using ridge ending and bifurca-
tion uniformly namely minutiae. In the final step, the
comparison of minutiae information provided from
the registered database, and the query fingerprint is
presented to the matching. In the second level, the
Linear Regression Line have been utilized to solve
the orientation of misalignments of finger vein im-
ages. Next, the region of interest of image is obtained
using canny method. After that, the histogram equal-
ization is applied to enhance the cropped finger vein
image.Furthermore, the features extracted is based on
Histogram of Oriented Gradient algorithm. Finally,
the score provided is compared and machted with
stored fingervein templates stored in the database. If
the first level is not identified then the second level
works. The fusion is applied at cascaded advanced
decision level. If the first level is passed then the sec-
ond level avoids for matching fingervein extracted. In
this section, we detail the proposed technique which
is illustrated in Figure 1. The details of each phase are
represented in the following.
Figure 1: Block diagram of proposed algorithm for finger
vein image recognition
3.1 Fingerprint Recognition
3.1.1 Image Enhancement
To overcome the background noise, non-uniform il-
lumination and low contrast of the fingerprint image
captured, the preprocessing is important step for char-
acteristic extraction and then the matching. The mean
and variance are used to normalize and estimate the
orientation of input image. After that, the frequency
image is computed from which the region mask is
provided using block classification of resulted im-
ICCSRE 2018 - International Conference of Computer Science and Renewable Energies
162

age. Then, gabor filters applied to normalized image
(Hong et al., 1998).
3.1.2 Feature Extraction
The features points are extracted from fingerprint im-
age such as ridge ending and bifurcation uniformly
namely minutiae. Before extracting the minutiae, the
binarization method is applied using block with size
3x3. This process is transformed the 8 bits gray im-
age to 1 bit with 1 value for the valleys and 0 value
for ridges based on a given threshold. Next, mor-
phological technique processing (dilatation and ero-
sion) is used as postprocessing to achieve more com-
pact blocks for reducing the noise region. Moreover,
the thinning operation is applied to remove basically
the redundant pixels until having a single pixel width
(Cui and Yang, 2011). Finally, the bifurcation and
ending points are detected by computing black pixels
of 8-directional nearest for each pixel point in finger-
print image. If the central pixel is black and has 3
black values nearest, then this pixel is a bifurcation.
When the number of black nearest is just 1, the feature
point is represented ending. The connection number
(CN) for a given ridge can be represented in equa-
tion 1 . Hence, the minutea characteristic extraction
of fingerprint pattern is represented by the following
parameters, 1) Type of the ridge, 2) x-coordinate, 3)
y-coordinate, 4) θ-orientation.
CN =
1
2
7
∑
i=0
|P
i
− P
i+1
| (1)
where P
i
is the pixel value at index i and P
8
=P
0
3.1.3 Features Mathching
The minutiae feature extraction of fingerprint pattern
is represented by the type of ridge, the spatial coor-
dinates x, y and orientation of minutiae points. The
Euclidian distance is used to find number of matched
two minutiae pairs. This distance is described as fol-
lows :
E
d
(M
i
, M
j
) =
q
(x
i
− x
j
)
2
+ (y
i
− y
2
j
) (2)
A
d
(M
i
, M
j
) = min(|θ
i
−θ
j
|, 360−|θ
i
−θ
j
|) ≤ θ
0
(3)
where M
i
and M
j
are the extracted minutiae points
pairs from the template in the enrolled database and
the input query fingerprint image respectively. A
d
(di-
rection difference between M
i
and M
j
) is smaller than
an angular tolerance θ
0
.
The similarity score S
f
based on minutiae points
between the queried and stored fingerprint images is
calculated using equation (6).
M(M
i
, M
j
) =
(
1 i f (E
d
≤ r
0
)and(A
d
≤ θ
0
)
0 Otherwise
(4)
where r
0
is allowed the diffrence between M
i
and M
j
.
N
m
=
n
∑
i=1
n
∑
j=1
M(M
i
, M
j
) (5)
S
Fscore
=
s
N
2
m
N
i
N
j
(6)
where N
m
is the total matching of M
i
and M
j
. N
i
and
N
j
are total number of M
i
and M
j
respectively.
3.2 Fingervein Recognition
3.2.1 Orientation Correction
In this section, the obtained region of finger vein
images is needed to determine that images are ori-
ented correctly or not. The orientation corrected angle
can affect to accurately extract feature extraction and
matching. Thus, Linear Regression Line is applied to
compute the estimated orientation angle θ represented
in the figure 2. First, all middle points are represented
the line function of the finger vein image, which is de-
fined in equation (7). Next, the orientation angle value
is calculated by using equation (8). Finally, these im-
ages are considered normal, if orientation angle value
is equal to 0, otherwise the finger vein image is not
correctly oriented. The figure 3 represents the results
of the orientation corrected angle.
Figure 2: Orientation angle detection.
y = ax + b (7)
a =
∑
M
i=1
(x
i
− ¯x) − (y
i
− ¯y)
∑
M
i=1
(x
i
− ¯x)
2
(8)
¯x =
1
M
M
∑
i=1
x
i
, ¯y =
1
M
M
∑
i=1
y
i
(9)
θ =





−arctan(a) i f (a < 0)
arctan(a) i f (a > 0)
0 i f (a = 0)
(10)
Multimodal Biometric Identification System based on Cascaded Advanced of Fingerprint and Finger Vein Images and AND Rule at
Decision Level Fusion
163

where x
i
= 1,2,3....M, i= 1,2,3....M. Therefore, the ori-
entation angle value θ between the estimted line and
X-axis is calculted using equation (10).
(a) (b)
Figure 3: Orientation correction of finger vein image : (a)
finger vein image distortion oriented ; (b) finger vein image
oriented correctly
3.2.2 ROI Detection
When the finger vein image is correctly oriented,
the regions of interest will be obtained. This region
has the ridge and lines patterns of the finger vein
that is exploited for recognition. Canny technique is
the famous edge detector algorithm. It is developed
by (Canny, 1986). For this reason, Canny method
is adopted to extract the ROI of finger vein image.
Firstly, this technique is applied to obtain the edge
outline of finger vein image. After that, the inner rect-
angle is used to extract the ROI. The result of ROI of
finger vein image using Canny edge detector scheme
is shown in Figure 4.
(a) (b)
(c) (d)
Figure 4: Illustration of ROI extraction and pre-processing
of finger vein image : (a) Original image ; (b) Canny
method ; (c) ROI detected ; (d) ROI pre-processing
3.2.3 Pre-processing
After extracted the ROI providing Canny edge detec-
tor, we evaluate the sharpness and contrast of finger
vein image. Thus, local histogram equalization is ap-
plied for image contrast enhancement (Kim, 2001).
The essential data of finger vein image can be shown
clearly which is represented in Figure 4.
3.2.4 Feature Extraction Method
HOG (Histogram of Oriented Gradients) descriptor
has shown outstanding success in recognition system.
HOG has been popular used as one of the better fea-
tures to acquire local shape points or the edge. For
this advantage, this technique is applied in our al-
gorithm for feature extraction in order to recognize
the person. The HOG orientation of each cell, small
connected areas, is separated. For better compensat-
ing the illumination, the normalized histogram is ob-
tained by accumulating a measure of the local his-
togram gradient orientation over blocks based on the
results to normalize each cell in the block. These his-
tograms are combined to represent the HOG feature
(Dalal, 2005). The process of extracting the HOG de-
scriptor is illustrated in Figure 5.
Figure 5: Illustration of HOG descriptor extraction
3.2.5 Features Comparison
The generated similarity score based on HOG fea-
tures Before to compute the generated similarity score
S
Hscore
based on HOG features, Hamming distances
is computed to match scores between the finger vein
template stored in database and the input test tem-
plate as calculated using equation (11). The similarity
score of HOG is given by equation (12).
D
H
=
k
∑
i=0
|F
Ei
− F
Ti
| (11)
where F
Ei
and F
Ti
are the extracted HOG from the
template in the enrolled database and the input query
finger vein image respectively.
S
Hscore
= min(D
H
) (12)
4 EXPERIMENTS AND RESULTS
The experimental operation platform in this study
is described as follows: the host configuration: CPU
ICCSRE 2018 - International Conference of Computer Science and Renewable Energies
164

Intel Core2 Duo at 2.00 GHz, RAM 3.00 GB, runtime
environment: Microsoft Visual Studio C++ 2013 with
OpenCV library. In order to validate the proposed al-
gorithm, the results have been tested on the VERA
Fingervein Database (Tome, 2015) and the public Fin-
gerprint Verification Competition 2004 dataset (Maio
et al., 2004). The performance measure is accuracy
rate as defined by equation (13).
Accuracy =
FAR +FRR
Total
NumAcc
(13)
where FAR (False Acceptance Rate) is the probabil-
ity of unauthorized users that are not recognized over
the total number tested, FRR (False Reject Rate) de-
scribes the percentage of authorized users that are
not recognized falsely to the total number tested and
Total
NumAcc
is the total number access.
Table 1: The accuracy rate for different recognition biomet-
ric system results.
Algorithms Accuracy Rate (%)
Fingerprint using Minutiae 96,93
Fingervein using HOG 97,45
Cascaded Multimodal 99,85
Cascaded Multimodal and And Rule 99,28
Figure 6: Recognition results with comparison of algo-
rithms at the accuracy rate.
Table 1 shows the performance of accuracy
rate based on single biometric system using finger-
print, fingervein images and the cascaded multimodal
recognition biometric system using fingerprint and
fingervein. Figure 6 shows the ROC curves using dif-
ferent recognition methods. In comparison with sin-
gle biometric system, our proposed algorithm espe-
cially with the cascaded multimodal biometric system
using fingerprint and fingervein shows superior per-
formance in terms of accuracy rate with 99,85% with
where Fingerprint using minutiae points fingervein
using HOG, cascaded multimodal and And Rule give
96,93% ,97,45% and 99,28% respectively. We can
conclude from these results that the cascaded multi-
modal recognition biometric system using fingerprint
and fingervein leads to an improvement in recognition
biometric system performance.
5 CONCLUSIONS
This paper presented multimodal biometric identifi-
cation system based on cascaded advanced of finger-
print and finger vein images and AND rule at decision
level fusion in order to achieve accurate recognition
of the person. In first step level, the fingerprint im-
age is enhanced based on gabor filter algorithm, bina-
rized. Moreover it is passed to thinning technique, ex-
tract minutiae points and finally the matching. If the
matching score is greater than the given fingerprint
threshold then recognition is stopped. Else the second
level is started with fingervein image. The orientation
correction, ROI detection based on canny method and
local histogram equalization to improve the quality of
fingervein image are applied.After that, the important
features are extracted using HOG method. In the next
level, the recognition of the both biometrics sources
are verified at decision level based on AND rule. The
results have shown that the proposed work performed
better in personal identification rate than others based
solely on one algorithm. The proposed method can be
further extended by matching the features with other
metrics.
REFERENCES
Canny, J. (1986). A computational approach to edge de-
tection. IEEE Transactions on pattern analysis and
machine intelligence, (6), pp. 679-698.
Cherrat, E. M., Alaoui, R., Bouzahir, H., and Jenkal, W.
(2017). High density salt-and-pepper noise suppres-
sion using adaptive dual threshold decision based al-
gorithm in fingerprint images. In 2017 Intelligent Sys-
tems and Computer Vision (ISCV), pages 1–4. IEEE.
Cui, F. and Yang, G. (2011). Score level fusion of finger-
print and finger vein recognition. Journal of Compu-
tational information systems, 7(16):5723–5731.
Dalal, N. (2005). Histograms of oriented gradients for hu-
man detection. IEEE Computer Society Conference
on, vol. 1, pp. 886-893.
Hong, L., Wan, Y., and Jain, A. (1998). Fingerprint image
enhancement: algorithm and performance evaluation.
IEEE transactions on pattern analysis and machine
intelligence, 20(8):777–789.
Jain, A. and Huang, J. (2004). Integrating independent
components and linear discriminant analysis for gen-
der classification. In Sixth IEEE International Con-
ference on Automatic Face and Gesture Recognition,
2004. Proceedings., pages 159–163. IEEE.
Multimodal Biometric Identification System based on Cascaded Advanced of Fingerprint and Finger Vein Images and AND Rule at
Decision Level Fusion
165

Jain, A. K., Hong, L., and Kulkarni, Y. (1999). A mul-
timodal biometric system using fingerprint, face and
speech. In 2nd Int’l Conf. AVBPA, volume 10.
Khellat-Kihel, S., Abrishambaf, R., Monteiro, J. L., and
Benyettou, M. (2016). Multimodal fusion of the fin-
ger vein, fingerprint and the finger-knuckle-print us-
ing kernel fisher analysis. Applied Soft Computing,
42:439–447.
Kim, J. (2001). An advanced contrast enhancement us-
ing partially overlapped sub-block histogram equal-
ization. IEEE transactions on circuits and systems for
video technology, 11(4), pp. 475-484.
Maio, D., Maltoni, D., Cappelli, R., Wayman, J. L., and
Jain, A. K. (2004). Fvc2004: Third fingerprint veri-
fication competition. In International Conference on
Biometric Authentication, pages 1–7. Springer.
Ross, A. (2003). Information fusion in biometrics. Pattern
recognition letters, 24(13), pp. 2115-2125.
Ross, A. A. and Govindarajan, R. (2005). Feature level
fusion of hand and face biometrics. In Biometric
Technology for Human Identification II, volume 5779,
pages 196–205. International Society for Optics and
Photonics.
Singh, S., Gyaourova, A., Bebis, G., and Pavlidis, I. (2004).
Infrared and visible image fusion for face recogni-
tion. In Biometric Technology for Human Identifica-
tion, volume 5404, pages 585–597. International So-
ciety for Optics and Photonics.
Son, B. and Lee, Y. (2005). Biometric authentication sys-
tem using reduced joint feature vector of iris and
face. In International Conference on Audio-and
Video-Based Biometric Person Authentication, pages
513–522. Springer.
Tome, P. (2015). The 1st Competition on Counter Measures
to Finger Vein Spoofing Attacks. The 8th IAPR In-
ternational Conference on Biometrics (ICB), pp. 513-
518.
Vishi, K. and Jøsang, A. (2017). A new approach for multi-
biometric fusion based on subjective logic. In Pro-
ceedings of the 1st International Conference on Inter-
net of Things and Machine Learning, page 68. ACM.
Wu, X., Wang, K., Zhang, D., and Qi, N. (2007). Com-
bining left and right irises for personal authentication.
In International Workshop on Energy Minimization
Methods in Computer Vision and Pattern Recognition,
pages 145–152. Springer.
Yang, F. and Ma, B. (2007). A new mixed-mode bio-
metrics information fusion based-on fingerprint, hand-
geometry and palm-print. In Fourth International
Conference on Image and Graphics (ICIG 2007),
pages 689–693. IEEE.
Yang, W. (2018). A fingerprint and finger-vein based can-
celable multi-biometric system. Pattern Recognition,
78, pp. 242-251.
ICCSRE 2018 - International Conference of Computer Science and Renewable Energies
166