A FULLY AUTOMATIC RED-EYES DETECTION AND

CORRECTION ALGORITHM BASED ON UNIFORM COLOR

METRIC AND BINOCULAR GEOMETRIC CONSTRAINT

Chun-Hsien Chou

, Kuo-Cheng Liu

*,**

and Shao-Wei Su

Department of Electrical Engineering, Tatung University, Taiwan

Foreign Language and Information Educating Center, Taiwan Hospitality and Tourism College

Keywords: Red-eye, Digital photography, Digital cameras, Uniform color space, Color difference.

Abstract: Red-eye is a highly objectionable defect that often occurs in digital images taken with a flash by modern

small cameras. Although many red-eye reduction algorithms were proposed and equipped in most of the

digital cameras, none of these algorithms is effective enough. In this paper, an algorithm for automatic de-

tection and correction of red-eyes is proposed. The color detector based on uniform color metric is devel-

oped to locate regions of major colors including red-eye color and skin tone. The structure of major colors

is adopted to locate candidate red-eye regions. The geometric relationship between the dimension of the

human pupil and binocular distance is employed to eliminate most false positives (image regions that look

like red-eyes but are not). More than one pairs of red-eyes snapped in different view angles are detected by

the proposed algorithm. Detected red-eyes are then corrected by modifying chroma, hue angles and lumi-

nance of the associated pixels such that red color is removed while maintaining a natural look of the eye.

Simulation results show that the proposed algorithm is pretty robust and effective.

1 INTRODUCTION

Red-eye is a common problem in digital photogra-

phy. When an image is captured with flash illumi-

nation by a camera that has an illumination source

very close to the camera lens, the bright flash light

reflected from the blood vessels on the retina, giving

the human eyes in the image an unnatural red hue.

Red-eye is a hardly acceptable defect that signifi-

cantly reduces the value of an image.

For this reason, many efforts have been made to

prevent it from occurring, or to detect and correct it

in the post-capture processing. A straightforward

way of preventing red-eye is to increase the distance

between the illumination source and the camera lens.

Another solution for red-eye prevention is the use of

a pre-exposure flash that decreases the size of the

subject’s pupil followed by a second flash for cap-

turing the image. The drawback of this approach is

the great consumption of power that shortens the

battery life. Moreover, this approach sometimes can

only reduce, but not eliminate, the red-eye artifacts.

Many research results have been developed and im-

plemented as software products such as “Picture

Maker” from Eastman Kodak Company and

“iPhoto” from Apple. These products require man-

ual manipulation to outline the red-eye region for

correction. It is obviously impractical and ineffi-

cient to process a large number of images manually.

A fully automatic red-eye detection and correction

algorithm is therefore needed. Furthermore, it is

highly expected that this automatic algorithm can be

realized as a piece of hardware and planted in digital

cameras. Recently, a number of researches on

automatic red-eye detection and correction have

been conducted (Schildkraut and Gray, 2002)-

(Zhang, Sun, Li, and Zhang, 2004). The AREA al-

gorithm proposed by Eastman Kodak Company uses

features based on red-eye defects to automatically

detect only a pair of red-eyes in each image

(Schildkraut and Gray, 2002). In (

Matthew and

Robert, 2002), the face must be successfully de-

tected before the red-eye detection where the infor-

mation of color, intensity and dimension is utilized.

In (Ioffe, 2003), a learning-based face detector is

also adopted for the detection of red-eye defects. In

these approaches, the face detection itself is another

challenging problem to be solved (

Xin, Xu, and Du,

1998). In (Zhang, Sun, Li, and Zhang, 2004), a heu-

ristic algorithm is used to detect candidate red-eye

263

Chou C., Liu K. and Su S. (2008).

A FULLY AUTOMATIC RED-EYES DETECTION AND CORRECTION ALGORITHM BASED ON UNIFORM COLOR METRIC AND BINOCULAR

GEOMETRIC CONSTRAINT.

In Proceedings of the First International Conference on Bio-inspired Systems and Signal Processing, pages 263-266

DOI: 10.5220/0001065302630266

 SciTePress

regions, and then an eye classifier is utilized to con-

firm whether the candidate region is a red-eye.

Color is important information to the detection

and correction of red-eye artifacts. In this paper,

color classifiers based on uniform color metric are

first designed to detect colors of red-eyes, skin tones

and colors without red hue. The relationship among

these colors and the geometric constraints inherent

in pairs of human eyes are exploited to eliminate

false positives. Finally, the information of hue,

chroma and luminance is utilized to restore the red-

eye color to a natural tone.

2 UNIFORMITY OF THE CIELAB

COLOR SPACE

Color is a visual perception of the light in the visible

region of the electromagnetic wave spectrum inci-

dent on the human retina. By the theory of trichro-

macy, any color in a color space can be represented

by a triple of numbers called tristimulus values (CIE,

1986), (Sangwine and Horne, 1998). However, col-

ors in many color spaces, such as RGB, XYZ, YUV,

and YC

, are not uniformly distributed in a sense

that the same perceptual color difference does not

correspond to the same distance enumerated in the

tristimulus space (Sangwine and Horne, 1998),

(Sharma and Trusell, 1997). If a color space is per-

ceptually uniform, the perceptual difference between

any two colors can be ideally represented as the

Euclidean distance between their coordinates. The

CIELAB color space is such a color space to over-

come the non-uniform color metric that had been

discussed by MacAdam (MacAdam, 1943). In this

paper, the color transformation to CIELAB color

space is

3 THE PROPOSED RED-EYE

DETECTION ALGORITHM

The functional block diagram of the proposed red-

eye detection algorithm is shown in Figure 1. The

color image is first transformed to CIELAB color

space. Through the uniformity in the CIELAB, the

red-eye color and skin color are roughly extracted.

According to the database containing all kinds of

red-eye samples, a more precise red-eye color filter

is then developed by the method of K-mean cluster-

ing and just-noticeable color difference in the

CIELAB color space. The correlation between the

size of the human eyeball and binocular distance is

finally employed to eliminate most false positives

Through statistically analyzing the manually ex-

tracted red-eye color pixel and skin color pixel from

various color images, the distribution of red-eye

colors and skin colors in CIELAB color space can be

found. For each pixel in the color image, the tris-

timulus values of red-eye color pixels in CIELAB

color space satisfy

⎪

⎩

⎪

⎨

⎧

5025-

8020

30 L

, (1)

and the tristimulus values of skin color pixels in

CIELAB color space satisfy

⎪

⎩

⎪

⎨

⎧

255-

205-

30 L

. (2)

In this paper, the skin color is the reference informa-

tion that is used to delete image regions that look

like red-eyes but are not.

The method of K-means clustering is used to cal-

culate the centroid in the red-eye set as the major

red-eye color. Clustering in pattern recognition is

the process of partitioning a set of pattern vectors

into subsets called clusters. In this paper, the num-

ber of centroids found by the clustering method is 32.

As described in Section 2, the perceptual difference

Transform an image into

CIELAB color space

Roughly red-eye

color filtering

Precisely red-eye color

filtering by MRCR

Noise reduction by

morphological process

Locate red-eye candidates

Find the geometric

relationship between each

pair of red-eye candidates

Skin color

segmentation

Test if the geometric

relationship satisfies

binocular constraint

Test if the red-eye

candidates are surrounded

by skin color pixels

Recover red-eye pixels

Red-eye correction

Color image

Detection result

Delete red-eye

candidates

Ye s

Figure 1: The function block diagram of the proposed

red-eye detection algorithm.

red-eye

candidate

area

Figure 2: Binocular geometric relationship between a

pair of red-eyes.

BIOSIGNALS 2008 - International Conference on Bio-inspired Systems and Signal Processing

264

between any two colors can be ideally represented as

the Euclidean distance between their coordinates in

the CIELAB color space, and loci of colors which

are perceptually indistinguishable from a particular

color form a sphere centralized at this color’s coor-

dinate. In this color space, two colors are considered

perceptually distinguishable if the Euclidean dis-

tance between these two colors’ coordinates exceeds

a threshold of just-noticeable color difference

(JNCD). That is

()

Lab

222

JNCDbaLE ≥Δ+Δ+Δ=Δ

(3)

where JNCD

Lab

has been found around 3.0. There-

fore, the major red-eye color region (MRCR) can be

defined as a sphere for each major red-eye color.

The colors locating in the MRCR of each centroid

are selected as the color pixel in the red-eye.

In previous detection steps, some noises occur in

the preliminary red-eye segmentation area. To re-

duce the noise, morphological process is utilized. In

this paper, two morphology operations, including

dilation “D” and erosion “E”, are used. The former

operation adds pixels to the object boundaries, while

the later operation removes pixels on object bounda-

ries in an image. The two operations are combined

to build a higher order opening operation “O” for

removing noises from the red-eye segmentation area

while preserving the shape and size of this red-eye

segmentation area in the image.

O(I, B

, B

)=D(E(I,B

), B

) (4)

where I is the binary image that marks the location

of selected red-eye pixel, B

the structuring element

for erosion, and B

the structuring element for dila-

tion. The structuring element of dilation is smaller

than that of erosion such that the morphological

process can reduce noises that occur in the red-eye

segmentation area and avoid losing the red-eye can-

didates.

By using color information and morphological

process, red-eye candidates are located. However,

many detected image regions that look like red-eyes

but are not. The color pixels in such regions are

called false positive candidates. Hence, geometric

relationship between each pair of red-eye candidates

is considered. Since the distance between a pair of

red-eyes is useful to eliminate the false positive can-

didate, binocular geometric relationship is used to

develop binocular geometric constraint for removing

false positives. For a red-eye candidate shown in

Figure 2, the green circular region centralized at this

candidate is used to test whether its red-eye counter-

part locates in this region. From the statistical

analysis of a large number of data sets, the range of

the green circular region is constrained by

> 4D

(5)

< 10D

(6)

The pair of red-eye candidates is further tested if its

width and height can satisfy

0.75 < D

< 1.30, (7)

0.75 < D

< 1.30. (8)

If the width and height of the pair of red-eye candi-

dates cannot satisfy Eq. (7) and (8), the pair of red-

eye candidates will be removed.

Finally, the pair of red-eye candidates that pass

through the binocular geometric constraint is tested

if it is surrounded by pixels of skin color tones. The

surrounding skin color pixels of red-eye candidates

are simply defined as two areas as shown in Figure 3.

The width and height of the area are 1.5D

and 3D

respectively. The area is recognized as skin color if

the amount of skin color pixels, N

skin

, that satisfy Eq.

(2) is high enough, or

skin

/(1.5D

× 3D

) >0.8. (9)

4 RECOVERY AND COLOR

CORRECTION OF RED-EYES

Since some red-eye color pixels are removed from

the red-eye candidates in the previous processes, it is

Skin Colo

Searc

Skin

Color

Searc

rea

1.5

eye

candidat

Figure 3: The surrounding skin color pixels of red-eye

candidates.

0.1a

The detected red-eye

The natural

pupil

0.2b

Figure 4: The color relationship between red-eye and

its natural appearance on color-opponent ab plane.

A FULLY AUTOMATIC RED-EYES DETECTION AND CORRECTION ALGORITHM BASED ON UNIFORM

COLOR METRIC AND BINOCULAR GEOMETRIC CONSTRAINT

265

therefore required to retrieve the lost red-eye color

pixels. The region of the red-eye candidate is sim-

ply extended by its neighboring red-eye color pixels.

That is, the boundary of the red-eye candidate region

centered on its center location is enlarged pixel by

one pixel to form a perfectly retrieved region of the

candidate.

Once the location and size of the red-eyes have

been detected, color correction of red-eyes is applied

to the detected red-eyes to obtain the natural appear-

ance of the pupil. To maintain the natural appear-

ance of the pupil at the location of red-eyes, the

value of luminance (L component) of the detected

red-eye is slightly adjusted and the values of hue and

chroma (a and b components) are adjusted based the

color relationship between red-eye and the corre-

sponding natural appearance of pupils on color-

opponent ab plane. For simplicity, the adjustment

for color correction is to scale down the value of a

component of red-eyes by a factor of 0.1 and to scale

down the value of b component of red-eyes by a

factor of 0.2 as shown in Figure 4. The value of

luminance of the detected red-eye is adjusted by a

factor of 0.9. That is,

corrected

= 0.9 × L

(10)

corrected

= 0.1 × a

(11)

corrected

= 0.2 × b

(12)

where (L

, a

, b

) and (L

corrected

, a

corrected

, b

corrected

) are

tristimulus values of the detected red-eye color pixel

and its corrected color pixel, respectively.

5 SIMULATION RESULTS AND

CONCLUSIONS

To evaluate the performance of the proposed algo-

rithm, the simulation of the red-eye detection algo-

rithm that is applied to red-eye digital images with

different size and quality is conducted. In Figure 5,

the “Pinksisters” image that has more than one pairs

of red-eyes is also detected and corrected by using

the proposed algorithm. In our experiments, over

200 red-eye digital photographs are tested and more

than 80% red-eyes are efficiently detected. The ex-

perimental results show that the proposed algorithm

is robust and effective under a variety of shooting

conditions and backgrounds.

In this paper, a fully automatic red-eyes detection

and correction algorithm is proposed. In the pro-

posed algorithm, a robust color classifier for detect-

ing red-eye color and other major colors in digital

images with red-eyes is developed and a multi-stage

criterion for detecting each single red-eye is de-

signed. The detected red-eyes are successfully cor-

rected by modifying chroma, hue angles and lumi-

nance of the associated pixels such that red color is

removed while maintaining a natural look of the eye.

The proposed system has very low false detection

rate. Simulation results show that more than 80% of

red-eyes can be detected and only 5% are false

alarm.

REFERENCES

Schildkraut, J.S. and Gray, R.T., 2002. “A fully automatic

red-eye detection and correction algorithm,” in Proc.

Int. Conf. on Image Processing ICIP, pp. I-801~803.

Matthew, G. and Robert, U. 2002. “Automatic red-eye

detection and correction,” in Proc. Int. Conf. on Image

Processing ICIP, pp. I-804~807.

Ioffe, S. 2003. “Red eye detection with machine learning,”

in Proc. Int. Conf. on Image Processing ICIP, pp. II-

871~874.

Zhang, L., Sun, Y. F., Li, M. J., and Zhang, H. J., 2004.

“Automated red-eye detection in digital photography,”

in Proc. Int. Conf. on Image Processing ICIP, pp.

2363-2366.

Xin, Z., Xu, Y. J., and Du, L. M., 1998. “Locating facial

features with color information,” in Proc. Int. Conf. on

Image Processing ICIP, Vol. 2, pp. 889-892.

CIE, 1986. Colorimetry, CIE Pub. No. 15.2, CIE Central.

Bureau, Vienna, Austria.

Sangwine, S. J. and Horne, R. E. N., 1998. The Colour

Image Processing Handbook. London: Chapman and

Hall.

Sharma, G. and Trusell, H. J. 1997. “Digital color imag-

ing,” IEEE Trans. on Image Processing, vol. 6, pp.

901-932.

MacAdam, D. L., 1943. “Specification of small chroma-

ticity differences,” J. Opt. Soc. Am, vol. 33, pp. 18-26.

(a)

Figure 5 (a): “Pinksisters” image with red-eyes, (b) the

image after correcting red-eye colors.

BIOSIGNALS 2008 - International Conference on Bio-inspired Systems and Signal Processing

266