Robust Object Segmentation using Active Contours and Shape Prior

Mohamed Amine Mezghich, Malek Sellami, Slim M’Hiri and Fouzi Ghorbel

GRIFT Research Group, CRISTAL Laboratory, National School of Computer Sciences,

University of Manouba 2010, Manouba, Tunisia

Keywords:

Active Contours, Shape Prior, Phase Correlation, Rigid Transformation.

Abstract:

In this paper, we intend to present new method to incorporate geometric shape prior into region-based active

contours in order to improve its robustness to noise and occlusions. The proposed shape prior is deﬁned after

the registration of binary images associated with level set functions of the active contour and a reference shape.

The used registration method is based on phase correlation. This representation makes it possible to manage

several objects simultaneously. Experimental results show the ability of the proposed geometric shape prior to

constrain an evolving curve towards a target shape. We highlight, on synthetic and real images, the beneﬁt of

the method on segmentation results in presence of partial occlusions, low contrast and noise.

1 INTRODUCTION

Active contour models have been introduced in 1988

(Kass et al., 1988). The principle of these methods

is to move a curve iteratively minimizing energy’s

functional. The minimum is reached at object bound-

aries. Active contour methods can be classiﬁed into

two families : parametric and geometric active con-

tours. The ﬁrst family, called also Snakes, uses an

explicit representation of the contours and depends

only on image gradient to detect objects (Kass et al.,

1988; Cohen, 1991). Theses models are able to seg-

ment only one object in the image. To overcome this

problem, an implicit representation of the active con-

tours via level set approach (i.e. geometric active

contours) has been used (Osher and Sethian, 1988)

to handle topological changes of the front. A num-

ber of active contour models based on level set theory

has been then proposed which can be divided into two

categories : The boundary-based approach which de-

pends on an edge stopping function to detect objects

(Malladi et al., 1995; Caselles et al., 1997) and the

region-based approach which is based on minimizing

an energy’s functional to segment objects in the im-

age (Chan and Vese, 2001). Experiments show that

region-based models can detect objects with smooth

boundaries and noise since the whole region is ex-

plored. However, there is still no way to characterize

the global shape of an object. Especially in presence

of occlusions and clutter, all the previous models con-

verge to the wrong contours. To solve the above men-

tioned problems, different attempts include shape

prior into the active contour model. Many works have

been proposed which can be classiﬁed into statistical

or geometrical shape priors. A statistical shape model

(Leventon et al., 2000) was associated to the geodesic

active contours (Caselles et al., 1997). A set of train-

ing shapes is used to deﬁne a Gaussian distribution

over shapes. At each step of the surface evolution,

the maximum a posteriori (MAP) position and shape

are estimated and used to move globally the surface

while local evolution is based on image gradient and

curvature. A new energy’s functional based on the

quadratic distance between the evolving curve and the

average shape of the target object after alignment was

deﬁned by (Chen et al., 2001). This term is then in-

corporated into the geodesic active contours. In 2007,

(Fang and Chan, 2007) introduced a statistical shape

prior into the geodesic active contour to detect par-

tially occluded object. PCA is computed on level set

functions used as training data and the set of points

in subspace is approximated by a Gaussian function

to construct the shape prior model. An additional ge-

ometric shape prior into region-based active contours

was introduced by (Foulonneau et al., 2004). Prior

knowledge is deﬁned as a distance between shape de-

scriptors based on the Legendre moments of the char-

acteristic function. A new geometric shape prior for

a region-based active contours (Chan and Vese, 2001)

was deﬁned by (Charmi et al., 2010) after alignment

of the evolving contour and the reference shape. The

model has been successful in case of single object in

547

Amine Mezghich M., Sellami M., M’hiri S. and Ghorbel F. (2013).

Robust Object Segmentation using Active Contours and Shape Prior.

In Proceedings of the 2nd International Conference on Pattern Recognition Applications and Methods, pages 547-553

DOI: 10.5220/0004263005470553

 SciTePress

the image. It is well known that the level set approach

solves the problem of topology changing of the para-

metric Snake. However, the introduction of shape

prior by the aforementioned models does to segment

a single object in the image. In this paper we focus on

adding shape prior to Chan and Vese’s model (Chan

and Vese, 2001) based on registration by phase corre-

lation of binary images associated with level set func-

tions of the evolving contour and the shape of ref-

erence. The improved model can retain all the ad-

vantages of (Chan and Vese, 2001) and have the ad-

ditional ability of being able to handle images with

multiple objects in presence of noise and partial oc-

clusions. The remainder of this paper is organized as

follows : In Section 2, we recall the principle of Chan

and Vese’s model. The proposed shape prior and the

used registration method will be presented in Section

3. Experimental results are presented and commented

in Section 4. Finally, we conclude the work and high-

light some possible perspectives in Section 5.

2 REGION-BASED ACTIVE

CONTOURS

The region-based active contours model (Chan and

Vese, 2001) tries to stop the curve evolution with an

energy minimization approach rather than using an

edge-stopping function. Based on level set technique

(Osher and Sethian, 1988), the propagating front cor-

responds always to the zeros level set of an evolving

surface φ. The proposed energy is

(φ, c

, c

) = µ

Ω

δ(φ)|∇φ(x, y)|dxdy

+ν

Ω

H(−φ(x, y))dxdy

+λ

Ω

| f (x, y) − c

H(−φ(x, y))dxdy

+λ

Ω

| f (x, y) − c

H(φ(x, y))dxdy,

(1)

where Ω is the image domain, f is the image intensity,

µ, ν ≥ 0 and λ

, λ

>0 are ﬁxed parameters. H is the

Heaviside function and δ is the Dirac measure. If we

consider z as a level set of φ we can write

H(z) =



1, i f z ≥ 0

0, i f z < 0

, δ(z) =

dH(z)

, (2)

Parameters c

and c

are respectively the averages of

gray-level intensities inside and outside the evolving

curve (Chan and Vese, 2001).

3 SHAPE PRIOR FORMULATION

Our goal is to deﬁne new geometric shape prior to

impose a constraint on the evolving contour to con-

verge to the target shape. If one takes a template shape

which is not deﬁned in the image reference, it is nec-

essary to apply a transformation to align it with the

shape to segment (rotation, translation, scaling fac-

tor). To estimate the parameters of this transforma-

tion, many approaches in literature (section 1) use

alignment methods of the evolving contour and that

of the reference shape. Contour alignment methods

force these approaches to segment only single object

in the image and go without the contribution of the

level set approach, i.e. its ability to segment multi-

ple objects at once. To segment images with multiple

objects, we propose in this work a shape prior that

uses the relative motion parameters, between objects

of the same shape and have different size and orien-

tation, estimated by phase correlation. Initially we

use the method of phase correlation in Fourier space

that is appropriate to estimate the translation vector.

Second, and as discussed in the next section, we use

phase correlation in Fourier-Mellin space for estimat-

ing the rotation and the scaling factor.

3.1 Translation Estimation by Phase

Correlation

Given two images f and g and their Fourier trans-

forms respectively F and G. Suppose that f (i, j) =

g(i − a, j − b). Translation vector (a, b) can be es-

timated by computing the Fourier transform of the

cross-power spectrum given by

ψ(u, v) =

F(u, v)

G(u, v)

= e

−2iπ(u a+v b)

(3)

In Fig.1, we estimate the translation between two bi-

nary images which differ by translation vector equals

to (a, b)=(25, 15). Fig.2 presents the prominent peak

Figure 1: Two binary images which differ by a translation.

which corresponds to the translation vector. We note

that this method of translation estimation by phase

correlation is robust under occlusions and noise,

which often occur in real applications, compared to

translation estimation by centroid which may lead to

wrong values.

ICPRAM2013-InternationalConferenceonPatternRecognitionApplicationsandMethods

548

Figure 2: Fourier transform of the cross-power spectrum.

3.2 Rotation and Scaling Factor

Estimation by the Analytical

Fourier-Mellin Transform

In order to introduce phase correlation in Fourier-

Mellin space, recall that the Fourier-Mellin Transform

(FMT) is the Fourier transform on the group of pla-

nar similarities G = (R

∗

, S

) where S

is the unit cir-

cle. Let f (r, θ) be a polar representation of the image

with the radius r according to the center of the im-

age to offset translation and θ the angle according to

the horizontal. It was pointed out in (Ghorbel, 1994)

that the crucial numerical difﬁculties in computing

the Fourier-Mellin transform of an image might be

solved by using the Analytical Fourier-Mellin Trans-

form (AFMT) given by

(k, v) =

2π

+∞

2π

f (r, θ)r

σ−iv

−ikθ

dθ,

(4)

for all k ∈ Z, v ∈ R, and σ0. The AFMT of an object

f can be seen as the usual FMT of the distorted object

(r, θ) = r

f (r, θ). The AFMT gives a complete de-

scription of gray-level objects since f can be retrieved

by its inverse transform given by

f (r, θ) =

+∞

∑

(k, v)r

−σ+iv

ikθ

dv,

(5)

Since no discrete transform exists, three approxima-

tions of the AFMT have been designed : the direct, the

cartesian and the fast algorithm (Derrode and Ghor-

bel, 2001). In the remainder of this work, we used the

fast algorithm as follows : with a variable change on

the integral (q = ln(r) instead of r), equation (4) can

be rewritten into Fourier transform as follows

(k, v) =

2π

+∞

−∞

2π

qσ

f (e

, θ)e

−i(kθ+qv)

dqdθ,

(6)

The fast algorithm is obtained by computing a two di-

mensional Fast Fourier Transform (FFT) on the log-

polar distorted object e

σq

f (e

, θ). In what follows,

we illustrate the steps of our approach to estimate the

rotation angle and the scaling factor with an exam-

ple of a reference image f

re f

which is rotated 300

◦

partially occluded and submitted Gaussian noise with

deviation equals to 75 (Fig.3). By deﬁnition, a level

Figure 3: Left: Template image, Right: Test image.

set function assigns negative sign for points inside the

evolving contour and positive sign for those outside

it. Thus, we associate with a distance map of a level

set function φ, a binary image f

. Let f

re f

and f

two binary images associated respectively with level

set functions φ

re f

and φ (Fig.4) obtained by Chan and

Vese’s model.

Figure 4: Left: The reference image ( f

re f

), Right: The

transformed image ( f

Let M

re f

and M

the AFMT of the two func-

tions f

re f

and f

. f

re f

and f

have the same shape if

and only if there is a similarity (α

, β

) ∈ G such that

∀(r, θ) ∈ G, f

(r, θ) = f

re f

(

, θ − β

(7)

The action of planar similarities in Fourier-Mellin

space leads to

(k, v) = α

σ−iv

−ikβ

re f

(k, v),

(8)

By calculating the normalized cross-spectrum, only

information on phase difference will be preserved.

Φ(k, v) =

∗

re f

(k, v)M

(k, v)

∗

re f

(k, v)||M

(k, v)|

= α

−iv

−ikβ

(9)

RobustObjectSegmentationusingActiveContoursandShapePrior

549

Phase correlation of two images represented respec-

tively by f

re f

and f

is deﬁned as (Sellami and Ghor-

bel, 2012)

T f m

(α, β) =

+∞

∑

Φ(k, v)α

ikβ

dv,

(10)

Fig.5 shows the surface of phase correlation using

Figure 5: Surface of phase correlation.

the log-polar representation of binary images f

re f

and

. The obtained prominent peak corresponds to (θ =

300

◦

, α = 1).

3.3 The Proposed Geometric Shape

Prior

After estimating the parameters of rigid transforma-

tion between binary images (Fig.4). We perform the

registration of the image f

re f

according to the follow-

ing formula (Chan and Zhu, 2005)

reg

re f

(x, y) =

α f

re f

(

(x−a)cos θ+(y−b)sin θ

−(x−a)sin θ+(y−b)cos θ

(11)

On the resulting left image of Fig.6, the pixels in black

(resp. white) correspond to positive (resp. negative)

areas of the distance map. The image on the right

shows the product function given by

prod

(x, y) = f

reg

re f

(x, y) · f

(x, y),

(12)

By construction, f

prod

is negative in the areas of

Figure 6: Left: f

reg

re f

, Middle : f

, Right : f

prod

variability between the two binary images (occlusion,

clutter, missing parts etc.). Thus we propose to mini-

mize the following energy

shape

Ω

H(g(x, y))dxdy, (13)

where g(x, y)=-φ(x, y) · sign(φ

re f

(x, y)). As we can

see, this energy corresponds to the area of variability

between shapes and it is minimum only if the evolv-

ing contours correspond to the true contours of the

objects to be detected. The new energy’s functional

to be minimized is

E(φ, c

, c

) = w E

+ (1 − w) E

Shape

(14)

where w is a weighting factor. Using the gradient de-

scent method for the level set function φ,

∂E

∂φ

=−

∂φ

∂t

the total discrete evolution equation is

n+1

i j

− φ

i j

∆t

= w δ

(φ

i j

)[−µ K

i, j

+ ν + λ

( f

i, j

− c

)

−λ

( f

i, j

− c

)

] + (1 − w) sign(φ

re f

i, j

) δ

i j

(15)

where K

i, j

= div(

∇φ

i, j

|∇φ

i, j

) and δ

is a regular form of δ

as in (Cremers et al., 2003). Since we are interested

in updating the level set function in the regions of

variability between objects of respectively the refer-

ence and the target images, we take δ

i j

) = 0 for all

points which are outside these regions. This property

recalls the technique of narrow-band used to acceler-

ate the evolution of the level set functions (Malladi

et al., 1995).

4 EXPERIMENTAL RESULTS

We will start by illustrating the ability of the proposed

geometric shape prior to constrain the evolving con-

tour to be similar to a reference shape (by cancel-

ing the Chan and Vese’s energy. i.e. w = 0). Then

we will treat the segmentation problem. In order to

reduce the computational complexity and to have a

good estimation of the parameters of the rigid trans-

formation, we ﬁrst evolve the active contour without

shape prior until convergence as in (Fang and Chan,

2007) and (Foulonneau et al., 2004). This ﬁrst re-

sult provides an initialization for the model with prior

knowledge. Then, the model will evolve under data

and prior terms with an important weight assigned

to the proposed prior term (w = 0.4) to promote the

convergence of the evolving contours to the target ob-

jects.

ICPRAM2013-InternationalConferenceonPatternRecognitionApplicationsandMethods

550

4.1 Robustness of the Proposed Shape

Prior

To illustrate the ability of the proposed shape prior to

constrain geometrically an active contour, we expe-

rience at ﬁrst the evolution of the contour under the

inﬂuence of the proposed energy term E

Shape

. Fig.7,

ﬁrst row, shows the evolution of the initial curve

which is a green rectangle until its convergence to-

ward the contours of two objects. Starting with the

Figure 7: Curve evolution under shape prior only.

ﬁnal contours, shape prior term will constrain these

curves to be similar to the contours of a pen and a

disc with a hole (Fig.7, second row). As it can be

seen, the shape prior allows forcing the evolving con-

tour to ﬁnd the target shapes of an arbitrary number

of objects.

4.2 Application to the Segmentation

We will evaluate the Chan and Vese’s model includ-

ing the proposed shape prior in the context of image

segmentation. We suppose that the image of reference

and its associated level set function are given.

4.2.1 Case of Single Object

In this part of experiments we will start by compar-

ing our model to that proposed by (Foulonneau et al.,

2004) on a synthetic image in which the object is par-

tially covered by a disk (Fig.8). We recall that Foulon-

Figure 8: Left: result obtained by Foulonneau et al.’s model,

Middle and Right: results obtained by our model.

neau et al.’s model presents invariance with respect to

translation and scale factor. As this model is based on

the quadratic distance between Legendre descriptors

of the target object and the template, then a small or-

der does not provide a satisfactory result and a high

order may lead to errors and a huge excution time as

it was mentioned by the authors. It is obvious that

the result obtained by our model is better. The im-

age on the right shows the robustness with respect to

rotation and noise. Table 1 shows the running time

until convergence without shape prior, whereas table

2 presents the running time until convergence with

shape prior. We recall that for rigid motion estima-

Table 1: Computation time for the experiment of Fig.8 (the

middle image) without shape prior.

Execution time

Foulonneau et al.’s model 1 minute

Our model 10.51 seconds

Table 2: Computation time for the experiment of Fig.8 (the

middle image) with shape prior.

Execution time

Foulonneau et al’s model 14 minutes

Our model 26.03 seconds

tion between binary images associated with the level

set functions, we used the AFMT of the log-polar

sampling of the image based on Fast Fourier Trans-

form (FFT 2D). Thus, time for parameter estimation

is 0.412 seconds. We deduce from table 2 that our

model is much faster.

4.2.2 Case of Multiple Objects

In this second part of experiments, we will present the

obtained results for a synthetic image with multiple

objects. Then, we will show those associated to the

real case. We present through the ﬁrst image of Fig.9

the template. The second image is the target image

in which the objects present missing parts. For the

third one, we added a Gaussian noise and for the last

image, the objects were rotated by an angle of −15

◦

The obtained segmentation is given by Fig.10.

Figure 9: The reference image and three target images.

In the next experiment, segmentation of objects

with non trivial topology is considered. In fact ob-

jects with holes can’t be segmented with an edge-

based active contours whereas Chan and Vese’s model

represents the adapted model for this kind of objects.

RobustObjectSegmentationusingActiveContoursandShapePrior

551

Figure 10: Objects detection without shape prior (ﬁrst row)

and with shape prior (second row).

Fig.11 shows the obtained results for a real image. In

the second image of this ﬁgure, the objects are par-

tially occluded and rotated −30

◦

. For the third im-

age, we added a Gaussian white noise with deviation

equals to 75. As it can be seen, the model can detect

the real contours. In (Cremers et al., 2003), the au-

Figure 11: Left: the reference image, Middle: the refer-

ence image with missing parts, partial occlusion and rota-

tion, Right: the reference image with missing parts, partial

occlusion, rotation and noise.

thors proposed a geometric shape prior that can han-

dle multiple objects. However, the problem of invari-

ance to geometric transformations was not treated.

The registration by phase correlation allowed us to

globally align the two binary images to estimate rigid

transformation’s parameters between them to calcu-

late the proposed shape prior. According to Fig.11,

we note that prior knowledge adjusts automatically

depending on missing parts or occlusions unlike (Co-

hen, 1991)’s model where initial position of the curve

must be known in advance to know if the curve should

inﬂate or deﬂate to achieve the desired contours. In

Fig.12, we seek to segment the left and right ventri-

cles of the heart on a real images with missing parts,

rotation and noise effects. The results seem to be sat-

isfactory.

Figure 12: Left: the reference image, Middle: the reference

image with missing parts, Right: the reference image with

missing parts, rotation (−60

◦

) and noise (σ = 75).

5 CONCLUSIONS

To summarize, new method of region-based active

contours with shape prior is presented. The proposed

energy constrains the evolving curve to be similar to a

given template. This approach uses the registration of

level set functions associated with the evolving con-

tour and a reference shape by phase correlation. Ex-

periments have shown the ability of the new added

term to improve the robustness of the segmentation

process in presence of textured background, missing

parts and partial occlusions of the target objects. The

addition of shape prior has not increased signiﬁcantly

the execution time of the algorithm given that the pro-

posed approach does the registration only once and it

is done by the Fast Fourier Transform unlike (Leven-

ton et al., 2000) and (Foulonneau et al., 2004)’s mod-

els where at each iteration parameters estimation is

performed. As future perspectives, we are working

on extending this approach to more general transfor-

mations such as afﬁne transformations.

REFERENCES

Caselles, V., Kimmel, R., and Sapiro, G. (1997). Geodesic

active contours. In Int. J. of Comp. Vis.

Chan, T. and Vese, L. (2001). Active contours without

edges. In IEEE Trans. Imag. Proc.

Chan, T. and Zhu, W. (2005). Level set based shape prior

segmentation. In CVPR.

Charmi, M., Mezghich, M., M’Hiri, S., Derrode, S., and

Ghorbel, F. (2010). Geometric shape prior to region-

based active contours using fourier-based shape align-

ment. In IST.

Chen, Y., Thiruvenkadam, S., Tagare, H., Huang, F., Wil-

son, D., and Geiser, E. (2001). On the incorpora-

tion of shape priors into geometric active contours. In

IEEE Workshop on Variational and Level Set Methods

in Computer Vision.

Cohen, L. (1991). On active contour models and balloons.

In Graphical Models Image Process.

Cremers, D., Sochen, N., and Schnorr, C. (2003). To-

wards recognition-based variational segmentation us-

ICPRAM2013-InternationalConferenceonPatternRecognitionApplicationsandMethods

552

ing shape priors and dynamic labelling. In Interna-

tional Conference on Scale Space Theories in Com-

puter Vision.

Derrode, S. and Ghorbel, F. (2001). Robust and ef-

ﬁcient fourier-mellin transform approximations for

gray level image reconstruction and complete invari-

ant description. In Computer Vision and Image Un-

derstanding.

Fang, W. and Chan, K. (2007). Incorporating shape prior

into geodesic active contours for detecting partially

occluded object. In Pattern Recognition.

Foulonneau, A., Charbonnier, P., and Heitz, F. (2004). Con-

traintes gomtriques de formes pour les contours actifs

orients rgion : une approche base sur les moments de

legendre. In Traitement du signal.

Ghorbel, F. (1994). A complete invariant description for

grey-level images by the harmonic analysis approach.

In Pattern Recognition Lett.

Kass, M., Witkin, A., and Terzopoulos, D. (1988). Snakes :

active contour models. In Int. J. of Comp.Vis.

Leventon, M., Grimson, E., and Faugeras, O. (2000). Sta-

tistical shape inﬂuence in geodesic active contours. In

Proc. of IEEE Conference on Computer Vision and

Pattern Recognition.

Malladi, R., Sethian, J., and Vemuri, B. (1995). Shape mod-

eling with front propagation: A level set approach. In

IEEE Trans. Patt. Anal. Mach. Intell.

Osher, S. and Sethian, J. (1988). Fronts propagating

with curvature-dependent speed: algotithms based on

hamilton-jacobi formulation. In J.of Computational

Physics.

Sellami, M. and Ghorbel, F. (2012). An invariant similarity

registration algorithm based on the analytical fourier-

mellin transform. In EUSIPCO.

RobustObjectSegmentationusingActiveContoursandShapePrior

553