The ﬁrst row of C is initialized by the corresponding
values from S. Starting from the second row, the cu
mulative weight at position (i, j) is computed for each
column as:
C(i, j) = min
n∈N(i,j)
[C(n)+T (n, (i, j)) + S(i, j)]
where N(i, j) is the set of possible predecessors
of (i, j) in the last row i − 1. In the case of 8
neighborhood, N(i, j) is usually set to be {(i −1,j−
1), (i−1,j), (i−1,j+1)}. T (n, (i, j)) represents the
transition weight from the predecessor n to the cur
rent position (i, j). In addition to computing C(i, j),
a pointer is set to the predecessor n that achieves the
minimum among all predecessors in N (i, j). To de
termine the optimal path we follow the pointers from
the last row to ﬁrst row, starting from the position in
the last row of C with the maximal value.
The standard dynamic programming can be easily
extended to handle closed contours. For this purpose
we need a point p in the interior of the contour. Then,
a polar transformation with p being the central point
brings the original image into a matrix, in which a
closed contour becomes a contour from top to bot
tom afterwards. The optimal path is computed in this
polar space and mapped back to the original image.
Note that this approach works well for all starshaped
contours including convex contours as a special case.
Currently, we ask the user to mark a single point
within the reﬂection point (although this part could be
automated as well). The contours of the iris and the
reﬂection point differ in the corresponding edge types.
In the polar transformed image of the eye, the con
tour of the reﬂection point is lying on an edge from
bright to dark and in contrast the contour of the iris
is lying on an edge from dark to bright. The weight
matrix, which is used for the dynamic programming
algorithm to detect the contours, is computed by ﬁ
nite backward differences (reﬂection point) and ﬁnite
forward (iris), respectively.
For the iris contour an additional step is introduced
to produce a smoother contour. Based on the points
on the detected iris contour we apply the method of
least median of squares (Roth and Levine, 1993) to
obtain a circle which will be used in the subsequent
operations.
The eye contour is detected after the iris is re
moved; see Figure 2(c) and Section 3.2. In princi
ple exactly the same procedure as used for detecting
the contour of the iris and the reﬂection point could
be applied here as well. However, based on the spe
cial structure of the eye part we combine two different
sources of edge magnitude to enhance the detection
quality. The ﬁrst source is the standard edge mag
nitude e
image
; see Figure 3(a) for an example where
the edge magnitude was computed by the Canny oper
ator. The second source results from an edge magni
(a) (b) (c)
Figure 3: Combined edge magnitude for detecting the eye
contour: (a) e
image
; (b) e
diff
; (c) combination.
tude computation e
diff
in an image of difference be
tween the red and the green channel; see Figure 3(b).
These two sources are combined by a weighted sum
e
image
+3· e
diff
. The result given in Figure 3(c)
shows a more uniform appearance of the eye contour
in this combined edge magnitude. This representa
tion is then transformed into the polar space in Fig
ure 4(a) and the application of dynamic programming
ﬁnds the optimal contour in Figure 4(b). Mapping it
back to the original image, we ﬁnally obtain the eye
contour as shown in Figure 2(d).
3.2 Iris Removal by Inpainting
For simulating strabismus the iris has to be removed
and placed at a new position within the eye contour
according to the desired angle. At least part of the
original iris area thus must be ﬁlled with new content.
The iris removal should be done with care. Typi
cally, the eyelashes partially interlay with the iris so
that a straightforward removal and subsequently ﬁll
ing this area by the eye background would produce
unnatural appearance in the eyelash (which was ac
tually observed in the experiments done in the initial
phase of this work). Instead, we have to ﬁll the miss
ing background and continue the missing eyelashes in
a natural way simultaneously.
We propose to apply image inpainting techniques
for this purpose. In our current implementation the in
painting algorithm reported in (Criminisi et al., 2000)
is used, due to its advantage of performing texture
synthesis under full consideration of the constraints
imposed by linear structures. Let Ω be the region of
the removed iris to be ﬁlled and δΩ its boundary (the
ﬁll front). For every point p ∈ δΩ a term called prior
ity is computed:
P (p)=C(p)D(p)
where C(p) is the conﬁdence term and D(p) the data
term and they are deﬁned as follows:
C(p)=
q∈ψ
p
∩(I−Ω)
C(q)
 ψ
p

,D(p)=
∇I
⊥
p
· n
p

α
where I represents the whole image, ψ
p
a small
square neighborhood of p, n
p
the normal to the
SYNTHESIZING FACE IMAGES BY IRIS REPLACEMENT  Strabismus Simulation
43