• a smoothness term E
smooth
, which makes neigh
boring pixels in the same image tend to have simi
lar disparities.
3 ENERGY BASED METHOD
The method we use in this paper is energy based. It
has the following features:
• We consider a weakly calibrated stereoscopic sys
tem. The stereoscopic system is not calibrated and
only the knowledge of the socalled fundamental
matrix is known.
• This method addresses the problem of accurately
determining the dense disparity map while regular
izing it along the contours of the gray level image
and inhibiting smoothing across the image discon
tinuities.
• We apply a multi–resolution scheme in order to
avoid convergence to irrelevant minima.
The energy function that we propose for 3D geom
etry reconstruction is as follows:
E(λ) =
Z
(I
l
(x) − I
r
(x + h(λ(x)))
2
dx
+ C
Z
Φ (∇I
l
, ∇λ) dx. (2)
In this case we have a matching function, h, that
depends on a scalar function, λ. This scalar function
represents the displacement of pixels on the epipolar
lines. In this case Φ (∇I
l
, ∇λ) = ∇λ
t
·D (∇I
l
)·∇λ,
D (I
l
∇) is a regularized projection matrix perpen
dicular to ∇I
l
,
D (∇I
l
) =
1
∇I
l

2
+ 2υ
2
·
(
∂I
l
∂y
−∂I
l
∂x
∂I
l
∂y
−∂I
l
∂x
t
+ υ
2
Id
)
(3)
where Id denotes the identity matrix. This projec
tion has been introduced by Nagel and Enkelmann in
the context of optical ﬂow estimation.
After minimising this energy and applying a
gradient descent method we obtain the following
diffusion–reaction PDE:
∂λ
∂t
= C div (D (∇I
l
) ∇λ)
+
I
l
(x) − I
λ
r
(x)
·
−b
∂I
r
∂x
λ
(x)
√
a
2
+ b
2
+
a
∂I
r
∂y
λ
(x)
√
a
2
+ b
2
(4)
The details of this method could be found in paper
(Alvarez et al., 2002).
4 COMBINING GRAPHCUTS
AND STEREOFLOW METHOD
In this section, we explain how the graphcuts (kz2)
and the previous explained PDE (stereoFlow) meth
ods work together for estimating the dense disparity
map. The graphcuts method labels the image obtain
ing a disparity map in integer precision. The stere
oFlow method obtains a disparity map in ﬂoat preci
sion. To improve the perfomance of our method, we
do not apply the graphcuts method in the input pair
of images. Using a pyramidal approach we scale the
image ”n” times. The number of scales is a parameter
deﬁned by user.
The basic idea of embedding our method in a pyra
midal approach is as follows: we replace the images
I
l
and I
r
by I
σ
l
:= Z(I
l
) and I
σ
r
:= Z(I
r
), where
Z(. . .) is the zoom operator. Thus, we do a 2X
zoom over each image. We start with a large ini
tial scale σ
0
. Next, we choose a number of scales
σ
n
< σ
n−1
< ··· < σ
0
and for each scale σ
i
we do
a zoom. When we reach last scale (σ
n
), we compute
the disparity λ
σ
n
with kz2 or with a correlation based
technique. Thus, we have an initial approximation.
Next, we compute the disparity λ
σ
i
as the asymptotic
state of the above PDE with initial data λ
σ
i+1
. So, the
disparity of I
l
and I
r
is deﬁned by λ
σ
0
. In Fig. 1, we
see an example how this algorithm works.
Both correlation–based and graphcuts methods
spend much CPU time to compute the disparity maps.
As we can see in Fig. 1, the kz2 is applied at the
smallest size of the images, so we assure that it is car
ried out faster than at larger images. Then the stere
oFlow technique is applied in the rest of the scales.
5 EXPERIMENTAL RESULTS
In this section we present a comparison between the
graph–cuts stereo method (kz2) and the combina
tion of our method (stereoFlow) with different ini
tial approximation (such as kz2 or correlation based
technique). We have used two datasets in our tests:
a stereoscopic pair from the University of Tsukuba
(Fig. 2) and a stereoscopic pair of a synthetic cylin
der (Fig. 6).
In paper (Kolmogorov et al., 2001), the head of
Tsukuba was used to show the results obtained with
graphcuts stereo method (kz2) in comparison with
similar methods. We have used the same dataset to
COMBINING TWO METHODS TO ACCURATELY ESTIMATE DENSE DISPARITY MAPS
211