RELATIONS BETWEEN RECONSTRUCTED 3D ENTITIES
Nicolas Pugeault
University of Edinburgh, United Kingdom
Sinan Kalkan, Florentin W
¨
org
¨
otter
University of G
¨
ottingen, Germany
Emre Baseski, Norbert Kr
¨
uger
Syddansk University, Denmark
Keywords:
Uncertainty, stereo, reconstruction, relations, geometry.
Abstract:
In this paper, we first propose an analytic formulation for the position’s and orientation’s uncertainty of local
3D line descriptors reconstructed by stereo. We evaluate these predicted uncertainties with Monte Carlo
simulations, and study their dependency on different parameters (position and orientation). In a second part,
we use this definition to derive a new formulation for inter–features distance and coplanarity. These new
formulations take into account the predicted uncertainty, allowing for better robustness. We demonstrate the
positive effect of the modified definitions on some simple scenarios.
1 INTRODUCTION
Many computer vision applications make use of 3D
objects models, provided to the system. Because
these models are designed specifically for the task at
hand, they can be precise, rich, and concise at the
same time, and thereby simplify greatly reasoning
problems. A common problem then is to relate the vi-
sually reconstructed 3D information about the scene
with this accurate model knowledge. Local descrip-
tors, as presented in section 3, have the advantage of
being numerous and of describing the shape of the
objects being witnessed. Their downside is that they
describe only a small part of the object, and there-
fore are not very distinctive, and that objects are not
uniquely described by local descriptors, due to sam-
pling. Therefore it is advantageous to consider, beside
the primitives themselves, relations between them:
distance, collinearity, coplanarity, etc. For example, a
square is described by parallel and orthogonal strings
of collinear 3D–primitives, positioned at fixed dis-
tance one from the other — see (Baseski et al., 2007)
for a discussion of visual representation with primi-
tives’ relations.
0
A more detailed version of this study, containing all
calculations, is available as a technical report, see refer-
ence (Pugeault et al., 2007).
When using exogenous knowledge about the ob-
jects in the scene, and the relations that define them,
one need to consider the fact that primitives are
only reconstructed up to a certain precision see,
e.g, (Hartley and Zisserman, 2000). Thus, inter–
primitives relations can only be defined up to a cer-
tain tolerance that depends on primitive uncertainty.
Moreover, the selectivity of a relation is inversely pro-
portional to this tolerance. A primitive’s uncertainty
is function of image noise, calibration imprecision,
and inaccuracies in primitive extraction, stereopsis,
and reconstruction processes. This leads to large vari-
ations in primitives’ uncertainties across the visual
field. Assuming that a primitive’s position and orien-
tation error have Gaussian distributions, their uncer-
tainties can be encoded by covariance matrices see,
e.g., (Clarke, 1998). A primitive’s position uncer-
tainty can be represented as an ovoid volume in space,
centred on the correct position, and containing the
plausible reconstructed positions; similarly, orienta-
tion’s uncertainty forms a distorted cone. This is illus-
trated in Fig. 1. In this work we will model parame-
ters uncertainty by their covariance matrices, and pre-
dict their propagation using an analytical first order
approximation proposed by (Durrant-Whyte, 1988;
Faugeras, 1993; Clarke, 1998). This is discussed in
the first part of this paper, in section 4.
The computation of inter–primitives relations can
186
Pugeault N., Kalkan S., Wörgötter F., Baseski E. and Krüger N. (2008).
RELATIONS BETWEEN RECONSTRUCTED 3D ENTITIES.
In Proceedings of the Third International Conference on Computer Vision Theory and Applications, pages 186-193
DOI: 10.5220/0001083901860193
Copyright
c
SciTePress
-0.5
0
0.5
-0.5
0
0.5
995
1000
1005
z
x
y
z
Figure 1: Illustration of the uncertainty. The red ovoid
shows the position’s uncertainty, and the green cone the ori-
entation’s uncertainty. The axes of the ellipse and the cone
are computed from the Eigen–values and associated Eigen–
vectors of the covariance matrices.
be severely affected by the imprecision in the 3D–
primitives’ reconstruction. For example, consider
the collinearity relation. If we make abstraction of
the primitives’ imprecision, we can use the stan-
dard mathematical definition: two 3D–primitives are
collinear if their orientation is parallel to the line that
joins them. Now if we add some imprecision in
the reconstruction process, these orientations will be
slightly different. Normally this could be addressed
by setting a threshold on the orientation difference,
but the primitives’ uncertainty depends on parameters
such as its orientation and position in space. In other
words, there is no single threshold that can be set to
define collinearity adequately for all cases. In the sec-
ond part of this paper, in section 5, we will consider
two relations:distance 5.1 and coplanarity 5.2. For
each relation we propose a classic Euclidian formu-
lation, and a second one taking into account the prim-
itives’ uncertainty, in a manner reminiscent of the Ma-
halanobis distance. We compare the robustness (how
regularly correct primitives pairs are identified) and
selectiveness (how often primitives are erroneously
paired) of the two formulations.
2 LITERATURE REVIEW
The computation, and propagation of uncertainties
has been studied for long, in particular in the field
of photogrammetry, yet for the sake of concision,
we will focus on studies related to computer vision.
Verri and Torre (Verri and Torre, 1986) studied re-
constructed points’ depth accuracy, and found that the
length of the baseline is critical for the accuracy. Cri-
minisi and colleagues (Criminisi et al., 1997) studied
point reconstruction uncertainty for planar surfaces.
Rodr
`
ıguez and Aggarwal (Rodr
`
ıguez and Aggarwal,
1988) proposed to approximate reconstruction uncer-
tainty by the relative range error, and Mandelbaum
and colleagues (Mandelbaum et al., 1998) handle the
depth uncertainty as a minimax risk confidence inter-
val. Kamberova and Bajcsy (Kamberova and Bajcsy,
1998) make use of such intervals to reject data points.
These works only consider the depth uncertainty in
the case of point reconstruction. The proposed formu-
lations do not allow for an easy inclusion of additional
parameters. Hartley and Zisserman (Hartley and Zis-
serman, 2000) argue that the angle between the op-
tical rays back–projected by a pair of image points
yields a better estimate of the reconstructed point’s
covariance than the disparity. Wolff (Wolff, 1989)
discussed the stereo–reconstruction of lines, and pro-
pose an estimation of the reconstructed orientation’s
uncertainty, demonstrating that reconstructing lines as
an intersection of planes lead to a better accuracy than
reconstructing the lines’ endpoints. The proposed an-
alytical derivation is less general specific than the one
used in this paper. Clarke (Clarke, 1998) also sug-
gests to use Monte–Carlo simulation to estimate un-
certainty, but points out the extreme computational
cost of this approach. We argue that this approach
is impractical when taking additional parameters into
account (orientation, sparseness, cameras’ projection
matrices), but provides an efficient way to evaluate
an analytic derivation (see section 4.4). Heuel and
colleages (Heuel and F
¨
orstner, 2001) proposed a 3D
line reconstruction using uncertain geometry. Their
approach focuses on polyhedral objects, whereas the
primitive–based framework used herein allows the
representation of curved contours using local edge de-
scriptors. This locality aspect requires us to recon-
struct a position on the reconstructed 3D–line.
In this work, we first estimate the 2D–primitive’s
extraction process uncertainty, then describe how it
propagates to 3D–primitives, using the formulation
proposed by (Durrant-Whyte, 1988; Faugeras, 1993;
Clarke, 1998). Note that (Haralick, 2000) discussed
the uncertainty propagation of processes based on
function minimisation, applied to computer vision.
Additional uncertainties stem from the projection ma-
trices (these should be obtained from camera calibra-
tion), from stereo matching (an estimation is proposed
here), and local curvature (that we will neglect in this
paper). We model parameters’ uncertainties with their
covariance matrices (see, e.g., (Clarke, 1998)). The
most similar work is the study of F
¨
orstner and col-
leagues (F
¨
orstner et al., 2000) that use Grassman al-
gebra to evaluate the confidence in several relations
between geometric entities. Their representation only
RELATIONS BETWEEN RECONSTRUCTED 3D ENTITIES
187
(a) image (b) 2D–primitives
(c) 2D–primitives (detail) (d) 3D–primitives (detail)
Figure 2: Illustration of the primitive–based vision frame-
work presented in (Kr
¨
uger et al., 2007) and used in this
study.
handles global lines, though, and is inappropriate for
local line descriptors. Moreover, they do not discuss
the coplanarity nor distance relations.
3 THE PRIMITIVE–BASED
VISION FRAMEWORK
In this paper we make use of a framework proposed
in (Kr
¨
uger et al., 2007). This representation describes
the image in terms of a sparse set of local, multi–
modal line descriptors called 2D–primitives. In this
work we are only interested in the primitives’ posi-
tion (m) and local orientation (defined by the tangent
vector t).
1
Therefore, primitives can be regarded as
local tangents to image contours. In this work, primi-
tives are extracted using the monogenic signal for the
early vision processing, but it is worthwhile to note
that Gaussian or Gabor wavelets could alternatively
be used — see (Sabatini et al., 2006) for a discussion.
A stereo–pair of 2D–primitives allows to recon-
struct a 3D–primitive: a local 3D contour descrip-
tor (which position is defined by M and orientation
by the tangent vector T ). Fig. 2 illustrates the 2D–
primitive extraction and 3D–primitive reconstruction
processes: (a) shows an image from an indoor naviga-
tion scenario; (b) shows the extracted 2D–primitives,
with a detail on the traffic sign in (c); finally, (d)
shows the 3D–primitives reconstructed by stereo.
1
Primitives also hold some aspect parameters such as
colour and phase, that are useful for, e.g., the stereo–
matching process. See (Kr
¨
uger et al., 2007).
4 COMPUTING UNCERTAINTIES
Assuming that the error of a vector x has a Gaussian
distribution, its uncertainty can be represented by its
covariance matrix Λ
x
. The uncertainties of the prim-
itive extraction has been evaluated in (Kr
¨
uger et al.,
2007), and therefore we only need to study how this
uncertainty is propagated by the stereo reconstruction
process.
4.1 Uncertainty Propagation
Given a function y = f (m), where x and y are vectors
with associated covariance matrices Λ
x
and Λ
y
, a first
order Taylor series expansion gives us:
f (x + x) = f (x) + f (x) · x + O(||x||
2
) (1)
from there (Clarke, 1998) derives that the relation be-
tween the covariance matrices of m and y is approxi-
mated by the relation
Λ
y
f · Λ
x
· f
>
(2)
where f is the Jacobian matrix for the function
f . This is the main result used hereafter to estimate
uncertainties’ propagation during stereo reconstruc-
tion. In the following we will equivalently denote
Λ = σ
2
the variances of scalar values, and Λ the co-
variance matrices of vector quantities. Also, in the
one–dimensional case, f (x) =
f (x)
x
is the derivative
of f (x).
4.2 2D–Primitive Uncertainty
In (Kr
¨
uger et al., 2007), the 2D–primitives’ position
and orientation error were evaluated. Although this
error depends on local noise, texture and blur, we will
assume in the following that these factors are con-
stant. Because a 2D–primitive is a local line descrip-
tor, the position error is only significant in the direc-
tion normal to this primitive’s orientation.
2
Therefore,
a primitive’s position covariance is approximated by:
Λ
˜m
= ε
2
·
sin(θ)
cos(θ)
0
· Λ
θ
·
sin(θ) cos(θ) 0
(3)
where ε was evaluated in (Kr
¨
uger et al., 2007) to
ε ' 0.0625. Note that this covariance matrix de-
scribes the 2D–primitive’s homogeneous position ˜m,
and therefore its third dimension’s variance is null. A
2D–primitive’s orientation variance is approximated
to its mean square error, evalutated in (Kr
¨
uger et al.,
2007) to Λ
θ
' 9 · 10
4
radians.
2
Note that this is only true if the local curvature is small
with regards to the position error. In general this assumption
is true, as large curvatures lead to the extraction of corners,
rather than lines primitives — see (Kr
¨
uger et al., 2007).
VISAPP 2008 - International Conference on Computer Vision Theory and Applications
188
4.3 Reconstruction Uncertainty
We then study the propagation of 2D–primitives’ un-
certainty during stereo–reconstruction, and estimate
the resulting 3D–primitives’ uncertainty.
The relation between points in space and their pro-
jection in the image is defined by the camera’s projec-
tion matrix
˜
P = (P p) (see (Faugeras, 1993; Hartley
and Zisserman, 2000)). In the following, and for the
sake of simplicity, we assume that the cameras’ pa-
rameters are known, and their projection matrix exact
Λ
˜
P
= 0
12×12
. In the general case, the projection ma-
trix will be estimated empirically through a process
called calibration that provides its uncertainty as a
by–product (Csurka et al., 1997). The precise deriva-
tion of the projection matrix uncertainty depends on
the format of the uncertainty provided by the calibra-
tion software. In the case of the Matlab calibration
toolbox (see (Bouguet, 2007)), the reader can find the
derivation of the projection matrix uncertainty in the
technical report (Pugeault et al., 2007).
Classical stereo–reconstruction tries to intersect
two optical rays containing the possible origins of (or
back–projected by) two corresponding points in two
images. Because of imprecision, it is unlikely that
the two lines intersect, and therefore the closest point
to both rays is usually chosen. This approach is in-
adapted in the case of local line descriptors because
the aperture problem makes reliable point matching
impossible. On the other hand, (Wolff, 1989) dis-
cussed that accurate line matching could be achieved
by intersecting the two planes back–projected from
the lines in each image. Moreover, because prim-
itives are local line descriptors we need a location
along this line. This is obtained by intersecting the
line containing the left 2D–primitive’s position pos-
sible origins with the plane containing the right 2D–
primitive’s possible origins. The computation of the
3D–primitives’ uncertainty is using the uncertainty
propagation formula in Eq. 2, as in (Clarke, 1998;
Heuel and F
¨
orstner, 2001). The computation of the
Jacobians will not be detailed here because of space
constraints.
4.4 Evaluation
We evaluate the quality of the uncertainties predicted
by the above formulae, using a Monte Carlo simu-
lation in a simple scenario. The focal length is set to
f = 10
3
and the baseline to b = 100, so that the optical
centres of the cameras are located at C
1
= (0, 0, 0)
>
and C
2
= (b, 0, 0)
>
.
3
3
These values were chosen for simplicity, but are never-
theless plausible: they are similar to the calibration param-
Consider a 3D–primitive at a location
ˆ
M =
(0, 0, 100)
>
and with an orientation
ˆ
T , projected on
both image planes as
ˆ
π
l
and
ˆ
π
r
. We apply a zero–
mean Gaussian perturbation on position and orienta-
tion of those 2D–primitives, with a standard devia-
tion of σ = 0.25 for position, and σ = 0.03 for ori-
entation. This is according to the measured mean
square error we assumed for our covariance predic-
tion. Because we are only interested in the reconstruc-
tion uncertainty, we assume that
ˆ
π
l
and
ˆ
π
r
are accu-
rate, and that all uncertainty comes from the added
perturbation, and therefore the covariance of the pro-
jected 2D–primitive’s position is Λ
m
= 0.0625I
2×2
;
they have a vertical orientation (i.e., θ = 0) with a
variance of Λ
θ
= 9 · 10
4
. Using a Monte Carlo sim-
ulation of 10
5
particles, we measured a relative error
between predicted and measured covariance matrices
ξ =
kΛ
0
Λk
kΛ
0
k
of 3% for position, and 4% for ori-
entation.
We then investigated how the 3D–primitive’s po-
sition and orientation impact the uncertainty thereof.
We compared the trace tr (Λ) of the reconstructed
position’s covariance matrix (sum of the Eigen–
values), at different locations in space (Figs. 3(a),
3(b), and 3(c) for different values of the x (horizon-
tal), y (vertical), and z (depth) coordinates) and for
different pairs of 2D–orientations (Fig. 4(a)).
These figures show that the reconstructed posi-
tion’s covariance is affected by the distance from the
primitive to the cameras’ optical centres and by the
right 2D–primitive’s orientation. The trace tr (Λ
m
)
in Fig. 4(a) is mostly affected by θ
2
. This is due to
the line reconstruction formula used in this work
see section 4.3. In this formulation, the right 2D–
primitive’s orientation is used to resolve the ambigu-
ity that stems from the aperture problem (we compute
the intersection between a back–projected left ray and
a back–projected right plane). This becomes impos-
sible when the primitive’s orientation is the same that
the epipolar line’s (in this case if θ
2
=
π
2
), and there-
fore the reconstructed 3D–primitive’s position uncer-
tainty increases to infinity for orientations close to
π
2
.
We then evaluated the 2D–primitives’ orientation
impact on the reconstructed 3D–primitive’s orienta-
tion uncertainty. Fig. 4 plots the trace of the recon-
structed orientation’s covariance matrix for a point
located at m = (0, 0, 100)
>
, reconstructed from dif-
ferent 2D–primitives’ orientations. In this figure we
see that the reconstructed orientation uncertainty in-
creases when either of the 2D–primitive’s orientation
becomes close to
π
2
. When both orientations become
close to θ
1
= θ
2
=
π
2
two primitives back–project the
eters of an actual stereo camera system.
RELATIONS BETWEEN RECONSTRUCTED 3D ENTITIES
189
-5000
0
5000
-5000
0
5000
0
10
20
tr(Λ
m
)
x
y
tr(Λ
m
)
(a) xy–Plane
1000
2000
-5000
0
5000
0
2000
4000
tr(Λ
m
)
z
x
tr(Λ
m
)
(b) xz–plane
0
500
1000
1500
2000
2500
3000
0 500 1000 1500 2000 2500
Z
λ
1
λ
2
λ
3
(c) z–axis
Figure 3: Traces of the covariance matrix Λ
M
, for (a) different locations M = (x, y, 100)
>
on the xy–plane; (b) different
locations M = (x, 0, z)
>
on the xz–plane; and (c) just considering the z–axis.
0
0.05
0.1
θ
1
θ
2
0 1.57 3.14
0
1.57
3.14
(a) tr(Λ
M
)
0
0.05
0.1
θ
1
θ
2
0 1.57 3.14
0
1.57
3.14
(b) tr(Λ
T
)
Figure 4: Effect of 2D–primitives’ orientations on (a) the
trace of Λ
M
; and (b) the trace of Λ
T
.
same plane P
1
= P
2
, and therefore their intersection
is undefined.
5 DESIGN OF 3D–PRIMITIVES
RELATIONS
In this section we consider distance and coplanarity
between 3D–primitives, and propose definitions that
take the uncertainties thereof into account, based on
the Mahalanobis distance.
5.1 3D–Primitives Normal Distance
The first relation that we consider is the normal dis-
tance between two reconstructed 3D–primitives. The
normal distance between two primitives Π
1
and Π
2
is
defined as the distance from the line defined by prim-
itive Π
1
position and orientation and primitive Π
2
po-
sition. This is a useful measure when considering lo-
cal line descriptors, as the exact positioning of a prim-
itive along a line is effectively an artefact of sampling.
Namely:
d
n
= k(M
2
M
1
) ×t
1
)k (4)
is the normal distance between Π
1
and Π
2
.
Consider the following scenario: We have three
parallel vertical lines L
A
, L
B
, and L
C
. We have prior
world knowledge available, stating that there a dis-
120 cm
nearby
far
far
nearby
Euclidian
Euclidian
Mahalanobis
Mahalanobis
Figure 5: All primitives that satisfy a normal distance crite-
rion with a selected primitive. The red lines indicate valid
pairs.
tance of a = 50 between the lines L
A
and L
B
, and that
L
C
is further away, at a distance of a + b = 60.
Consider three primitives, located at points M
A
=
(100, 100, z)
>
L
A
, M
B
= M
A
+ u L
B
(u =
(a, 0, 0)
>
) and M
C
= (a + b + 100, 100, z)
>
L
C
, all
vertically oriented. These points’ projections on both
image planes are subjected to a zero–mean Gaussian
perturbation applied to the projected 2D–primitives’
position and orientation, with a standard deviation of
σ = 0.25 and σ = 0.03 respectively. Then we re-
construct the 3D–primitives Π
i
as described in sec-
tion 4.3. We want to use our world knowledge to
VISAPP 2008 - International Conference on Computer Vision Theory and Applications
190
0
0.2
0.4
0.6
0.8
1
0 3000
ratio
depth
ETP
EFP
MTP
MFP
Figure 6: Comparison of the robustness of Euclidian (E)
and Mahalanobis (M) distances, for the values a = 50, b =
5, α = 20, and β = 5.
identify the primitives Π that belong to L
A
, L
B
, and
L
C
. This is illustrated in a concrete scenario in Fig. 5.
In this scenario, we know that the two red lines on
the ground, delimiting the road, are parallel and sep-
arated by a distance a of 120cm. Using this world
knowledge, we search for pairs of primitives that are
separated by this distance, plus or minus 10cm. The
figure shows the valid pairs for nearby and far 3D–
primitives. In each case the red lines indicate with
which other primitive it forms a valid pair according
to each definition for distance.
We compare the performance of different distance
measures for this task:
Euclidian Distance Threshold (E): We defined the
threshold on the Euclidian distance as follows:
|
d
n
(Π
1
, Π
2
) a
|
< α
2
(5)
where d
n
(Π
1
, Π
2
) stands for the normal distance be-
tween Π
1
and Π
2
, as defined in Eq. (4).
Mahalanobis Distance (M): The second criterion
is based on the Mahalanobis distance:
(d
n
(Π
1
, Π
2
) a)
2
· Λ
dn
< β (6)
where Λ
dn
is the variance of the computed normal dis-
tance that comes directly from the uncertainty of Π
1
and Π
2
— see technical report (Pugeault et al., 2007)
for a full derivation.
5.1.1 Evaluation
We compared the performance and robustness of both
formulations using artificial images. We set a = 50,
b = 5, α = 20, and β = 5. The results are sum-
marised in Fig. 6, the true positives curves (ETP and
MTP) express the ratios of experiments wherein the
reconstructed 3D–primitives A
0
and B
0
comply with
the criterion (respectively E and M). The false pos-
itive curves (EFP and MFP) express the ratios of ex-
periments wherein the reconstructed 3D–primitives A
0
and C
0
satisfy the criterion. In this figure, we see
that the number of true positive of the Euclidian cri-
terion (ETP) decreases with depth.
4
On the other
hand, the ratio of true positive (MTP) is stable for the
Mahalanobis distance. The false positives (MFP) in-
crease progressively for large uncertainties, when the
distribution of B and C overlap significantly. This
shows that the normalised Mahalanobis distance is
better suited for drawing spatial relations between re-
constructed 3D–primitives.
This trend is illustrated qualitatively on real im-
ages in Fig. 5. There we have the values: a = 120,
α = 10, and β = 0.5.
5.2 Coplanarity Relation
The second relation we studied is the coplanarity be-
tween two reconstructed 3D–primitives. As before,
we consider three 3D–primitives, A, B, and C, with
cop(A, B) = 1 and cop(A,C) ' 0.70 — this means an
angle of
π
4
. The 3D–primitives are projected onto the
image planes as before, the same Gaussian perturba-
tion is applied, and both coplanarity criteria are ap-
plied to the reconstructed 3D–primitives Π
i
.
Coplanarity is defined as follows:
cop(Π
1
, Π
2
) = (V × T
1
) · (V × T
2
) (7)
where V =
1
||M
2
M
1
||
· (M
2
M
1
). By using Eq.(2) in
Eq.(7) we obtain the variance of the coplanarity mea-
sure:
Λ
cop
=
η
>
2
η
>
1
·
Λ
V ×T
1
Λ
V ×T
2
·
η
2
η
1
(8)
with η
i
= V × T
i
the normal to the plane formed by
the orientation T
i
and the points M
1
and M
2
. There-
fore, we propose the two following criteria for copla-
narity:
Euclidian Coplanarity: The first definition simply
applies a threshold on the coplanarity value:
1 cop(Π
1
, Π
2
) < α (9)
4
Note that the performance of the Euclidian distance (E)
could be improved for a certain region of the space by al-
tering α. Nonetheless, the general trend will be the same:
larger α lead to more false positives for nearby structures,
and the number of true positives tend to zero for far struc-
tures.
RELATIONS BETWEEN RECONSTRUCTED 3D ENTITIES
191
0
0.2
0.4
0.6
0.8
1
0 1000 2000
ratio
depth
ETP
EFP
ATP
AFP
Figure 7: Proportion of coplanar pairs correctly labelled,
using a fixed (E) and a variance dependent threshold (A),
respectively.
(a) Euclidian (b) Mahalanobis
Figure 8: Illustration of the coplanar pairs extracted. The
red lines show the primitives coplanar near (bottom) and far
(top) from the camera.
Mahalanobis Coplanarity: The second definition
makes use of the estimated coplanarity variance to de-
rive a Mahalanobis–like criterion:
Λ
cop
· (1 cop(Π
1
, Π
2
))
2
< β (10)
These two criteria, in Eq. 9 and 10, are compared
in Fig 7, for values α = 0.01 and β = 0.5. In this fig-
ure: ETP is the ratio of cases where Eq. 9 is verified
between A
0
and B
0
, EFP where it is between A
0
and
C
0
; ATP the ratio where Eq. (10) is satisfied between
A
0
and B
0
and AFP the ratio where it is satisfied be-
tween A
0
and C
0
. In Fig. 7 we see that the ratio ETP
reduces quickly with the increase of depth. The ATP
ration, on the other hand, is stable, while the AFP ra-
tio increases with depth. This shows that the variance
adapted threshold is a more robust criterion for recon-
structed features’ coplanarity than the naive Euclidian
criterion, and this across a wide range of depth.
The result is further illustrated in Fig. 8. We see
that when using the Mahalanobis version, the copla-
nar structures (red) are more densely connected than
when using the Euclidian threshold, thus coplanarity
is more reliably asserted. Furthermore, it is visible
that the Euclidian criterion interpretes some of the far-
ther green primitives as coplanar with the red ones.
6 CONCLUSIONS
This paper presented an analytical derivation of the
uncertainty propagation in a vision framework using
the primitives proposed by (Kr
¨
uger et al., 2007), and
the scene description in terms of inter–primitives re-
lations discussed in (Baseski et al., 2007).
In a first part we discussed how image and cali-
bration uncertainty propagates during the reconstruc-
tion process. This result, although classic in nature
(e.g., (Clarke, 1998)), allowed us to formalise the pe-
culiarities in the uncertainty space that stems from
our use of local line descriptors (mainly its strong
dependence on 2D orientation). The derivation pre-
sented here is specific to the representation proposed
in (Kr
¨
uger et al., 2007), yet it could easily be adapted
to other line–based features. The advantage of an
explicit analytic formulation of the uncertainty is, it
allows us to accurately model the whole complexity
of the uncertainty space. Estimating such a high di-
mensional space by Monte Carlo simulation would
be impractical. This analytic derivation of uncer-
tainty propagation was demonstrated to be accurate
by Monte Carlo simulations.
The second and most important part of this paper
considers inter–primitives geometric relations, focus-
ing on the cases of normal distance and coplanarity.
In (Baseski et al., 2007) it was discussed that such
relations form a good base for interpreting visual in-
formation. Moreover, such relations form a way to
provide prior geometrical knowledge about the scene,
and compare this prior knowledge with the recon-
structed 3D representation. Such relations need to
allow for a certain imprecision in the 3D–primitives,
imprecision that is itself a function of the parameters
thereof. The 3D–primitives’ uncertainties computed
in the first part were used to design alternative for-
mulations of those relations that take uncertainty into
account. The new formulations were shown to de-
tect geometric relations in a more robust fashion than
the naive Euclidian ones, and across wide ranges of
depth.
We direct the reader interested in the detailed
derivation of the uncertainties discussed in this paper
towards the more detailed technical report (Pugeault
et al., 2007). Future work includes defining a com-
plete set of relations, and using it to formulate world
knowledge in concrete scenarios.
VISAPP 2008 - International Conference on Computer Vision Theory and Applications
192
ACKNOWLEDGEMENTS
This work was funded by the European project
(DRIVSCO, 2009).
REFERENCES
Baseski, E., Pugeault, N., Kalkan, S., Kraft, D., W
¨
org
¨
otter,
F., and Kr
¨
uger, N. (2007). A scene representation
based on multi-modal 2D and 3D features. In 3D Rep-
resentation for Recognition Workshop (in conjunction
with ICCV).
Bouguet, J.-Y. (2007). Camera Calibration Toolbox
for Matlab. http://www.vision.caltech.edu/
bouguetj/calib_doc/.
Clarke, J. C. (1998). Modelling uncertainty: A primer.
Technical report, Department of Engineering Science,
Oxford University.
Criminisi, A., Reid, I., and Zisserman, A. (1997). A plane
measuring device. In Proceedings of the British Ma-
chine Vision Conference.
Csurka, G., Zeller, C., Zhang, Z., and Faugeras, O. (1997).
Characterizing the Uncertainty of the Fundamental
Matrix. Computer Vision and Image Understanding,
68(1):18–36.
DRIVSCO (2006-2009). DRIVSCO: Learning to Emulate
Perception-Action Cycles in a Driving School Sce-
nario (FP6-IST-FET, contract 016276-2).
Durrant-Whyte, H. F. (1988). Uncertain Geometry in
Robotics. IEEE Journal of Robotics and Automation,
4(1):23–31.
Faugeras, O. (1993). Three–Dimensional Computer Vision.
MIT Press.
F
¨
orstner, W., Brunn, A., and Heuel, S. (2000). Statistically
testing uncertain geometric relations. In Sommer, G.,
Kr
¨
uger, N., and Perwass, C., editors, Mustererken-
nung 2000, pages 17–26. DAGM, Springer.
Haralick, R. M. (2000). Propagating covariance in com-
puter vision. In Proceedings of the Theoretical Foun-
dations of Computer Vision, TFCV on Performance
Characterization in Computer Vision, pages 95–114,
Deventer, The Netherlands, The Netherlands. Kluwer,
B.V.
Hartley, R. and Zisserman, A. (2000). Multiple View Geom-
etry in Computer Vision. Cambridge University Press.
Heuel, S. and F
¨
orstner, W. (2001). Matching, reconstruct-
ing and grouping 3d lines from multiple views using
uncertain projective geometry. In CVPR ’01. IEEE.
Kamberova, G. and Bajcsy, R. (1998). Sensor Errors and the
Uncertainties in Stereo Reconstruction. In K. Bowyer
and P. Jonathon Phillips, editor, Empirical Evaluation
Techniques in Computer Vision. IEEE Computer Soc.
Press.
Kr
¨
uger, N., Pugeault, N., and W
¨
org
¨
otter, F. (2007). Multi-
modal primitives: local, condensed, and semanti-
cally rich visual descriptors and the formalization of
contextual information. Technical Report 2007-4,
Robotics Group Maersk Institute, University of South-
ern Denmark.
Mandelbaum, R., Kamberova, G., and Mintz, M. (1998).
Stereo depth estimation: a confidence interval ap-
proach.
Pugeault, N., Kalkan, S., Baseski, E., W
¨
org
¨
otter, F., and
Kr
¨
uger, N. (2007). Reconstruction uncertainty and
3d relations. Technical Report 6, Maersk Mc–Kinney
Moller Institute, University of Southern Denmark.
Rodr
`
ıguez, J. J. and Aggarwal, J. K. (1988). Quantization
error in stereo imaging. In Proceedings of the CVPR.
Sabatini, S., Gastaldi, G., Solari, F., Pauwels, K., van Hulle,
M., D
´
ıaz, J., Ros, E., Pugeault, N., and Kr
¨
uger, N.
(2006). Compact and accurate early vision processing
in the harmonic space. In 2nd International Confer-
ence on Computer Vision Theory and Applications.
Verri, A. and Torre, V. (1986). Absolute depth estimate
in stereopsis. Journal of Optical Society of America,
3:297–299.
Wolff, L. B. (1989). Accurate measurements of orientation
from stereo using line correspondence. In IEEE Com-
puter Society Conference on Computer Vision and
Pattern Recognition (CVPR).
RELATIONS BETWEEN RECONSTRUCTED 3D ENTITIES
193