MEASUREMENT NOISE IN PHOTOMETRIC STEREO BASED
SURFACE RECONSTRUCTION
Toni Kuparinen, Ville Kyrki
Department of Information Technology, Lappeenranta University of Technology, P.O. Box 20, 53851 Lappeenranta, Finland
Pekka Toivanen
Department of Computer Science, University of Kuopio, P.O.Box 1627, 70211 Kuopio, Finland
Keywords:
Surface reconstruction, photometric stereo, gradient fields, Fourier domain, Wiener filter, denoising, white
noise.
Abstract:
In this paper, we present a noise reduction method for photometric stereo based surface reconstruction of
surfaces with high frequency height variation. Such surfaces are important for many industrial settings, for
example, in paper and textile manufacturing. The paper presents the derivation of the effect of white image
noise to gradient fields. Based on the derivation, a denoising approach of the gradient fields using the Wiener
filter is proposed. Several known surface reconstruction methods with and without the proposed denoising
approach are evaluated experimentally, with respect to the effect of the noise, and the boundary conditions
of the reconstruction. The experimental results validate that the proposed approach improves the surface
reconstruction on surfaces with high frequency height variation.
1 INTRODUCTION
Surface topography is a highly important quality pa-
rameter in many industrial applications, such as paper
and textile manufacturing. Undesired surface topog-
raphy variations can reflect imperfections in manufac-
turing process, product operational efficiency, and life
expectancy. Depth recovery techniques, such as shape
from shading (SfS) (Horn, 1990), and photometric
stereo (PS) (Woodham, 1978), can provide surface
gradients in a fast and non-contact manner. In order
to obtain the surface topography, that is, the relative
height values of the surface, the differential surface
gradients have to be integrated. However, in practice
the surface gradients are corrupted by noise, resulting
from imaging and other measurement errors.
Several solutions have been proposed to integrate
the measured gradient fields. A traditional method
for integrating the surface height from gradient infor-
mation is the Frankot-Chellappa algorithm (Frankot
and Chellappa, 1988). Another popular method is the
Poisson solver (Simchony et al., 1990). Typically,
the performance of the surface reconstruction meth-
ods has been evaluated on surfaces with large, smooth
objects, such as flower pots, faces, peaks, and ramps
with rather strong additive Gaussian noise (Noakes
and Kozera, 2003; Karacali and Snyder, 2004; Wei
and Klette, 2003; Agrawal et al., 2006).
However, monitoring of surface roughness and
texture, that is, higher frequency variations with lim-
ited noise levels, are frequently of interest in manu-
facturing processes. McGunnigle (McGunnigle and
Chantler, 2003) presented a framework for measure-
ment and modelling of rough surfaces. They evalu-
ated several surface description models, but did not
reconstruct surfaces with or without noise. Recently
Hansson (Hansson and Johansson, 2000) has studied
two-light photometric stereo in paper surface recon-
struction. Based on Hansson’s work, Kuparinen ex-
tended the Hansson’s two light method to four-light
photometric stereo using Symmetrical binary weights
in (Kuparinen et al., 2007). Kuparinen also showed
that the minimization based Frankot-Chellappa and
Poisson methods smooth the reconstructed surface re-
moving the inherent high-frequency containing sig-
nal (Kuparinen et al., 2007).
This paper studies the photometric surface recon-
struction of high-frequency varying surfaces. As par-
ticular contributions of the paper, a model of noise for
the gradient images is derived, and an evaluation of
the noise on reconstructed surfaces is presented. In
addition, a denoising method based on Wiener filter-
571
Kuparinen T., Kyrki V. and Toivanen P. (2008).
MEASUREMENT NOISE IN PHOTOMETRIC STEREO BASED SURFACE RECONSTRUCTION.
In Proceedings of the Third International Conference on Computer Vision Theory and Applications, pages 571-576
DOI: 10.5220/0001083805710576
Copyright
c
 SciTePress

τ
σ
Camera
x−axis
Light
Figure 1: Geometry of the imaging setup. (Kuparinen et al.,
2007)
ing is presented and evaluated on both artificial and
real data. Drbohlav (Drbohlav and Chantler, 2005)
derived also models for additive noise in photomet-
ric stereo, but did not present experimental results on
noise reduction. Finally, the effect of the boundary
conditions of the reconstruction is studied.
2 PHOTOMETRIC STEREO
2.1 The Estimation of Gradient Field
In photometric stereo, the viewing direction is held
constant while the direction of the illumination be-
tween successive images is varied. Image radiance
values in successive views are used to determine the
surface orientation at each image point (Woodham,
1978). For Lambertian surfaces, the reflected inten-
sity is independent of the viewing direction. How-
ever, the intensity depends on the direction of the light
source.
Lambert’s Law (Lambert, 2001) represents the
image intensity i at the point (x,y)
i = ρλ(l
T
·n), (1)
where ρ is the surface albedo, λ is the in-
tensity of the light source, n = [n
1
,n
2
,n
3
]
T
=
[p,q,1]
T
√
p
2
+q
2
+1
is the unit normal to the surface and l =
[cos(τ)sin(σ),sin(τ)sin(σ),cos(σ)]
T
is the unit vec-
tor toward the light source. Elements p and q are
surface partial derivatives measured along the x and
y axes, respectively. The angles of illumination, the
tilt τ, and slant σ are illustrated in Fig. 1. Orthogonal
projection and constant illumination over the surface
are assumed in Lambert’s Law.
For a four-light case, if the same slant angle σ is
used for all light sources, and the tilt angles 0
◦
, 180
◦
,
90
◦
and 270
◦
are used for images i
1
through i
4
, re-
spectively, the surface gradient fields can be derived
using photometric stereo as follows:
p =
2
tan(σ)
i
1
−i
2
(i
1
+ i
2
+ i
3
+ i
4
)
(2)
and
q =
2
tan(σ)
i
3
−i
4
(i
1
+ i
2
+ i
3
+ i
4
)
, (3)
where i
1
and i
2
are intensity vectors of 0
◦
and 180
◦
tilt angles, and i
3
and i
4
intensity vectors of 90
◦
and
270
◦
tilt angles.
2.2 Reconstruction Methods
In order to obtain surface topography, the surface gra-
dients have to be integrated. However, in practice the
surface gradients contain noise, which can be derived
from imaging and other measurement errors.
If the gradient fields are samples from a larger sur-
face, e.g., textile or paper surface, the boundary con-
ditions of the gradient fields are of high importance.
Traditionally, the boundary conditions are omitted,
since the object to be reconstructed is extracted from
its surroundings, i.e., the edges of images are black,
and the boundary conditions are not then relevant.
In surface reconstruction, one of three different
boundary conditions is usually applied: Dirichlet,
Neumann, or periodic boundary condition. Dirichlet
boundary condition assumes that the height values of
the boundary are known. Usually, this accounts to set-
ting the height values at the boundary to zero. Neu-
mann boundary conditions assume that the directed
derivatives of the boundary are known, usually that
the directed gradients of the boundary can be set to
zero. In periodic boundary conditions, the surface
is assumed to continue periodically, i.e., the surface
continues identically to the other side, similar to the
discrete Fourier transform (Simchony et al., 1990).
Next, particular solutions to the problem of in-
tegrating the measured gradient fields are described
and then subsequently experimentally evaluated in the
next section.
Hansson and Johansson presented a two-light PS
method in (Hansson and Johansson, 2000). The imag-
ing was modeled as
s
′
p
(x,y) = s
p
(x,y) ∗PSF + n(x, y), (4)
where s
p
(x,y) is the directional derivative, n(x,y) is
the noise, and s
p
(x,y) ∗PSF represents convolution
of the signal by a point-spread function. They used a
Wiener filter for the computation of the surface height
from the directed derivatives. Boundary conditions in
Hansson method are twofold: in gradient field esti-
mation of p-gradient, the sum of height values are as-
sumed to be zero, whereas in the integration phase,
the periodic boundary conditions are used as the dis-
crete Fourier transform is used. Thus, it is important
to note that the method is not isotropic.
In Symmetric, similar assumptions for boundary
conditions hold as for Hansson’s method.
VISAPP 2008 - International Conference on Computer Vision Theory and Applications
572

The Frankot-Chellappa (Frankot and Chellappa,
1988) algorithm aims to minimize the least square re-
construction error given by
J(Z) =
Z Z
((Z
x
− p)
2
+ (Z
y
−q)
2
)dxdy, (5)
where Z is the surface to be obtained, {Z
x
,Z
y
}the gra-
dient field of Z, and {p,q} the given non-integrable
gradient field. The gradient field of Z can be written
as {Z
x
,Z
y
}= {p,q}+{ε
x
,ε
y
}, where {ε
x
,ε
y
}denotes
the correction gradient field, which makes the non-
integrable field integrable (Agrawal et al., 2006). In
Frankot-Chellappa method, periodic boundary condi-
tions are applied, and the non-integrable gradient field
is projected on to a set of integrable functions using
the Fourier basis functions.
Simchony presented Poisson solver for surface re-
construction in (Simchony et al., 1990). The approach
is similar to Frankot-Chellappa, such that the norm
of the correction gradient field, Equation 5, is min-
imized. The second partial derivatives in Poisson
equation are approximated using central differencing
method. Simchony (Simchony et al., 1990) develops
two algorithms for Poisson solver: 1) finite difference
calculation in Fourier domain with periodic boundary
conditions, and 2) finite difference calculation in time
domain with Neumann boundary conditions.
2.3 Noise in Photometric Stereo
In practice, the surface gradients contain noise, which
can be derived from imaging and other measurement
errors. So far, the effect of the noise has not been
thoroughly examined. If the imaging conditions are
stable, the noise level of the sensors can be assumed
stationary and measurable, and the information can be
used in order to evaluate the effect of the noise on the
gradient field and the final reconstructed surface.
Next, we propose a derivation for the influence of
imaging noise to the gradient fields. We also propose
a method for the restoration of the degraded gradient
fields for photometric stereo. We make the assump-
tion that additive white noise, N(0,s
2
), affects all the
captured images: zero mean, Gaussian random varia-
tion with variance s
2
is added to images.
Taking into account the noise, Equation 2 can be
written as
p =
2
tan(σ)
(i
1
+ N
1
(0,s
2
)) −(i
2
+ N
2
(0,s
2
))
∑
4
k=1
(i
k
+ N
k
(0,s
2
))
=
2
tan(σ)
i
1
−i
2
+ N(0, 2s
2
)
∑
4
k=1
i
k
+ N(0, 4s
2
)
. (6)
Note that the variance of the sum of two indepen-
dent normal distributions is the sum of the distribution
variances.
In the denominator of Equation 6, the effect of
the noise can be excluded, when
∑
4
k=1
i
k
>> 0 and
N(0,4s
2
) is close to zero, that is, when the noise vari-
ance is small. This assumption is true for many indus-
trial applications where the imaging conditions can be
kept good and stable. Using the assumption and sep-
arating the gradient and the noise, the gradients are
then
p =
2
tan(σ)
i
1
−i
2
(
∑
4
k=1
i
k
)
+ n, (7)
and
q =
2
tan(σ)
i
3
−i
4
(
∑
4
k=1
i
k
)
+ n, (8)
with the noise n for both gradient fields
n =
2
tan(σ)
N(0,2s
2
)
(
∑
4
k=1
i
k
)
. (9)
Making a further assumption that the noise vari-
ance is constant over the whole gradient image, the
power spectrum of the noise field, N(u,v), can be ap-
proximated as a constant field by averaging the sum
of intensity images:
N(u,v) =
2
tan(σ)
2s
2
1
MN
∑
M
x=1
∑
N
y=1
i
a
(x,y)
, (10)
where i
a
=
∑
4
k=1
i
k
, and the image size is M ×N pix-
els. The effect of white noise can be derived in an
identical manner for other combinations of slant and
tilt angles for photometric stereo.
Wiener filtering is an optimal approach for restor-
ing images degraded by noise, when the Signal-to-
Noise Ratio (SNR) and Point Spread Function (PSF)
are correctly known (Gonzalez and Woods, 2002).
We propose that Wiener filtering is applied to gradi-
ent fields to restore them from the noise. In this work,
PSF is omitted, and the Wiener filter is utilized in the
Fourier domain as follows
H =
1
1+ SNR(u,v)
−1
, (11)
where SNR(u, v) = |G(u,v)|
2
/|N(u,v)|
2
is the signal-
to-noise ratio in the frequency domain. G(u,v) is the
power spectrum of the gradient field from photomet-
ric stereo, and N(u,v) the power spectrum of the noise
calculated using Equation 10.
3 EXPERIMENTS
Surface reconstruction methods Hansson, Symmet-
ric, Frankot-Chellappa and Poisson solver are next
evaluated on gradient fields of textured surfaces with
high frequency variation and noise. The focus is to
MEASUREMENT NOISE IN PHOTOMETRIC STEREO BASED SURFACE RECONSTRUCTION
573

evaluate the effect of noise level, denoising, and the
boundary conditions for the integration methods in
surface reconstruction. Evaluation measure is the sur-
face reconstruction error. Two versions of the Poisson
solver are evaluated: 1) finite difference calculation
in time domain with Neumann boundary conditions,
and 2) finite difference calculation in Fourier domain
with periodic boundary conditions, denoted as Pois-
son, and Poissonf, respectively.
The evaluation is performed for both simulated
data and real application data. Gradient fields are cal-
culated using two approaches: 1) analytically for the
simulated surface, and 2) applying photometric stereo
to real image data from paper surface for the applica-
tion. Both surfaces contain high-frequency variation.
3.1 Simulated Data
A chirp-surface contains periodic variation of increas-
ing frequency with time, the amplitude of the surface
remaining constant, as shown in Figure 2. In the ex-
periments, the frequency was increased logarithmi-
cally given the following equation for the surface:
S = sin(
2π f
0
ln(k)
(k
t
−1)), (12)
where f
0
is the frequency at t = 0, k is the rate of ex-
ponential increase in frequency, and t is the time. For
a surface S(x,y), time t was defined as t = 0.5(x+ y).
The chirp-function was differentiated analytically in
order to obtain the horizontal and vertical gradient
fields, which were degraded with two levels of addi-
tive white noise: 5 dB, and 10 dB. The noise levels
were then calculated using Equation 10 from the stan-
dard deviation of gradient fields. The degraded gra-
dient fields were Wiener filtered, and surfaces were
integrated from the degraded and Wiener filtered gra-
dient fields.
Figure 2 and Table 1 present the reconstructed sur-
faces and the reconstruction errors. In Fig. 2, the ef-
fects of boundary conditions are visible. For meth-
ods applying periodic boundary conditions , that is,
Hansson, Symmetric, Frankot-Chellappa, and Pois-
sonf, periodic reconstruction errors near the surface
boundaries are clearly observable. Poisson solver
with Neumann boundary conditions reconstructs the
boundaries without any noticeable error. For Hans-
son, also the horizontal surface reconstruction error
deteriorates the reconstruction result.
The reconstruction errors in Table 1 were cal-
culated from surfaces, which were scaled to the
zero mean and unit variance. Reconstruction er-
rors between the chirp-surface, and reconstructed sur-
faces were analyzed with two measures: 1) root
Chirp-surface Hansson
Symmetric Frankot-Chellappa
Poisson Poissonf
Figure 2: The original surface and the reconstructed sur-
faces from Wiener filtered gradient fields of simulation data.
mean square error (RMSE), and 2) high-pass RMSE
(HPRMSE). The HPRMSE were calculated from sur-
faces, which were high-pass filtered with cutoff at
10% of resolution. RMSE emphasizes reconstruc-
tion errors on longer wavelengths, HPRMSE errors
on higher frequencies.
In RMSE error, the Poisson solver with Neumann
boundary conditions outperforms clearly the other
methods in all the noise levels. At the same time,
the reconstruction error in longer wavelengths for
Hansson is significantly larger. Wiener filtering has
only a minor effect to RMSE in surface reconstruc-
tion. In high-frequency reconstruction error, Poissonf
outperforms the other methods, with Symmetric and
Frankot-Chellappa being comparably close. Hansson
and Poisson exhibit remarkably larger reconstruction
errors in HPRMSE. It is also evident that the proposed
Wiener filtering enhances the reconstruction results in
high frequencies on 5 dB noise level.
VISAPP 2008 - International Conference on Computer Vision Theory and Applications
574

Table 1: Surface reconstruction errors: RMSE, and High-pass RMSE calculated from 10% high pass filtered surfaces. S, S
n
,
and S
wnr
are the reconstruction errors from original gradient fields, noise added gradient fields, and Wiener filtered gradient
fields at 10 dB and 5dB noise levels. The smallest reconstruction errors for each experiment are bolded.
RMSE HPRMSE
S S
n
S
wnr
S
n
S
wnr
S S
n
S
wnr
S
n
S
wnr
Simulated data 10dB 10dB 5dB 5dB 10dB 10dB 5dB 5dB
Gradient field 0.140 0.140 0.417 0.409 0.134 0.134 0.395 0.388
Hansson 0.417 0.513 0.513 0.861 0.852 0.136 0.292 0.292 0.622 0.615
Symmetric 0.252 0.260 0.260 0.306 0.306 0.022 0.030 0.030 0.070 0.068
Frankot-Chellappa 0.239 0.246 0.246 0.288 0.288 0.020 0.027 0.027 0.061 0.060
Poisson 0.161 0.168 0.168 0.223 0.223 0.095 0.097 0.097 0.119 0.118
Poissonf 0.239 0.245 0.245 0.286 0.285 0.013 0.019 0.019 0.048 0.047
Application data
Gradient field 0.003 0.003 0.010 0.008 0.003 0.003 0.009 0.007
Hansson 0.035 0.035 0.108 0.105 0.023 0.023 0.073 0.069
Symmetric 0.034 0.034 0.106 0.101 0.028 0.028 0.089 0.082
Frankot-Chellappa 0.013 0.013 0.043 0.042 0.006 0.006 0.020 0.018
Poisson 0.014 0.014 0.044 0.042 0.007 0.007 0.022 0.019
Poissonf 0.013 0.013 0.041 0.040 0.005 0.005 0.016 0.015
3.2 Application Example
In the second experiment, paper surfaces were recon-
structed from gradient fields calculated using photo-
metric stereo. The OTF and SNR functions for Hans-
son, and Symmetric methods were as Hansson pro-
posed in (Hansson and Johansson, 2000). The paper
surface images for photometric stereo were acquired
using a CCD camera with a resolution of 2048 x 2048
pixels with 12 bits per pixel. In the experiments, the
image area was 15 mm x 15 mm.
Because no ground truth of the surface topology
was available, the influence of white noise in surface
reconstruction was studied using artificial noise, in
a similar fashion to the simulated data. The recon-
struction results from the two noise degraded gradient
fields were contrasted to original reconstructed sur-
faces without added noise. Similar to previous ex-
periment, Gaussian white noise was added to images,
and the gradient fields were calculated from the de-
graded images by photometric stereo. Standard de-
viations of gray-scale values of images were applied
in determination of the 5 dB and 10 dB noise levels.
In the gradient field restoration with Wiener filter, the
white noise field was determined using Equation 10.
The reconstruction results are presented in Fig. 3,
and Table 1. In Fig. 3, the smoothing effect of
minimization based Frankot-Chellappa and Poisson
solvers can be observed. Hansson and Symmetrical
sharpen the reconstruction result by Wiener filtering,
which includes SNR and a PSF optimized for paper
surfaces. The reconstruction errors due to noise are
remarkably smaller for minimization based methods,
than for Hansson and Symmetric, see Table 1. Wiener
filtering of gradient fields can be seen to enhance the
reconstruction results in 5 dB noise level. The effects
of boundary conditions are not clearly visible on this
application data.
Hansson Symmetric
Frankot-Chellappa Poisson
Poissonf
Figure 3: A fragment of reconstructed paper surfaces from
Wiener filtered gradient fields.
For all the methods, the reconstruction errors in-
creased with the noise level with simulated and real
application data. In 10 dB noise level, the reconstruc-
tion results did not differ remarkably from the noise-
less surfaces. Wiener filtering was able to denoise
the gradient fields at 5 dB noise level. In RMS er-
ror, the Wiener filtering could not provide significant
improvement in reconstruction result. However, the
MEASUREMENT NOISE IN PHOTOMETRIC STEREO BASED SURFACE RECONSTRUCTION
575

high-pass reconstruction error was decreased 1%–2%
with simulated data, and 5%–11% with real applica-
tion data depending on the reconstruction method. In
general, Wiener filtering improved the reconstruction
result for all the methods and data sets.
4 CONCLUSIONS
In this paper, surface reconstruction techniques were
studied in the context of surfaces with high frequency
variation. The effect of imaging noise to gradient
fields was also evaluated and a denoising approach
was proposed.
The experiments demonstrated that the Wiener fil-
tering based denoising of the gradient fields is useful
and applicable, if the power spectrum of the noise
is known. Minimization based surface reconstruc-
tion techniques, such as Poisson solver and Frankot-
Chellappa, are more robust against the noise with sim-
ulated and real application data compared to Hans-
son and Symmetric, which perform the integration
in Fourier domain with Wiener filtering. Hansson’s
method applies only one gradient field and was found
to be more sensitive to correct Wiener filter parame-
ters than Symmetric using two gradient fields. Pois-
son solvers with Neumann and periodic boundary
conditions provided the best results in total and high
frequency scale reconstruction, respectively. Neu-
mann boundary conditions provided correct surface
boundaries, whereas periodic boundary conditions
deteriorated the surface boundary with a non-periodic
pattern.
High frequency surfaces reconstructed using min-
imization based methods are is still not adequate for
some real applications, as they smoothen the surface
making, for example, roughness measurements in-
valid. Frankot-Chellappa and Poisson solvers provide
a robust and parameter free surface reconstruction in
many cases, but the methods have to developed fur-
ther for high frequencycontaining data. Recent devel-
opments in surface reconstruction, such as α surfaces,
M-estimators, regularization and diffusion, seem not
to provide improvement for these problems, as their
main effect is additional adaptive smoothing of the
surface, as noted in (Agrawal et al., 2006). Thus,
one problem in the reconstruction of high frequency
surfaces is in the definition of the function to be min-
imized. In future work, minimization constraints for
the surface reconstruction methodsneed to be studied.
Another approach would be to model the imaging sys-
tem more accurately, for example, using Wiener filter-
ing with SNR and PSF. In this work, only the effect of
the SNR was evaluated, and in future works also the
PSF has to be studied.
ACKNOWLEDGEMENTS
The authors gratefully appreciate the provided fund-
ing from European Regional Development Fund
(ERDF), Finnish Funding Agency for Technology
and Innovation (TEKES), Stora Enso, UPM, Metso,
and Future Printing Center (FPC).
REFERENCES
Agrawal, A., Raskar, R., and Chellappa, R. (2006). What
is the range of surface reconstructions from a gradient
field? In Proc. ECCV, pages 578–591.
Drbohlav, O. and Chantler, M. (2005). On optimal light
configurations in photometric stereo. In Proc. IEEE
ICCV.
Frankot, R. and Chellappa, R. (1988). A method for enforc-
ing integrability in shape from shading algorithms.
IEEE PAMI, 10(4):435–446.
Gonzalez, R. and Woods, R. (2002). Digital Image Process-
ing. Prentice Hall.
Hansson, P. and Johansson, P.-A. (2000). Topography and
reflectance analysis of paper surfaces using a photo-
metric stereo method. Opt. Eng., 39(9):2555–2561.
Horn, B. (1990). Height and gradient from shading. Int. J.
Comput. Vision, 5(1):37–75.
Karacali, B. and Snyder, W. (2004). Noise reduction in sur-
face reconstruction from a given gradient field. Inter-
national Journal of Computer Vision, 60:25–44.
Kuparinen, T., Kyrki, V., Mielikainen, J., and Toivanen,
P. (2007). On surface reconstruction from gradient
fields. In Proc. of IEEE ICIP, pages II 545–548.
Lambert, J. (2001). Photometry, or, on the measure and gra-
dations of light, colors, and shade: translation from
the Latin of photometria, sive, de mensura et gradibus
luminis, colorum et umbrae. Illuminating Engineering
Society of North America.
McGunnigle, G. and Chantler, M. (2003). Rough surface
description using photometric stereo. Measurement
Science and Technology, 14:699–709.
Noakes, L. and Kozera, R. (2003). Nonlinearities and noise
reduction in 3-source photometric stereo. Journal of
Mathematical Imaging and Vision, 18:119–127.
Simchony, T., Chellappa, R., and Shao, M. (1990). Direct
analytical methods for solving Poisson equations in
computer vision problems. IEEE PAMI, 12(5):435–
446.
Wei, T. and Klette, R. (2003). Depth recovery from noisy
gradient vector fields using regularization. In Proc. of
CAIP, pages 116–123.
Woodham, R. (1978). Photometric stereo. MIT A.I. Labo-
ratory Memo. No. 479.
VISAPP 2008 - International Conference on Computer Vision Theory and Applications
576