Light Field Scattering in Participating Media

Takuya Mokutani, Fumihiko Sakaue and Jun Sato

Department of Computer Science, Nagoya Institute of Technology, Gokiso Showa, Nagoya, Japan

Keywords:

Participating Media, Light Scattering Model, Light Field Scattering.

Abstract:

In this paper, we propose a representation of the light scattering in participating media, which can represent

all order light scattering simply. To achieve the model, we focus on the light ﬁeld in the participating media,

and it is shown that the convolution of the light ﬁeld can describe the attenuation of the light rays and scat-

tering of them. By analyzing the convolution kernel, we derive a simple kernel that represents all order light

scattering. Also, we introduce the estimation method of the characteristics of the participating media based on

our proposed model. Several experimental results show that our proposed model can describe light scattering

more appropriately than existing models.

1 INTRODUCTION

In recent years, image sensors such as cameras are

one of the most important devices to obtain scene in-

formation, and they ordinary utilized for various ap-

plications, e.g., 3D measurement, object recognition,

and so on. In ordinary cases, we assume that the cam-

era obtains ‘clear’ information from the scene. How-

ever, images taken in the outdoor scenes are often dis-

turbed by participating media such as fogs, smokes,

and so on. Figure 1 shows example images taken

in the participating media. The ﬁgure shows that the

fogs and smokes disturb appropriate imaging in par-

ticipating media. Furthermore, water also disturbs

imaging if we want to take underwater images. The

effect of the participating media disturbs various ap-

plications based on the camera images such as au-

tonomous driving and driving assistant systems. In

addition, 3D sensors, e.g., LiDAR, also cannot obtain

an accurate distance in participating media because

several sensors in the system also cannot get the ap-

propriate data as same as the cameras.

In order to solve the problem, various kinds of

methods are studied(Narasimhan et al., 2006; He

et al., 2011; Kitano et al., 2017; Naik et al., 2015;

Figure 1: Example images taken in participating media,

fogs, smokes, and water.

Satat et al., 2018). To solve the problem essentially,

we need to use the accurate light scattering model to

describe the phenomena in the participating media.

In the ﬁeld of computer graphics, several models are

used to describe the light behavior accurately(Pharr

et al., 2016). In tradition, light behavior is classi-

ﬁed into single scattering and multiple scattering, and

several models are proposed for each scattering. Al-

though the models can describe the light scattering

in the participating media in limited case, there are

lots of situations which cannot be explained by these

models. In order to describe the light scattering ap-

propriately, light ray tracing techniques and the pho-

ton mapping techniques are utilized. Although they

provide more realistic results, they are not suitable for

analyzing images because they require a high compu-

tational cost. In this research, we propose a light scat-

tering description model that can explain the single

scattering as well as multiple scattering in low com-

putational cost. In addition, we also show the partici-

pating media analysis method based on the proposed

model. In this light ray explanation, we focus on the

5D light ﬁeld in the scattering media, and we show

that light scattering can be described efﬁciently and

effectively using the light ﬁeld.

544

Mokutani, T., Sakaue, F. and Sato, J.

Light Field Scattering in Participating Media.

DOI: 10.5220/0009180505440549

In Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2020) - Volume 5: VISAPP, pages

544-549

ISBN: 978-989-758-402-2; ISSN: 2184-4321

Copyright

c

2022 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved

2 LIGHT SCATTERING IN

PARTICIPATING MEDIA

2.1 Participating Media and Light

Transport Equation

We ﬁrst explain the characteristics of participating

media(Mukaigawa et al., 2010). The participating

media includes lots of micro-particles, and the par-

ticles affect input light rays. For example, fog and

smoke are representatives of the participating media,

and they consist of particles such as water and dust.

The particles scatter light rays, and thus, observed im-

ages in the media become unclear, as shown in Fig.1.

The effect of the media on the light rays are clas-

siﬁed into light attenuation, in-scattering, and out-

scattering. These effects are described by light trans-

port equations (LTE). Let us consider the case when

a light ray L(x, ω) passes through the media at a point

x directed to ω. Under the assumption that dL(x,ω)

denote the change of the light ray, the effect of the

participating media is described as follows:

L

0

(x,ω) = L(x, ω) + dL(x,ω) (1)

where L

0

is an affected light ray by the media.

The dL(x,ω) consists in attenuation term, in-

scattering term and out-scattering term. First, the at-

tenuation describes light absorbing by the media, and

it is described as follows:

dL(x,ω) = −σ

a

(x)L(x,ω)ds (2)

where σ

a

is absorption coefﬁcient and ds shows thick-

ness of the media. Second, several light rays contact

the particles and reﬂected in directions other than ω.

Therefore, light rays directed to ω is attenuated, and

it is described as follows:

dL(x,ω) = −σ

s

(x)L(x,ω)ds (3)

where σ

s

denotes a scattering coefﬁcient at x. Fi-

nally, scattered light rays from any directions other

than ω are added to light rays as out-scattering. The

out-scattering is described as follows:

dL(x,ω) = σ

s

(x)

Z

Ω

Lp(x,ω

0

,ω)L(x,ω

0

)dω

0

ds (4)

where p denote phase function, and it describes the

probability that the light ray from ω

0

is reﬂected di-

rection ω.

Several functions are proposed as phase function

p. In this paper, we utilize the Henyei-Greenstein

phase function deﬁned as follows:

p(θ) =

1

4π

1 − g

2

(1 + g

2

− 2g cos θ)

2/3

(5)

Figure 2: Single scattering in the participating media.

where θ = arccos(ω, ω

0

), and g (−1 ≤ g ≤ 1) de-

termine directional characteristics of the scattering.

By these three equations, Eq.(1) is rewritten as fol-

lows:

L

0

(x,ω) = L(x, ω) − σ

a

(x)L(x,ω)ds

−σ

s

(x)L(x,ω)ds

+σ

s

R

Ω

p(x,ω

0

,ω)L(x,ω

0

)dω

0

ds

(6)

By solving the LTE, the status of the light ray in scat-

tering media can be described. However, the deriva-

tion of an analytical solution to the equation is dif-

ﬁcult because of the out-scattering term. Therefore,

several algorithms based on numerical analysis are

utilized in the ﬁeld of computer graphics(Pharr et al.,

2016). However, they require lots of computational

costs, and it is not easy to utilize them for analysis of

the media.

2.2 Single Scattering Model

In order to relax the computational complexity, the

approximated model is utilized generally. Under the

assumption that the density of the micro-particles is

low, a single scattering model is used for describing

the light scattering. In this case, we assume that the

light rays are scattered (reﬂected) just one time by

micro-particles in the media. Under this assumption,

routes of the light rays are constrained completely by

an input light ray L(x

i

,ω

i

) and an output light ray

L(x

o

,ω

o

) as shown in Fig.2.

In this case, L

0

(x

o

,ω

o

) is described as follows:

L

0

(x

o

,ω

o

) = σ

s

p(θ)e

−(σ

a

+σ

s

)(d

1

+d

2

)

L(x

i

,ω

i

) (7)

where d

1

and d

2

denote distance from x

i

to s and s to

x

o

, respectively. The θ denotes an angle between ω

i

and ω

o

.

The computation of the single scattering model is

straightforward. In addition, this model is approxi-

mately valid for shallow participating media. There-

fore, the model is often utilized for analysis of the

scattering media.

2.3 Multiple Scattering Model

We next consider the case when the density of the

media is very high. In this case, multiple scatter-

Light Field Scattering in Participating Media

545

ing model is utilized for describing the light scatter-

ing. In this model, it is assumed that the light rays

are scattered repeatedly, and then, the directionality

of the light ray is lost. Therefore, light rays into the

dense participating media are distributed in any direc-

tion evenly. By using this model, light scattering by

dense media such as milk is appropriately described.

2.4 Low-order Scattering

By using the two models, light scattering in the

participating media is described. However, general

scenes often include more complicated scattering.

Mukaigawa et al.(Mukaigawa et al., 2010) indicate

light rays input to the participating media scattered

a few times, and then most scenes include not sin-

gle scattering but low-order scattering. Although the

method which separates each order scattering is pro-

posed by them, the method requires several numbers

of images because the method uses high-frequent pat-

tern projection(Nayar et al., 2006) to separate the light

rays. Therefore, it is difﬁcult to apply the method for

ordinary image analysis.

3 LIGHT FIELD SCATTERING

MODEL

3.1 Light Field Scattering

In order to describe the light scattering not only single

and multiple scattering but also any order scattering in

the participating media, we focus on the light ﬁeld in

the media. As described in Eq.(1), LTE describes the

relationship between input light rays and output rays.

Therefore, we describe the light scattering by light ray

transition in the light ﬁeld. By using this proposed de-

scription, light scattering can be explained efﬁciently.

In addition, this description achieves describing sin-

gle scattering, low-order scattering, and multiple scat-

tering in the same manner.

In this manner, we separately consider input light

rays to a point and output light rays from the point. By

combining the input light rays and output light rays,

we achieve a description of the light scattering efﬁ-

ciently.

3.2 Input Light Rays

We ﬁrst consider input light rays to a point in the

scene. Let L(x,ω) denote a light ray at a point x

directed to ω. Under the assumption that light rays

go straight in scattering media other than reﬂection

by particles, integrated input light ray L

0

(x,ω) to the

point x directed to ω are computed as follows:

L

0

(x,ω) =

Z

d

e

σ

t

d

L(x + dω, ω)dd (8)

Let g(d) denote a function which describe light atten-

uation and it is deﬁned as follows:

g(d) =

e

σ

t

d if d ≥ 0

0 otherwise

(9)

By using the g, Eq.(8) is rewritten as follows:

L

0

(x,ω) =

Z

d

g(d)L(x + dω,ω)dd (10)

This is convolution of L(x,ω) by a convolution kernel

g. Therefore, integration of the input light ray can be

computed by just convolution of the light ﬁeld.

3.3 Output Light Rays

We next consider output light rays scattered by parti-

cles. The integrated input light rays L

0

are scattered

by the particles based on a phase function p. In this

scattering, we focus on an output light ray L

00

(x,ω

0

)

directed to ω

0

. Under the assumption that the phase

function is isotropic, the scattering is computed as fol-

lows:

L

00

(x,ω

0

) = σ

s

Z

ω

p(ω,ω

0

)L

0

(x,ω + ω

0

)dω (11)

In this case, the phase function p can be regarded as a

convolution kernel. Therefore, the light scattering by

the particles is also described by just convolution.

Note that ω +ω

0

in Eq.(11) is not correct since this

convolution is done on a sphere. The effective convo-

lution and fourier transfomation on the sphere can be

done by using shpherical harmocis(Su and Grauman,

2017). Therefore, we simply describe this convolu-

tion by just ω + ω

0

for convinience.

3.4 N-th Order Scattering Description

As described in the previous sections, integration of

input light rays and scattering of the input light rays

are described by just convolution. By integrating both

of convolutions, light scattering in participating media

is described as follows:

L

0

(x,ω) = σ

s

g ∗ s ∗ L(x,ω) = σ

s

k ∗ L(x,ω) (12)

where

0

∗

0

denotes convolution and k = g ∗ s is the in-

tegrated kernel.

By convolution σ

s

k, the input light ﬁeld L is up-

dated to the ﬁrst-order scattering light L

0

. Since the

scattered light is also scattered by media, again and

VISAPP 2020 - 15th International Conference on Computer Vision Theory and Applications

546

Figure 3: 2D image as a slice of the light ﬁeld.

again, N-th order scattering light L

N

is described as

follows:

L

N

(x,ω) = σ

s

k ∗ L

N−1

(x,ω) (13)

where L

0

= L and L

0

= L

1

. This equation indicates

that σ

s

L

N

is scattered by the media, and the rest (1 −

σ

s

)L

N

is observed directly. Therefore, the observed

light ﬁeld L

a

, including all scattering component, is

described as follows:

L

a

= (1 − σ

s

)

∞

∑

j=0

L

j

(14)

In this equation, we derive N-th order light scattering

by image convolution.

Note that the image convolution can be described

by just element multiplication in the frequency do-

main. In addition, the techniques of the fast Fourier

transformation require low computational cost for im-

age convolution. Therefore, the light scattering repre-

sentation based on image convolution can be achieved

by just low computational cost.

4 PARTICIPATING MEDIA

CHARACTERISTICS

ESTIMATION

In this section, we describe an estimation method for

the participating media characteristics based on a light

scattering model described in the previous section.

Under the assumption that the participating media is

uniform, an attenuation coefﬁcient σ

a

, symmetry fac-

tor g in a phase function, and a scattering coefﬁcient

σ

s

determine the characteristics in this model. There-

fore, we need to estimate the three parameters.

In ordinary cases, we cannot obtain the status of

the light ﬁeld directly because conventional sensors

cannot observe a 5D light ﬁeld. Therefore, we need

to estimate the parameters from a 2D image, which is

a slice of the5D light ﬁeld, as shown in Fig.3.

Let an image I(x,y) denote a slice of the light ﬁeld

L by a direction ω

o

to a camera and a plane in the

scene. Under the assumption that input light ray L

i

is

known, we deﬁne an evaluation function E as follows:

E = kI(x, y) − S (k(σ

t

,σ

s

,g)L

i

)k

2

(15)

Figure 4: Experimental environment.

where k(σ

t

,σ

s

,g) is a convolution kernel determined

by the parameters, and S denotes slicing of the light

ﬁeld correspond to the input image. Values that min-

imize the E are suitable parameters to describe the

characteristics of the participating media.

5 EXPERIMENTAL RESULTS

5.1 Environment

In this section, we show several experimental results

by our proposed method. In this experiment, we com-

bined a few amounts of white ink and water. The com-

bined water was used as the participating media. The

white water ﬁlled a water tank, and the tank was ob-

served from the upper side, as shown in Fig.4. In this

environment, light rays were input to the tank by a

projector. The relative position between a camera and

a projector was calibrated beforehand, and then the

input light ray is controlled and known. An Example

of the input image is shown in Fig.5. In order to help

the understanding, the grayscale images are colored,

as shown in the right image. In these images, densities

of the white ink were changed to survey our proposed

model can describe the different density participating

media. Not only our proposed model but also a single

scattering model was used for the estimation of the

parameters for comparison.

5.2 Results

Figure 6 shows input images and estimated results.

In this results, result images were synthesized from

input light rays and estimated parameters by our

proposed method. For comparison, images including

1st order scattering (single scattering) and until

several order scatterings are shown, respectively.

RMSE between an input image and a reconstructed

image is shown below of each image.

Light Field Scattering in Participating Media

547

Figure 5: Examples of observed images. The left images

show the original grayscale images, and the right image

shows the colored images. In the upper image, the density

of the participating media is high, and it is low in the lower

images.

(a) input image

(b) 1st order scattering

(RMSE = 7.73)

(c) 3rd order scattering

(RMSE = 6.29)

(d) 20th order scattering

(RMSE = 5.68)

Figure 6: Reconstructed images and input images when the

density of the participating media is low. Figure(a) shows

an input image and (b)∼ (d) show images including until

1st order, 3rd order and 20th order scatterings.

In this result, RMSE between the input image and

the reconstructed image becomes smaller according

to scattering order increasing. The result indicates

that the input image includes not only lower light scat-

tering but also higher-order scattering even if the den-

sity of the media is not so high. Our proposed method

can describe the various order scatterings, and thus

RMSE becomes lower when the scattering order be-

comes higher.

Figure 7 shows estimated results of the light scat-

tering in denser participating media. In these re-

sults, RMSE becomes In these results, RMSE be-

comes lower according to the scattering order as sim-

(a) input image

(b) 1st order scattering

(RMSE = 8.13)

(c) 3rd order scattering

(RMSE = 4.10)

(d) 20th order scattering

(RMSE = 2.70)

Figure 7: Reconstructed images and input images when the

density of the participating media is high. Figure(a) shows

an input image and (b)∼ (d) show images including until

1st order, 3rd order and 20th order scattering.

ilar to the previous results. The results indicate that

our proposed method can describe not only shallow

participating media, but also the dense media by us-

ing the same manner.

6 CONCLUSION

In this paper, we propose a light scattering model that

can describe not only low-order scattering but also

high-order scattering in participating media. In this

model, we describe light attenuation and light scat-

tering by the convolution of the light ﬁeld. By using

the kernel, light scattering in the participating media

can be described accurately and efﬁciently. Based on

the convolution description, we introduce the estima-

tion method of characteristics of participating media.

Several experimental results show that the proposed

method can describe the light scattering in the par-

ticipating media. Besides, our proposed method es-

timate the characteristics of the participating media.

We will consider a more stable estimation method and

also consider the case when the participating media is

not uniform in future work.

REFERENCES

He, K., Sun, J., and Tang, X. (2011). Single image haze

removal using dark channel prior. IEEE Trans. Pattern

VISAPP 2020 - 15th International Conference on Computer Vision Theory and Applications

548

Anal. Mach. Intell., 33(12):2341–2353.

Kitano, K., Okamoto, T., Tanaka, K., Aoto, T., Kubo, H.,

Funatomi, T., and Mukaigawa, Y. (2017). Recover-

ing temporal psf using tof camera with delayed light

emission. IPSJ Transactions on Computer Vision and

Applications, 9(15).

Mukaigawa, Y., Yagi, Y., and Raskar, R. (2010). Anal-

ysis of light transport in scattering media. In

Proc.CVPR2010.

Naik, N., Kadambi, A., Rhemann, C., Izadi, S., Raskar,

R., and Kang, S. B. (2015). A light transport model

for mitigating multipath interference in time-of-ﬂight

sensors. In Proc. CVPR2016, pages 73–81.

Narasimhan, S. G., Gupta, M., Donner, C., Ramamoorthi,

R., Nayar, S. K., and Jensen, H. W. (2006). Acquiring

scattering properties of participating media by dilu-

tion. In ACM Transactions on Graphics (TOG), vol-

ume 25, pages 1003–1012. ACM.

Nayar, S. K., Krishnan, G., Grossberg, M. D., and Raskar,

R. (2006). Fast separation of direct and global com-

ponents of a scene using high frequency illumination.

SIGGRAPH.

Pharr, M., Jakob, W., and Humphreys, G. (2016). Phys-

ically Based Rendering: From Theory to Implemen-

tation. Morgan Kaufmann Publishers Inc., San Fran-

cisco, CA, USA, 3rd edition.

Satat, G., Tancik, M., and Raskar, R. (2018). Towards

photography through realistic fog. 2018 IEEE Inter-

national Conference on Computational Photography

(ICCP), pages 1–10.

Su, Y.-C. and Grauman, K. (2017). Learning spherical con-

volution for fast features from 360 imagery. In Ad-

vances in Neural Information Processing Systems 30,

pages 529–539.

Light Field Scattering in Participating Media

549