A Real–time Global Illumination Approach for High Resolution

Reﬂective Shadow Maps in Open World Scenes

Daniel Bischoff, Tobias Schwandt and Wolfgang Broll

Ilmenau University of Technology, Virtual Worlds and Digital Games Group, Ehrenbergstraße 29, Ilmenau, Germany

Keywords:

Global Illumination, Indirect Lighting, Real–time Rendering, Bidirectional Reﬂective Shadow Maps, Open

World.

Abstract:

The complexity of ofﬂine and real–time rendering of global illumination effects is a vast ﬁeld of research.

Covering the complexity of dynamic open–world environments with changing light conditions and moving

objects is challenging in real–time rendering. Especially the quality of light properties needs to be physically

correct and sufﬁciently fast even if the light and environment conditions are changing. In this paper, we

present a fast global illumination approach suitable to achieve indirect lighting using high–resolution reﬂective

shadow maps in real time. Based on LightSkin, we present an enhancement applying bidirectional shadow

maps to improve the quality of shadows and global illumination effects in open–world environments. A

novel combination of accumulating virtual area lights from the reﬂective shadow map with an indirect light

representation using LightSkin’s proxy light sources shows a signiﬁcant improvement. In an urban scene

typical for open–world environments, the proposed approach is up to approx. 12 times faster than the original

LightSkin approach. This makes our novel approach suitable for high–quality indoor as well as outdoor global

illumination.

1 INTRODUCTION

Indirect illumination contributes signﬁcantly to the

overall quality of rendered images - in particular as

it increases their plausiblity. However, due to its rat-

her high rending costs, it is generally difﬁcult to pro-

vide sophisticated results in real-time. In particular

the speciﬁc requirements for providing indirect illu-

mination for open-world scenes in real-time were wi-

dely neglected by previous research.

Particularly light types like the sun usually have

a shadow map size of 1024 × 1024px or higher.

This high resolution is difﬁcult to render in case of

Reﬂective Shadow Maps (RSMs) and Global Illu-

mination (GI), since the na

ıve approach is to light

each frame buffer pixel by each virtual light de-

ducted from the RSM. Consequently this leads to a

high computational complexity. An option is to pre–

compute the lighting reducing the per frame com-

plexity. However, dynamically changing objects or

light conditions in the environment cannot be suppor-

ted. Modern applications—like open–world compu-

ter games—consist of many integrated objects that are

affected by environment properties like weather con-

ditions, the position of the sun, or even clouds. The-

refore, an on–the–ﬂy rendering of dynamically chan-

ging open–world environments with GI needs to be

addressed to enhance the visual output in terms of re-

alism and physical correctness.

Similar to (Ritschel et al., 2011), we employ a

reduced representation of the RSM for accumula-

ting indirect light. This paper combines sampling

the RSM using Bidirectional Reﬂective Shadow Maps

(BRSMs) introduced in (Ritschel et al., 2011) with the

coarse model representation introduced along with

the LightSkin GI approach (Lensing, 2014; Lensing

and Broll, 2013b). The indirect light is accumula-

ted for a set of sparsely distributed light caches rat-

her than all surface points and subsequently interpo-

lated for the remaining surface points. A reduction of

the amount of virtual lights is achieved by an impor-

tance sampling strategy. This allows us to use high–

resolution RSMs in complex scenes while maintai-

ning real–time frame rates. Apart from the contribu-

tion to open–world scenarios, our approach is appli-

cable for GI in general.

This paper is structured as follows: in Section 2

we investigate related work and indicate where our

novel contributions differ from previous approaches.

Section 3 presents our approach for sampling large

116

Bischoff D., Schwandt T. and Broll W.

A Realâ

A ¸Stime Global Illumination Approach for High Resolution Reﬂective Shadow Maps in Open World Scenes.

DOI: 10.5220/0006135601160126

In Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2017), pages 116-126

ISBN: 978-989-758-224-0

RSMs more efﬁciently and dynamically. In Section 4,

we provide details on our approach combining the

principles of LightSkin and BRSMs while Section 5

& Section 6 present an evaluation of the results we

achieved along with the limitations that are still in

place. Finally, Section 7 concludes our ﬁndings and

gives a prospect on future work.

2 RELATED WORK

Indirect Lighting

Global illumination is a very comprehensive ﬁeld of

research. As early as 1997, Alexander Keller (Keller,

1997) described the ﬁrst rendering technique to com-

pute global illumination in textured scenes. Based on

quasi–random walk and a Monte Carlo integration he

achieves physically correct results. This algorithm

has a complexity of O(NK) where N is the number

of rays and K the number of elements in the scene.

However, ray–based approaches or radiosity are not

suitable for real–time graphic applications because of

the requirements, especially with open–world scena-

rios in mind.

Dachsbacher and Stamminger presented an appro-

ach called Reﬂective Shadow Maps in 2005 (Dachs-

bacher and Stamminger, 2005). It extends shadow

maps by additional buffers that hold world position,

normal values, and the color of the surfaces being lit

by the light source. The additional buffers are used in

combination with a screen–space technique to achieve

interactive frame rates even for complex scenes. This

approach provides the basis of many of the following

rendering techniques.

Ritschel et al. presented Imperfect Shadow Maps

(ISMs) (Ritschel et al., 2008) which create low–

resolution shadow maps rendered around the current

camera based on Virtual Point Lights (VPLs). A ﬁ-

nally generated shadow map atlas is used for indi-

rect illumination in fully dynamic scenes. Using point

sampling shows to be sufﬁcient in most cases, but has

some disadvantages in terms of accuracy.

A study by Kaplanyan & Dachsbacher demon-

strated the possibilities of using lattices and Spheri-

cal Harmonics (SH) based on RSMs (Kaplanyan and

Dachsbacher, 2010). Overall, a computation time

of a few milliseconds can be reached without requi-

ring pre–computation. Based on a grid–based lighting

they achieve single–bounce indirect illumination with

occlusion. The Light Propagation Volumes (LPV)

lookup inside the cascaded grids is the most expen-

sive part of the approach.

Crassin et al. have presented the Voxel Cone Tra-

cing (VCT) method (Crassin et al., 2011). It is ba-

sed on a pre–ﬁltered voxel representation of the scene.

They use a voxel–based cone that corresponds to the

individual mipmap levels of the pre–integrated scene.

Therefore, only a small number of sampling rays is

required as they can be approximated via the cone.

Lensing & Broll addressed global illumination in

terms of diffuse and glossy surfaces by a new ap-

proach called LightSkin (Lensing and Broll, 2012;

Lensing and Broll, 2013b; Lensing and Broll, 2013a;

Lensing, 2014). It is based on ideas from irradi-

ance caching (Ward et al., 1988) and radiance caching

riv

anek et al., 2005) as it interpolates the indirect

light from so called light caches. These caches are

distributed in model space during a per–model pre–

computation step which can be parametrized accor-

ding to the desired density.

With a RSM from the light source, indirect lights

are generated as Virtual Area Lights (VALs). The ge-

nerated caches are used to compute diffuse and glossy

incident lighting by accumulating the indirect light

from the RSM into two proxy light sources for each

cache, one representing diffuse light and the other re-

presenting glossy light. Moreover, the caches’ proxy

lights are attenuated to account for occlusion by a

double–projection method that approximates occlu-

sion computation efﬁciently. The ﬁnal output is gene-

rated by interpolating the accumulated lighting from

the proxy lights to the surface points in the vicinity of

a cache.

However, for a single surface point, the proxy lig-

hts of several surrounding caches are used and weigh-

ted according to their contribution. The approach de-

livers plausible results with reasonable performance

and is applicable to virtual and mixed reality scena-

rios when RSM sizes are small compared to com-

monly used shadow map resolutions, e.g. for 128

or 256

px.

Our analysis of the approach shows that high–

resolution reﬂective shadow maps result in signiﬁ-

cantly lower frame rates, since all of the pixels in the

RSM are used as virtual lights for cache lighting.

A new radiance caching approach from Vardis,

Papaioannou & Gkaravelis combines a radiance ﬁeld

chrominance with an optimized cache population

scheme (Vardis et al., 2014). The main idea is to ge-

nerate caches only on surface that ﬁnally contribute to

the result.

Laurent et al. use a stochastic decimation process

chained with a partitioning strategy (Laurent et al.,

2016). For every triangle inside the scene, a proper

VPL is considered. The approach doesn’t need any

A Realâ

A¸Stime Global Illumination Approach for High Resolution Reﬂective Shadow Maps in Open World Scenes

117

pre–computation and can be used for dynamic scenes

as well.

Importance Sampling

Of the many different approaches associated with the

use of importance sampling, we would like to mention

Monte-Carlo Simulation, pseudo-random, and Perlin

noise textures (Anderson, 1999; Green, 2005). They

allow an efﬁcient sampling of virtual light sources.

Moreover, Dachsbacher & Stamminger develo-

ped an importance sampling strategy based on RSMs

improving the performance signiﬁcantly (Dachsba-

cher and Stamminger, 2006). Using a resolution of

512 × 512 px for the ﬁnal image and indirect lights,

37 frames per seconds can be reached. This is not

sufﬁcient for our needs of a high–resolution output.

Ritschel et al. presented a view–adaptive appro-

ach for imperfect shadow maps by using BRSMs

(Ritschel et al., 2009; Ritschel et al., 2011). Depen-

ding on the current camera view, a set of potentially

inﬂuencing VPLs is estimated. Hereby, high frame

rates can be achieved even in large scenes. However,

for occlusion evaluation, ISMs are used which are ba-

sed on a point–based representation of the scene and,

thus, provide an approximated visibility evaluation.

Prutkin, Kaplanyan & Dachsbacher present a

method called Reﬂective Shadow Map Clustering

(RSMC) for clustering VPLs (Prutkin et al., 2012).

In contrast to BRSMs, they use VALs instead of

VPLs. Overall, a single frame can be provided in

140 − 250ms depending on the number of clusters

and RSM size. The importance sampling is similar

to BRSMs and provides a decent quality for com-

plex scenes. Nevertheless, the performance of ren-

dering indirect lights is not sufﬁcient in case of high–

resolution shadow maps.

Another ﬁltering method was presented by Bar

ak,

Bittner & Havran (Bar

ak et al., 2013). They develo-

ped an adaptive sampling of VPLs based on G–buffer

information. It is based on Metropolis–Hastings sam-

pling. In comparison to BRSMs, this approach exhi-

bits better temporal coherence.

With an heuristic sampling Hedman, Karras & Le-

htinen indroduced an efﬁcient VPL distribution using

multiple frames (Hedman et al., 2016). Basically,

they achieve a signiﬁcant performance boost by reu-

sing VPLs from previous frames and creating new

VPLs if necessary.

3 EFFICIENT LIGHTING WITH

RSMs

Among the methods summarized in the previous

section, one approach stands out due to its method

of accumulating and applying indirect light to sur-

faces: LightSkin uses light caches for indirect light

accumulation and subsequently interpolates the accu-

mulated results for the remaining surface points. This

is a quite efﬁcient approach to avoid shading each

pixel in the viewbuffer by each virtual light deducted

from the RSM. However, LightSkin uses each pixel

in the RSM as a VAL. Therefore, this work focuses

on sampling the RSM more efﬁciently by employing

Bidirectional Reﬂective Shadow Maps as presented in

(Ritschel et al., 2011) and adapt the LightSkin rende-

ring pipeline to use only a selected subset of VALs

for computing the proxy light parameters. In order

to make our work easier to understand, we provide a

short summary of the LightSkin method presented in

(Lensing, 2014; Lensing and Broll, 2013b) ﬁrst, and

subsequently, we discuss its drawbacks.

LightSkin Overview

Cache Distribution

As mentioned earlier, LightSkin requires caches to

be distributed on the scene objects in model space.

With our implementation, this is performed in a pre–

computation step for each model. For this step, two

control parameters may be used to affect the distri-

bution results: the maximum distance between cache

and surface point d

, and the maximum angular dif-

ference between cache and surface normal ϑ

. The

main steps for cache generation are:

1. The model is voxelized into cubes with an edge

length of

/2.

2. Surface normals are clustered into bins in a nor-

mal container. These bins hold every normal in a

voxel that matches the bin’s direction.

3. For each voxel, the normal bin representing the

largest surface in the voxel is selected. A cache’s

normal and position are averaged by taking into

account the points with normals deviating less

than an angle of ϑ

from the bin’s center normal.

The surfaces that have been used for averaging are

removed from the normal container. This proce-

dure is repeated until the normal containers are

empty.

4. Optionally, the caches are snapped to existing tri-

angles in case they are “ﬂoating”.

GRAPP 2017 - International Conference on Computer Graphics Theory and Applications

118

5. In a ﬁnal step, caches with similar properties (ge-

nerated in separate voxels) are joined.

For each cache, the position in model space (3 com-

ponents), normal (3 components), represented area

(ﬂoat), and reﬂection coefﬁcient (ﬂoat) are stored in

two RGBA32F textures.

Cache Lighting

In order to generate the proxy lights that represent

the incident indirect light to a cache, ﬁrst, virtual

light sources are generated by rendering a reﬂective

shadow map for the light source. However, instead of

virtual point lights, rather virtual area lights are used.

The area of each VAL is estimated by the projective

area A

disk

of the RSM pixel according to

disk

= z

near

RSM

, (1)

where z

denotes the distance between the surface

point and the camera and w

near

, h

near

, and z

near

de-

note the width, height and depth of the near clipping

plane while w

RSM

and h

RSM

hold the width and height

of the RSM. Using a simple shape like a disk allows

using form factors as analytical solutions for integra-

tion over the surface area in later computation steps.

Proxy Lights Generation

For each of the pre–computed caches, in each frame,

two proxy light sources are generated. One represen-

ting the diffuse indirect light and the other represen-

ting the glossy indirect light. To this end, the reﬂected

light at surface point x in the direction ω is split up

into two separate components:

(x,ω) = L

o,d

(x,ω) +L

o,g

(x,ω), (2)

where L

is the overall outgoing luminance, L

o,d

and

o,g

are the diffuse and glossy components respecti-

vely. Lensing computes these components according

o,d

(x) ≈

VAL

∑

j=1

, j

cos

, j

+ π



, j

− x



(3)

and

o,g

(x,ω) ≈

VAL

∑

j=1

, j

(k + 2)cos

2π

cos

, j

+ π



, j

− x



(4)

where N

VAL

is the number of RSM pixels, Φ

is the

light ﬂux of the VAL, ρ

and ρ

are the diffuse and

glossy color respectively, and x

is a single VAL’s po-

sition. ϑ

is the angle between the light’s normal and

the vector from the light’s center to the surface point

x, ϑ

is the angle between the surface normal and the

vector from the surface point to the light’s center, ϑ

is the angle between the view direction and the re-

ﬂection vector while k is the reﬂection exponent and

is the area of a VAL. We let cos

denote the cosine

clamped to values above zero.

Subsequently, proxy lights are created that repre-

sent this luminance at the cache position. First, the

position of the proxy lights is computed according to

∑

j=1

w(c,x

, j

∑

j=1

w(c,x

)

. (5)

The weights w for the diffuse and glossy term are

computed differently according to

j, d

+ F

~c←l

,disk

j, g

+ cos

~c←l

,disk

(6)

where ε

is a very small, non–zero value avoiding di-

vision by zero in Equation 5 and F

~c←l

is the form fac-

tor between the VAL’s disk area and an inﬁnitesimal

surface at the cache’s position c. Once the position is

known, the proxy light source’s ﬂux can be computed

4πL

,VAL

(x,ω)



− c



(c,ω,ω

)(n

· ω

)

, (7)

where f

(c,ω,ω

) denotes the Bidirectional Reﬂec-

tance Distribution Function (BRDF) at the cache’s po-

sition, in viewing direction ω and incident direction

, and n

is the cache’s normal. Here, we let (...)

denote clamping the value in parentheses to positive

values. This makes (n

· ω

)

the clamped dot pro-

duct. In order to create the diffuse and glossy proxy

light, Equation 3 and Equation 4 are plugged in as

,VAL

Interpolation using the Proxy Lights

The proxy lights of multiple caches are used to inter-

polate indirect light for the surface points in the vie-

wbuffer. However, the proxy lights are not treated as

stationary, but rather moved along with the surface

point in focus. This means interpolating both, proxy

light position and ﬂux. To this end, the closest n ca-

ches need to be known. Lensing stores the referen-

ces of 16 adjacent caches for each vertex during the

per–model pre–computation step. The contributions

of these caches’ proxy lights need to be weighted be-

fore being applied to a surface point. These weights

are deﬁned based on (Ward et al., 1988) and adapted

A Realâ

A¸Stime Global Illumination Approach for High Resolution Reﬂective Shadow Maps in Open World Scenes

119

in (Lensing and Broll, 2013b; Lensing, 2014) for the

diffuse (w

) and glossy (w

) terms according to

(x,c) =



1 −

x − c

c,max



· n

)

(x,c) =



1 −

x − c

c,max



· r

)

(8)

where d

c,max

is the largest distance to one of the 16

caches, n

is the normal at the surface point x, and r

and r

are the reﬂection vectors (the incident light di-

rection mirrored at the surface normal) at the surface

point and cache positions.

With these weights, the interpolation of both,

proxy light position and ﬂux, can be carried out ac-

cording to

v,d

∑

j=1

, j

(x,c)x

,d, j

∑

j=1

, j

(x,c)

v,d

∑

j=1

, j

(x,c)Φ

,d, j

∑

j=1

, j

(x,c)

(9)

These parameters are used to light each surface point

in the view buffer.

Occlusion

As pointed out in (Lensing and Broll, 2013b; Lensing,

2014), visibility is only evaluated for the proxy light

sources rather than all VALs representing it. Instead

of testing for a binary visibility result, the proxy lig-

hts are extended with an estimate for their geometrical

extent: an axis–aligned bounding box with an extent

of (2σ

,2σ

), where σ represents the standard

deviation of the averaged VAL positions for proxy

light creation along each of the main axes. This boun-

ding box is converted to a sphere with a radius mat-

ching the largest dimension of the bounding box. In-

spired by Bunnel’s ambient occlusion computation

scheme (Bunnell, 2005), occlusion is approximated

by creating a disk for each cache with an area that

is computed after cache distribution during the pre–

computation step based on a Voronoi diagram.

Both, the bounding sphere of the proxy light

source as well as the disks of the other caches are ﬁrst

projected onto a unit sphere and, subsequently, to a

unit disk centered around the target cache using the

form factors for a sphere and a disk (cf. (Lensing and

Broll, 2013b)). There, they are converted to quads

with the same area in order to simplify computing the

overlap. The amount of cache quads overlapping with

the proxy light source’s quad is then used as an esti-

mate for the occlusion of that same proxy light.

Drawbacks

The performance ﬁgures of the original LightSkin ap-

proach in (Lensing and Broll, 2013b; Lensing and

Broll, 2013a; Lensing, 2014) drop signiﬁcantly if the

RSM resolution exceeds 256×256 px while rendering

scenes that contain more than 4, 000 caches. This ef-

fect can be attributed to using all virtual lights deri-

ved from the RSM for cache lighting. In open–world

scenes, RSM resolutions of 1024 × 1024 px or higher

are common, which illustrates why the original Light-

Skin approach was only suited for scenes with a rather

small extent such as mixed–reality scenes containing

a small number of virtual objects for which the algo-

rithm was originally designed.

In order to improve on this and to make the Lig-

Create the RSM as seen from the

light source with a high resolution.

Compute the bidirectional importance map.

Compute the vertical and

horizontal phase using SATs.

Retrieve the coordinates of the selected VALs.

Accumulate the contributions of the

selected VALs into LightSkin s proxy light sources

respecting importance sampling.

Interpolate the indirect light from the proxy light

sources for the remaining surface points.

Figure 1: This ﬁgure shows the our provided pipeline for

an efﬁcient sampling of RSMs. After performing the highly

efﬁcient sampling using BRSMs, we combine this approach

with LightSkin.

GRAPP 2017 - International Conference on Computer Graphics Theory and Applications

120

htSkin approach feasible for open–world scenes with

larger extent, we examined several approaches that

focus on ﬁnding virtual lights more efﬁciently, like

BRSMs (Ritschel et al., 2011) and RSMC (Prutkin

et al., 2012). Both methods rely on importance sam-

pling in combination with an efﬁcient approach to

propagate the accumulated light into a reduced repre-

sentation of the scene.

We decided to combine the importance sampling

and virtual light selection of BRSMs with the Lig-

htSkin rendering pipeline since the additional feature

of RSMC–the area representation of VPLs–is cove-

red by the LightSkin approach already. In contrast

to RSMC, where the geometric extent of the VPLs is

transferred to polygons or discs, in (Lensing, 2014),

the geometric extent is approximated using the stan-

dard deviation of the world position of the VALs.

With our contribution combining BRSMs with the

LightSkin approach, we achieve plausible results at

real–time frame rates for RSM resolutions of up to

2048 × 2048 px.

To this end, we suggest the pipeline illustrated in

Figure 1 in order to ﬁrst select a subset of VALs based

on their estimated importance. Initially, we accumu-

late the importance map into cumulative sum textures

vertically and horizontally and subsequently perform

a binary search on these cumulative sums as described

in (Ritschel et al., 2011). The selected VALs need to

be plugged into LightSkin’s indirect light accumula-

tion which means adapting the generation of proxy

lights. For the reduced subset, the contributions have

to be weighted with respect to the importance sam-

pling of the RSM. With the proxy light parameters

computed, LightSkin’s interpolation can be carried

out as described in (Lensing and Broll, 2013a; Len-

sing, 2014). The details of this pipeline are described

in the following section.

4 COMBINING LIGHTSKIN GI

WITH BRSMs

Initially, we create a traditional RSM with a high

resolution from a light source inside the scene as

described in (Dachsbacher and Stamminger, 2005).

Based on the RSM and the current view information,

we estimate the importance of each virtual light using

512 Halton–distributed view samples according to

VAL

∑

· d

VAL→VS

)

· (n

· (−d

VAL→VS

))

D(l

VAL→VS

)

(10)

where I

VAL

denotes the importance of a single VAL,

is the number of view samples used for impor-

tance estimation, n

and n

are the VAL’s and the

view sample’s normals respectively, and d

VAL→VS

the normalized version of the vector between the VAL

and the view sample which is denoted as l

VAL→VS

. In

the denominator, we us an expression that penalizes

importance depending on the distance between VAL

and view sample by clamping the squared distance to

values above 1, according to

D(l

VAL→VS

) =

(

1 if ||l

VAL→VS

< 1

||l

VAL→VS

if ||l

VAL→VS

> 1.

(11)

The resulting importance map is stored in a two–

dimensional R32F texture with the same size as the

RSM. Every pixel represents the estimated impor-

tance of one VAL. Figure 2 illustrates the importance

sampling with 512 Halton–distributed samples.

100 200 300 400

500 600

700 800 900 1,000

100

200

300

400

500

600

700

U - Coordinate [px]

V - Coordinate [px]

View Buffer Sampled with 512 Halton-distributed Samples

Figure 2: This ﬁgure shows the view buffer of size 1024 ×

768 of a street scene in which the coordinates of 512

Halton–distributed samples are illustrated. The view buffer

is sampled at these coordinates to estimate the bidirectional

importance of each VAL in the RSM.

Subsequently, we generate a cumulative sum ver-

tically and, based on that result, horizontally. We use

Summed Area Tables (SATs) to build the cumulative

sums (Crow, 1984; Hensley et al., 2005). This com-

putation can be done in O(log(n)) time with m draw

calls per dimension according to

m = ceil(log

(M)), (12)

A Realâ

A¸Stime Global Illumination Approach for High Resolution Reﬂective Shadow Maps in Open World Scenes

121

where M is the number of elements along the accor-

ding dimension. One two–dimensional R32F texture

is used to store the result of the cumulative sum in

the vertical direction (same size as RSM) and anot-

her single–dimensional R32F texture is used for the

horizontal result (same width as the RSM). To ﬁnd

the VAL selection, we do a binary search within an

extra draw call and store the pixel coordinates of the

selected VALs into a RG32F texture (1 × N

sVAL

) as

described in (Ritschel et al., 2011).

In order to plug the VALs indexed by these coordi-

nates into LightSkin’s rendering pipeline, the selected

subset of VALs needs to be integrated into Light-

Skin’s computation of proxy light parameters for each

cache. For each of these proxy lights, the light’s posi-

tion (Equation 5) is computed according to

∑

sVAL

w(c,x

) · x

, j

VAL

∑

sVAL

w(c,x

) ·

VAL

. (13)

where N

sVAL

is the number of selected VALs, and

VAL

is the probability that the VAL is being selected

by the importance sampling.

Using this position, LightSkin computes the proxy

light’s ﬂux according to Equation 7 where the out-

going luminance is represented as a diffuse and glossy

component (cf. Equation 2)

The components’ terms in Equation 3 and Equa-

tion 4 have to be adapted accordingly to support using

only the selected subset which, for the diffuse term,

yields

o,d

(x) ≈

sVAL

∑

, j

cos

, j

+ π



, j

− x



VAL

(14)

and consequently for the glossy term

o,g

(x,ω) ≈

sVAL

∑

, j

(k + 2)cos

2π

cos

, j

+ π



, j

− x



VAL

(15)

By these modiﬁcations, the existent LightSkin

rendering pipeline is extended to enable compatibility

with BRSM–based VAL selection. With the compu-

ted parameters for each of the two (diffuse and glossy)

proxy lights representing the incident light at each ca-

che, the interpolation can be carried out as described

in Section 3. Subsequently, the incident light at each

cache is occluded with the double–projection method

summarized earlier. The resulting scene is lit with in-

direct diffuse and glossy light with indirect occlusion,

the quality of which depends on cache density.

5 RESULTS

To assess the results and as a reference for the perfor-

mance, we use the original LightSkin approach (accu-

mulating all VALs) as a comparison. LightSkin has

been compared to other state–of–the–art methods and

ground truth data extensively in (Lensing, 2014; Len-

sing and Broll, 2013b). Our method is an extension

to this approach to enhance the performance of indi-

rect lighting without losing visual quality. Therefore,

we evaluate our approach by comparing our results

to the standard LightSkin approach. For our evalua-

tion, we examined several levels of RSM resolutions:

square buffers with sizes of 128

, 256

, 512

, 1024

and 2048

pixels. These were each reduced to sub-

sets of different sizes: 2048, 4096, and 8192 VALs

were selected by importance sampling the RSM. For

all these computations, we used 512 view samples for

importance estimation. However, we also examined

the results for reducing that number to 256 in order to

gain performance with larger selection sets in high–

resolution RSMs as described later in this section.

Visual Quality

The visual quality of the results rendered using only

the selected subset of 2048 VALs is almost indistin-

guishable from the original results when comparing

any RSM resolution. This is illustrated by Figure 6

which compares the indirect light buffers computed

using all VALs (left column) and only the selected

VALs (center column). To illustrate the differences

spatially, we computed the absolute difference bet-

ween the two buffers (right column). The top row

shows the scene for which the buffers were compu-

ted. We used a commonly employed urban street

scene and planted some colored planes to illustrate

color bleeding (note that the buildings on the right

are lit by the light reﬂected off the planes) with ap-

prox. 5500 caches. Despite these images using only

2048 VALs, we also examined the errors for subsets

of 4096 and 8192 VALs. The average over the entire

absolute difference image is plotted in Figure 3. We

observe that the average error decreases when more

VALs are selected but it also increases with the RSM

resolution. This is due to a constant number of VALs

being selected while an increasing number is rejected.

However, the difference images demonstrate that this

increasing error is barely noticeable.

Even in other scenes the results of the indirect lig-

hting are comparable to other approaches. In Figure 4,

we demonstrate the output of the indirect lighting in-

side the Sponza scene. It is clearly visible that the

indirect reﬂection of the green curtain is visible on

GRAPP 2017 - International Conference on Computer Graphics Theory and Applications

122

128 x 128 256 x 256 512 x 512 1024 x 1024 2048 x 2048

0.000

0.005

0.010

0.015

0.020

0.025

0.030

0.035

0.040

RSM resolution

Average Absolute Difference (Averaged over all Channels)

Average Absolute Difference to Full Accumulation (Standard LightSkin)

(1) LS + BRSM (2048 VALs)

(2) LS + BRSM (4096 VALs)

(3) LS + BRSM (8192 VALs)

Figure 3: Average absolute differences between the results

computed with the standard LightSkin algorithm and the re-

sults computed with our method. The evaluation is perfor-

med for different numbers of selected VALs and the results

are plotted over the increasing RSM resolution.

the stone wall. We have used LightSkin with BRSMs

and a sun light with a RSM resolution of 2048

pixels.

8192 VALs are selected to compute the ﬁnal indirect

lighting.

Figure 4: This ﬁgure shows the indirect lighting output

using LS + BRSM with a RSM resolution of 2048

pixels

and a subset of 8192 VALs. Using an NVIDIA GeForce

GTX 1080 we achieve more than 40 FPS.

Temporal Coherence

An important aspect to consider with importance sam-

pling is temporal coherence. Small changes inside

the scene change only a few pixels of the RSM and

therefore the importance sampling output. But with

signiﬁcant changes in scene geometry, light sources,

or camera, we observe some ﬂickering. Especially

for the smaller VAL selection sets and higher reso-

128 x 128 256 x 256 512 x 512 1024 x 1024 2048 x 2048

RSM resolution

Frames per Second (FPS)

Performance Figures of the Compared Algorithms

(1) Standard LightSkin

(2) LS + BRSM (2048 VALs)

(3) LS + BRSM (4096 VALs)

(4) LS + BRSM (8192 VALs)

(5) LS + BRSM (8192 VALs, 256 VS)

Figure 5: Performance ﬁgures comparing the standard Lig-

htSkin algorithm to our new approach which uses a reduced

subset of VALs. The performance is measured in terms of

frames per second for several parameterizations of our met-

hod.

lution RSMs, the ﬂickering becomes clearly visible.

However, increasing the subset to 8192 VALs made

the ﬂickering imperceptible, while still performing at

real–time frame rates—even with only 256 view sam-

ples for importance estimation. This way, we achieve

a stable output despite camera, object, or light source

movements.

Performance

The main advantage of our more sophisticated appro-

ach is that it maintains quality as shown earlier while

decreasing computational complexity drastically. Fi-

gure 5 compares performance in terms of frames per

second of LightSkin with our enhanced algorithm.

Even though LightSkin (1) delivers a competitive

number of frames per second for smaller, impractical

RSM resolutions, our method excels for higher reso-

lution shadow maps. For evaluating our method, we

used 512 view samples for importance estimation and

2048 (2), 4096 (3), and 8192 (4) selected VALs. The

number of VALs directly affects performance, howe-

ver, the more limiting factor is the number of view

samples: when using only 256 view samples (5), we

can use 8196 VALs and achieve higher frame rates for

the largest resolution RSMs.

6 LIMITATIONS

The most noticeable limitation compared to the ori-

ginal LightSkin approach is that we do not support

A Realâ

A¸Stime Global Illumination Approach for High Resolution Reﬂective Shadow Maps in Open World Scenes

123

materials with Sub–surface Scattering (SSS). Sub–

surface Scattering does not affect importance estima-

tion in Equation 10. Therefore, VALs that illuminate

models on their non–camera–facing side are not con-

sidered important. If SSS is to be supported, impor-

tance estimation needs to be adapted and indirect light

accumulation for SSS materials needs to be adapted

as demonstrated and implemented in (Lensing, 2014).

Furthermore, indirect shadows do become less

accurate due to using the standard deviation of the ge-

ometric distribution of the VALs as an estimate for

the indirect light’s area. Since the selected subset

contains less VALs than the original RSM, this stan-

dard deviation becomes less representative. However,

the occlusion itself is based on the caches distributed

on the models. Therefore, the approximation is still

accurate for the selected subset of VALs.

Finally, we still consider temporal coherence an

issue even though we could make it unnoticeable by

increasing the number of selected VALs. In the fu-

ture we plan to explore further methods to achieve

inter–frame stability, for example by employing lazy–

update schemes for the VAL selection, using pseudo-

random seeds, reuse information of the previous

frame, or more sophisticated importance estimation.

This could improve performance further and allow us

to employ this approach for more than just the pri-

mary RSM.

7 CONCLUSION

We presented an approach extending LightSkin.

While preserving its outstanding visual quality it has

proven to be much more efﬁcient. As for the perfor-

mance gain, we achieve approximately 30 fps for a

RSM resolution of 1024 × 1024px where the original

approach delivered only 2.5 fps which results in our

method being roughly 12 times faster than the origi-

nal approach. Furthermore, our method is highly cu-

stomizable to a desired trade–off between quality and

performance by adapting parameters like the num-

ber of view samples, number of selected VALs, the

RSM resolution, and ﬁnally, cache distribution den-

sity. Although some limitations exist, our method de-

livers promising results considering real–time global

illumination in open–world scenes with high resolu-

tion RSMs.

In order to further improve this combination, se-

veral paths can be undertaken: For better temporal co-

herence and to reduce the susceptibility to ﬂickering,

methods for improving the selected subset could be

explored. For example, updating only part of the sub-

set could be possible, but this lazy–updating would

introduce some lag to the computation. One possi-

bility would be to use the visible caches’ view buf-

fer position as view samples for importance estima-

tion. In the light of the ﬁndings of this work, RSMC

can be considered an additional possibility for impro-

ving temporal stability as the more accurate area ap-

proximation might decrease ﬂickering artifacts. Fu-

ture work should explore the effect on temporal stabi-

lity and mitigations to ﬂickering while observing the

additional complexity introduced to the rendering pi-

peline. Moreover, other approaches for rendering the

indirect lighting should be taken into account.

REFERENCES

Anderson, E. C. (1999). Monte Carlo Methods and Impor-

tance Sampling. Lecture Notes, 1999(October):1–8.

Bar

ak, T., Bittner, J., and Havran, V. (2013). Temporally co-

herent adaptive sampling for imperfect shadow maps.

Computer Graphics Forum, 32(4):87–96.

Bunnell, M. (2005). Dynamic Ambient Occlusion and In-

direct Lighting. GPU Gems, 2(2):223–233.

Crassin, C., Neyret, F., Sainz, M., Green, S., and Eise-

mann, E. (2011). Interactive Indirect Illumination

Using Voxel Cone Tracing. Computer Graphics Fo-

rum, 30(7):1921–1930.

Crow, F. C. (1984). Summed-area tables for texture map-

ping. In Proceedings of the 11th annual conference

on Computer graphics and interactive techniques -

SIGGRAPH’84. Association for Computing Machi-

nery (ACM).

Dachsbacher, C. and Stamminger, M. (2005). Reﬂective

Shadow Maps. In Proceedings of the 2005 symposium

on Interactive 3D graphics and games - SI3D ’05, pa-

ges 203–231. Association for Computing Machinery

(ACM).

Dachsbacher, C. and Stamminger, M. (2006). Splatting in-

direct illumination. In Proceedings of the 2006 sym-

posium on Interactive 3D graphics and games - SI3D

’06. Association for Computing Machinery (ACM).

Green, S. (2005). Implementing improved perlin noise.

GPU Gems, 2:409–416.

Hedman, P., Karras, T., and Lehtinen, J. (2016). Sequen-

tial Monte Carlo instant radiosity. In Proceedings of

the 20th ACM SIGGRAPH Symposium on Interactive

3D Graphics and Games - I3D ’16. Association for

Computing Machinery (ACM).

Hensley, J., Scheuermann, T., Coombe, G., Singh, M., and

Lastra, A. (2005). Fast summed-area table genera-

tion and its applications. Computer Graphics Forum,

24(3):547–555.

Kaplanyan, A. and Dachsbacher, C. (2010). Cascaded light

propagation volumes for real-time indirect illumina-

tion. In Proceedings of the ACM SIGGRAPH Sympo-

sium on Interactive 3D Graphics and Games - I3D 10.

Association for Computing Machinery (ACM).

GRAPP 2017 - International Conference on Computer Graphics Theory and Applications

124

Keller, A. (1997). Instant radiosity. In Proceedings of the

24th annual conference on Computer graphics and

interactive techniques - SIGGRAPH ’97. Association

for Computing Machinery (ACM).

riv

anek, J., Gautron, P., Pattanaik, S., and Bouatouch,

K. (2005). Radiance caching for efﬁcient global il-

lumination computation. Visualization and Computer

Graphics, IEEE Transactions on, 11(5):550–561.

Laurent, G., Delalandre, C., Riviere, G. D. L., and Boube-

keur, T. (2016). Forward Light Cuts: A Scalable Ap-

proach to Real-Time Global Illumination. Computer

Graphics Forum, 35(4):79–88.

Lensing, P. (2014). LightSkin: Echtzeitbeleuchtung f

ur Vir-

tual und Augmented Reality. PhD thesis, Ilmenau Uni-

versity of Technology.

Lensing, P. and Broll, W. (2012). Instant indirect illumina-

tion for dynamic mixed reality scenes. In 2012 IEEE

International Symposium on Mixed and Augmented

Reality (ISMAR). Institute of Electrical & Electronics

Engineers (IEEE).

Lensing, P. and Broll, W. (2013a). Efﬁcient shading of indi-

rect illumination applying reﬂective shadow maps. In

Proceedings of the ACM SIGGRAPH Symposium on

Interactive 3D Graphics and Games - I3D ’13. Asso-

ciation for Computing Machinery (ACM).

Lensing, P. and Broll, W. (2013b). LightSkin: Real-time

Global Illumination for Virtual and Mixed Reality. In

Proceedings of the 5th Joint Virtual Reality Confe-

rence, JVRC ’13, pages 17–24, Aire-la-Ville, Swit-

zerland, Switzerland. Eurographics Association.

Prutkin, R., Kaplanyan, A., and Dachsbacher, C. (2012).

Reﬂective Shadow Map Clustering for Real-Time

Global Illumination. In Andujar, C. and Puppo, E.,

editors, Eurographics 2012 - Short Papers. The Euro-

graphics Association.

Ritschel, T., Eisemann, E., Ha, I., Kim, J. D. K., and Sei-

del, H.-P. (2011). Making imperfect shadow maps

view-adaptive: High-quality global illumination in

large dynamic scenes. Computer Graphics Forum,

30(8):2258–2269.

Ritschel, T., Grosch, T., Kim, M. H., Seidel, H.-P., Dachs-

bacher, C., and Kautz, J. (2008). Imperfect shadow

maps for efﬁcient computation of indirect illumina-

tion. ACM Transactions on Graphics, 27(5):1.

Ritschel, T., Grosch, T., and Seidel, H.-P. (2009). Approx-

imating dynamic global illumination in image space.

In Proceedings of the 2009 symposium on Interactive

3D graphics and games - I3D ’09. Association for

Computing Machinery (ACM).

Vardis, K., Papaioannou, G., and Gkaravelis, A. (2014).

Real-time radiance caching using chrominance com-

pression. Journal of Computer Graphics Techniques

(JCGT), 3(4):111–131.

Ward, G. J., Rubinstein, F. M., and Clear, R. D. (1988). A

Ray Tracing Solution for Diffuse Interreﬂection. ACM

SIGGRAPH Computer Graphics, 22(4):85–92.

A Realâ

A¸Stime Global Illumination Approach for High Resolution Reﬂective Shadow Maps in Open World Scenes

125

APPENDIX

Full accumulation vs. 2048 selected VALs

Camera view (all VALs) Camera view (selected VALs) nothing

RSM 1024 × 1024px

Indirect light (all VALs) Indirect light (selected VALs) Difference

RSM 128 × 128px

Avg. abs. diff.: 0.007030

RSM 256 × 256px

Avg. abs. diff.: 0.006244

RSM 512 × 512px

Avg. abs. diff.: 0.009551

RSM 1024 × 1024px

Avg. abs. diff.: 0.012184

RSM 2048 × 2048px

Avg. abs. diff.: 0.028902

Figure 6: This ﬁgure illustrates the indirect light quality comparing the accumulation of all VALs (left column) to using

only a selected subset of 2048 VALs (center column). The right column illustrates the difference between the two indirect

light buffers. Adjacent to the difference images the average absolute error is listed. This comparison is presented for several

resolutions of the RSM (rows) while he ﬁrst row of images provides the camera view of the full accumulation vs. using only

2048 VALs. The scene shown in these images holds approx. 5500 caches.

GRAPP 2017 - International Conference on Computer Graphics Theory and Applications

126