Authors:
Benedikt Kottler
1
;
Ludwig List
1
;
Dimitri Bulatov
1
and
Martin Weinmann
2
Affiliations:
1
Fraunhofer Institute for Optronics, System Technologies and Image Exploitation (IOSB), Gutleuthausstrasse 1, 76275 Ettlingen, Germany
;
2
Institute of Photogrammetry and Remote Sensing, Karlsruhe Institute of Technology (KIT), Englerstr. 7, 76131 Karlsruhe, Germany
Keyword(s):
Edges, Façades, GAN, Inpainting, Semantic Segmentation, Texture Synthesis.
Abstract:
Realistic representation of building walls from images is an important aspect of scene understanding and has many applications. Often, images of buildings are the only input for texturing 3D models, and these images may be occluded by vegetation. One task of image inpainting is to remove these clutter objects. Since the disturbing objects can also be of a larger scale, modern deep learning techniques should be applied to replace them as realistically and context-aware as possible. To support an inpainting network, it is useful to include a-priori information. An example of a network that considers edge images is the two-stage GAN model denoted as EdgeConnect. This idea is taken up in this work and further developed to a three-stage GAN (3GAN) model for façade images by additionally incorporating semantic label images. By inpainting the label images, not only a clear geometric structure but also class information, like position and shape of windows and their typical color distribution
, are provided to the model. This model is compared qualitatively and quantitatively with the conventional version of EdgeConnect and another well-known deep-learning-based approach on inpainting which is based on partial convolutions. This latter approach was outperformed by both GAN-based methods, both qualitatively and quantitatively. While the quantitative evaluation showed that the conventional EdgeConnect method performs minimally best, the proposed method yields a slightly better representation of specific façade elements.
(More)