Unsupervised Domain Extension for Nighttime Semantic Segmentation in Urban Scenes

Sebastian Scherer, Robin Schön, Katja Ludwig, Rainer Lienhart

2021

Abstract

This paper deals with the problem of semantic image segmentation of street scenes at night, as the recent advances in semantic image segmentation are mainly related to daytime images. We propose a method to extend the learned domain of daytime images to nighttime images based on an extended version of the CycleGAN framework and its integration into a self-supervised learning framework. The aim of the method is to reduce the cost of human annotation of night images by robustly transferring images from day to night and training the segmentation network to make consistent predictions in both domains, allowing the usage of completely unlabelled images in training. Experiments show that our approach significantly improves the performance on nighttime images while keeping the performance on daytime images stable. Furthermore, our method can be applied to many other problem formulations and is not specifically designed for semantic segmentation.

Paper Citation

in Harvard Style

Scherer S., Schön R., Ludwig K. and Lienhart R. (2021). Unsupervised Domain Extension for Nighttime Semantic Segmentation in Urban Scenes. In Proceedings of the 2nd International Conference on Deep Learning Theory and Applications - Volume 1: DeLTA, ISBN 978-989-758-526-5, pages 38-47. DOI: 10.5220/0010551500380047

in Bibtex Style

@conference{delta21,
author={Sebastian Scherer and Robin Schön and Katja Ludwig and Rainer Lienhart},
title={Unsupervised Domain Extension for Nighttime Semantic Segmentation in Urban Scenes},
booktitle={Proceedings of the 2nd International Conference on Deep Learning Theory and Applications - Volume 1: DeLTA,},
year={2021},
pages={38-47},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010551500380047},
isbn={978-989-758-526-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 2nd International Conference on Deep Learning Theory and Applications - Volume 1: DeLTA,
TI - Unsupervised Domain Extension for Nighttime Semantic Segmentation in Urban Scenes
SN - 978-989-758-526-5
AU - Scherer S.
AU - Schön R.
AU - Ludwig K.
AU - Lienhart R.
PY - 2021
SP - 38
EP - 47
DO - 10.5220/0010551500380047