Semi-Supervised Domain Adaptation with CycleGAN Guided by Downstream Task Awareness

Annika Mütze, Matthias Rottmann, Matthias Rottmann, Hanno Gottschalk

2023

Abstract

Domain adaptation is of huge interest as labeling is an expensive and error-prone task, especially on pixel-level like for semantic segmentation. Therefore, one would like to train neural networks on synthetic domains, where data is abundant. However, these models often perform poorly on out-of-domain images. Image-to-image approaches can bridge domains on input level. Nevertheless, standard image-to-image approaches do not focus on the downstream task but rather on the visual inspection level. We therefore propose a “task aware” generative adversarial network in an image-to-image domain adaptation approach. Assisted by some labeled data, we guide the image-to-image translation to a more suitable input for a semantic segmentation network trained on synthetic data. This constitutes a modular semi-supervised domain adaptation method for semantic segmentation based on CycleGAN where we refrain from adapting the semantic segmentation expert. Our experiments involve evaluations on complex domain adaptation tasks and refined domain gap analyses using from-scratch-trained networks. We demonstrate that our method outperforms CycleGAN by 7 percent points in accuracy in image classification using only 70 (10%) labeled images. For semantic segmentation we show an improvement of up to 12.5 percent points in mean intersection over union on Cityscapes using up to 148 labeled images.

Paper Citation

in Harvard Style

Mütze A., Rottmann M. and Gottschalk H. (2023). Semi-Supervised Domain Adaptation with CycleGAN Guided by Downstream Task Awareness. In Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP; ISBN 978-989-758-634-7, SciTePress, pages 80-90. DOI: 10.5220/0011630900003417

in Bibtex Style

@conference{visapp23,
author={Annika Mütze and Matthias Rottmann and Hanno Gottschalk},
title={Semi-Supervised Domain Adaptation with CycleGAN Guided by Downstream Task Awareness},
booktitle={Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP},
year={2023},
pages={80-90},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011630900003417},
isbn={978-989-758-634-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP
TI - Semi-Supervised Domain Adaptation with CycleGAN Guided by Downstream Task Awareness
SN - 978-989-758-634-7
AU - Mütze A.
AU - Rottmann M.
AU - Gottschalk H.
PY - 2023
SP - 80
EP - 90
DO - 10.5220/0011630900003417
PB - SciTePress