PanDepth: Joint Panoptic Segmentation and Depth Completion

Juan Lagos, Esa Rahtu

2023

Abstract

Understanding 3D environments semantically is pivotal in autonomous driving applications where multiple computer vision tasks are involved. Multi-task models provide different types of outputs for a given scene, yielding a more holistic representation while keeping the computational cost low. We propose a multi-task model for panoptic segmentation and depth completion using RGB images and sparse depth maps. Our model successfully predicts fully dense depth maps and performs semantic segmentation, instance segmentation, and panoptic segmentation for every input frame. Extensive experiments were done on the Virtual KITTI 2 dataset and we demonstrate that our model solves multiple tasks, without a significant increase in computational cost, while keeping high accuracy performance. Code is available at https://github.com/juanb09111/PanDepth.git.

Paper Citation

in Harvard Style

Lagos J. and Rahtu E. (2023). PanDepth: Joint Panoptic Segmentation and Depth Completion. In Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP; ISBN 978-989-758-634-7, SciTePress, pages 635-643. DOI: 10.5220/0011685200003417

in Bibtex Style

@conference{visapp23,
author={Juan Lagos and Esa Rahtu},
title={PanDepth: Joint Panoptic Segmentation and Depth Completion},
booktitle={Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP},
year={2023},
pages={635-643},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011685200003417},
isbn={978-989-758-634-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP
TI - PanDepth: Joint Panoptic Segmentation and Depth Completion
SN - 978-989-758-634-7
AU - Lagos J.
AU - Rahtu E.
PY - 2023
SP - 635
EP - 643
DO - 10.5220/0011685200003417
PB - SciTePress