Iterative 3D Deformable Registration from Single-view RGB Images using Differentiable Rendering

Arul Periyasamy, Max Schwarz, Sven Behnke

2022

Abstract

For autonomous robotic systems, comprehensive 3D scene parsing is a prerequisite. Machine learning techniques used for 3D scene parsing that incorporate knowledge about the process of 2D image generation from 3D scenes have a big potential. This has sparked an interest in differentiable renderers that provide approximate gradients of the rendered image with respect to scene and object parameters. An efficient differentiable renderer facilitates approaching many 3D scene parsing problems using a render-and-compare framework, where the object and scene parameters are optimized by minimizing the difference between rendered and observed images. In this work, we introduce StilllebenDR, a light-weight scalable differentiable renderer built as an extension to the Stillleben library and use it for 3D deformable registration from single-view RGB images. Our end-to-end differentiable pipeline achieves results comparable to state-of-the-art methods without any training and outperforms the competing methods significantly in the presence of pose initialization errors.

Download


Paper Citation


in Harvard Style

Periyasamy A., Schwarz M. and Behnke S. (2022). Iterative 3D Deformable Registration from Single-view RGB Images using Differentiable Rendering. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP, ISBN 978-989-758-555-5, pages 107-116. DOI: 10.5220/0010817100003124


in Bibtex Style

@conference{visapp22,
author={Arul Periyasamy and Max Schwarz and Sven Behnke},
title={Iterative 3D Deformable Registration from Single-view RGB Images using Differentiable Rendering},
booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP,},
year={2022},
pages={107-116},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010817100003124},
isbn={978-989-758-555-5},
}


in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP,
TI - Iterative 3D Deformable Registration from Single-view RGB Images using Differentiable Rendering
SN - 978-989-758-555-5
AU - Periyasamy A.
AU - Schwarz M.
AU - Behnke S.
PY - 2022
SP - 107
EP - 116
DO - 10.5220/0010817100003124