Multi-View Video Synthesis Through Progressive Synthesis and Refinement

Mohamed Ilyes Lakhal, Oswald Lanz, Andrea Cavallaro

2023

Abstract

Multi-view video synthesis aims to reproduce a video as seen from a targeted viewpoint. This paper proposes to tackle this problem using a multi-stage framework to progressively add more details on the synthesized frames and refine wrong pixels from previous predictions. First, we reconstruct the foreground and the background by using 3D mesh. To do so, we leverage the one-to-one correspondence between rendered mesh faces between the input and the target view. Then, the predicted frames are defined with a recurrence formula to correct wrong pixels and adding high-frequency details. Results on the NTU RGB+D dataset show the effectiveness of the proposed approach against frame-based and video-based state-of-the-art models.

Download


Paper Citation


in Harvard Style

Ilyes Lakhal M., Lanz O. and Cavallaro A. (2023). Multi-View Video Synthesis Through Progressive Synthesis and Refinement. In Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP; ISBN 978-989-758-634-7, SciTePress, pages 230-238. DOI: 10.5220/0011759400003417


in Bibtex Style

@conference{visapp23,
author={Mohamed Ilyes Lakhal and Oswald Lanz and Andrea Cavallaro},
title={Multi-View Video Synthesis Through Progressive Synthesis and Refinement},
booktitle={Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP},
year={2023},
pages={230-238},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011759400003417},
isbn={978-989-758-634-7},
}


in EndNote Style

TY - CONF

JO - Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP
TI - Multi-View Video Synthesis Through Progressive Synthesis and Refinement
SN - 978-989-758-634-7
AU - Ilyes Lakhal M.
AU - Lanz O.
AU - Cavallaro A.
PY - 2023
SP - 230
EP - 238
DO - 10.5220/0011759400003417
PB - SciTePress