Transformer-Based Two-level Approach for Music-driven Dance Choreography

Yanbo Cheng, Yingying Wang

2024

Abstract

Human dance motions are complex, creative, and artistic expressions. Synthesizing high-quality dance motions and synchronizing them to music has always been a challenge in animation research. Three problems in synthesizing dance motions are presented: 1) dance movements are complex non-linear motions that follow high-level structures of the dance genre over a long horizon, yet must maintain a stylistic consistency; 2) even for the same genre, dance movements require diversity, expressiveness, and nuances to appear natural and realistic; 3) spatial-temporal features of dance movements can be influenced by music. In this paper, we address these issues using a novel two-level transformer-based dance generation system that can synthesize dance motions to match the audio input. Our high-level transformer network performs the choreography and generates dance movements with consistent long-term structure, and our low-level implementer infuses diversity and realizes actual dance performances. This two-level approach not only allows us to generate dances that are consistent in structure, but also enables us to effectively add styles learnt from a wide range of dance datasets. When training the choreography model, our approach fully utilizes existing dance datasets, even those without musical accompaniment, and thus differs from previous research that requires dance training data to be accompanied by music. Results in this work demonstrate that our two-level system generates high-quality dance motions that flexibly adapt to varying musical conditions trained on a dataset of dance sequences without accompanying music.

Paper Citation

in Harvard Style

Cheng Y. and Wang Y. (2024). Transformer-Based Two-level Approach for Music-driven Dance Choreography. In Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 1: GRAPP; ISBN 978-989-758-679-8, SciTePress, pages 127-139. DOI: 10.5220/0012434500003660

in Bibtex Style

@conference{grapp24,
author={Yanbo Cheng and Yingying Wang},
title={Transformer-Based Two-level Approach for Music-driven Dance Choreography},
booktitle={Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 1: GRAPP},
year={2024},
pages={127-139},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012434500003660},
isbn={978-989-758-679-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 1: GRAPP
TI - Transformer-Based Two-level Approach for Music-driven Dance Choreography
SN - 978-989-758-679-8
AU - Cheng Y.
AU - Wang Y.
PY - 2024
SP - 127
EP - 139
DO - 10.5220/0012434500003660
PB - SciTePress