Let's All Dance: Enhancing Amateur Dance Motions

Abstract

Professional dancing is characterized by high impulsiveness, elegance, and aesthetic beauty. In order to reach the desired professionalism, it requires years of long and exhausting practice, good physical condition, musicality, but also, a good understanding of the choreography. Capturing dance motions and transferring them into digital avatars is commonly used in the film and entertainment industries. However, so far, access to high-quality dance data is very limited, mainly due to the many practical difficulties in motion capturing the movement of dancers, which makes it prohibitive for large-scale acquisitions. In this paper, we present a model that enhances professionalism to amateur dance movements, allowing the movement quality to be improved in both the spatial and temporal domains. The model consists of a dance-to-music alignment stage responsible for learning the optimal temporal alignment path between the dance and music, and a dance-enhancement stage that injects features of professionalism in both the spatial and temporal domains. To learn a homogeneous distribution and credible mapping between the heterogeneous professional and amateur datasets, we generate amateur data from professional dances taken from the AIST++ dataset. We demonstrate the effectiveness of our method by comparing it with two baseline motion transfer methods via thorough qualitative visual controls, quantitative metrics, and a perceptual study. We also provide temporal and spatial module analysis to examine the mechanisms and necessity of key components in our framework.

Main Contributions

• We introduce the concept of enhancing professionalism in dance movements and provide a first definition of what dance professionalism is and how a professional dance can be distinguished from an amateur one.
• We design a novel two-stage deep learning framework that extracts meaningful features from motion inputs, according to the newly defined professionalism criteria, to improve the quality of dance motions. It integrates a reconstruction loss to preserve the original content of the dance and a consistency loss to maintain the temporal coherency of the reconstructed motion.
• We propose a novel model designed to synchronize 3D dance motions with reference audio under non-uniform and irregular misalignment.
• We present thorough evaluations and an ablation study to examine the efficiency and necessity of our method.

BibTeX

@article{Zhou:2023:LetsAllDance,
 author    	= {Zhou, Qiu and  Li, Manyi and Zeng, Qiong and Aristidou, Andreas and Zhang, Xiaojing and Chen, Lin and Tu, Changhe},
 title     	= {Let’s all dance: Enhancing amateur dance motions}, 
 journal   	= {Computational Visual Media}, 
 issue_date	= {September 2023},
 volume    	= {9},
 number    	= {3},
 month     	= {sep},
 pages		= {531--550},
 doi 		= {10.1007/s41095-022-0292-6},
 publisher 	= {Springer},
 address   	= {},
 year      	= {2023}
}