Quaternionic signal processing techniques for automatic evaluation of dance performances from MoCap data

Abstract

In this paper, the problem of automatic dance performance evaluation from human Motion Capture (MoCap) data, is addressed. A novel framework is presented, using data captured by Kinect-based human skeleton tracking, where the evaluation of user's performance is achieved against a goldstandard performance of a teacher. The framework addresses several technical challenges, including global and local temporal synchronization, spatial alignment and comparison of two "dance motion signals". Towards the solution of these technical challenges, a set of appropriate quaternionic vector-signal processing methodologies is proposed, where the 4D (spatiotemporal) human motion data are represented as sequences of pure quaternions. Such a quaternionic representation offers several advantages, including the facts that joint angles and rotations are inherently encoded in the phase of quaternions and the three coordinates variables (X,Y ,Z) are treated jointly, with their intra-correlations being taken into account. Based on the theory of quaternions, a number of advantageous algorithms are formulated. Initially, global temporal synchronization of dance MoCap data is achieved by the use of quaternionic crosscorrelations, which are invariant to rigid spatial transformations between the users. Secondly, a quaternions-based algorithm is proposed for the fast spatial alignment of dance MoCap data. Thirdly, the MoCap data can be temporally synchronized in a local fashion, using Dynamic Time Warping techniques adapted to the specific problem. Finally, a set of quaternionic correlationbased measures (scores) are proposed for evaluating and ranking the performance of a dancer. These quaternions-based scores are invariant to rigid transformations, as proved and demonstrated. A total score metric, through a weighted combination of three different metrics is proposed, where the weights are optimized using Particle Swarm Optimization (PSO). The presented experimental results using the Huawei/3DLife/EMC2 dataset are promising and verify the effectiveness of the proposed methods. View