Body Motion Analysis for Emotion Recognition in Serious Games

Abstract

In this paper, we present an emotion recognition methodology that utilizes information extracted from body motion analysis to assess affective state during gameplay scenarios. A set of kinematic and geometrical features are extracted from joint-oriented skeleton tracking and are fed to a deep learn-ing network classifier. In order to evaluate the performance of our methodolo-gy, we created a dataset with Microsoft Kinect recordings of body motions ex-pressing the five basic emotions (anger, happiness, fear, sadness and surprise) which are likely to appear in a gameplay scenario. In this five emotions recog-nition problem, our methodology outperformed all other classifiers, achieving an overall recognition rate of 93%. Furthermore, we conducted a second series of experiments to perform a qualitative analysis of the features and assess the descriptive power of different groups of features. View