Integrated deep learning classification of Mudras of Bharatanatyam: A case of hand gesture recognition

Abstract

Bharatanatyam is a famous Indian classical dance which incorporates the essence of hand gestures or mudras as a means of communication by performers. Bharatanatyam has around 28 single and 23 double hand gestures respectively. Recognizing the gestures is challenging due to minor structural differences between the gestures and presence of gestures with higher structural similarity. This work proposes a deep learning strategy combining Scale invariant feature transform features with convolutional neural network to address the challenge in accurate recognition of mudras. The gestures are segmented using active net based segmentation model to reduce the influence of background in gesture recognition. The gestures are then grouped based on similarity and convolutional neural network is trained for each group to solve the problem of classification in presence of higher structural similarity. Convolutional neural network with structural conflict minimization kernel is used to classify the gestures. The proposed model attained an accuracy of 95% in classification of mudras and it has lower false positives of 2%.