Gesture Recognition in Music Interaction via Computer Vision

M. Sotirios (Greece)


Gesture recognition, image processing, computer vision,modeling, music interaction


This paper presents a computer vision method, which allows pianist fingering retrieval. Computer vision can lead to gesture recognition in music interaction, while the pianist feel and move freely. The method has been developed keeping in mind the specific position of the pianist’s hand. It can be applied for classical musical instruments too. Spatial and temporal models of the gesture are created while a skin model is used for detecting dermal regions on each video frame. An upgraded image processing algorithm is applied in order to extract fingers using less computational power and reducing localization errors. The method produces pairs of notes and fingerings.

Important Links:

Go Back