Towards the Design of Humanoids as Interactive Musical Participants

Daniel M. Lofaro, David Grunberg, Paul Oh, Youngmoo Kim, and Jun Ho Oh


Humanoid Robot, Human Robot Interaction, Musical Beat Tracking, Visual Beat Tracking


An interactive participant in a live musical performance requires a multitude of senses in order to perform. These senses include hearing, sight, and touch. Our long-term goal is to have a full-sized humanoid (Jaemi Hubo) become an interactive participant in a live music ensemble. This paper focuses on Jaemi Hubo's sense of sight and hearing as it pertains to a musical environment. The humanoid's musical awareness is increased through a hybrid approach that combines real-time auditory and visual beat tracking. Experiments contrast audio-only, vision-only and hybrid tracking. These results demonstrate the efficacy of hybrid tracking as a means to realizing a robot's ability to become interactive musical participants.

Important Links:

Go Back