Real-Time Detection-based Modeling of Finger Segments

Samuel de Sousa and Jan Ernst


Hand Modeling and Tracking, Computer Vision, Machine Learning, Computer Human Interaction


The bottleneck of many data-intensive and business-critical applications used to be the available computing power. With the advent of distributed computing and data mining, a new frontier is now the efficient manual exploration of large data to elucidate business value. This paper describes a novel vision-based hand modeling system as one step towards new ways of naturally interacting with large heterogeneous data. The many degrees of freedom of the hand make it uniquely suited to this purpose, but also pose a computational challenge in automatic reconstruction. We aspire to obtain real-time performance in a purely frame-by-frame detection-based architecture and meet this challenge by combining bottom-up hypothesis generation with top-down pruning. As real data is critical for training, we furthermore describe a novel method for gathering large corpora of automatically annotated hand data. Our experiments were conducted on 13 users under large pose variability of both male and female hands.

Important Links:

Go Back