[ACM Press the 5th international conference - Vancouver, British Columbia, Canada (2003.11.05-2003.11.07)] Proceedings of the 5th international conference on Multimodal interfaces - ICMI '03 - Pointing gesture recognition based on 3D-tracking of face, hands and head orientation

✍ Scribed by Nickel, Kai; Stiefelhagen, Rainer

• Book ID: 123615027
• Publisher: ACM Press
• Year: 2003
• Tongue: English
• Weight: 205 KB
• Category: Article
• ISBN-13: 9781581136210
• DOI: 10.1145/958432.958460

✦ Synopsis

In this paper, we present a system capable of visually detecting pointing gestures and estimating the 3D pointing direction in real-time. In order to acquire input features for gesture recognition, we track the positions of a person's face and hands on image sequences provided by a stereocamera. Hidden Markov Models (HMMs), trained on different phases of sample pointing gestures, are used to classify the 3D-trajectories in order to detect the occurrence of a gesture. When analyzing sample pointing gestures, we noticed that humans tend to look at the pointing target while performing the gesture. In order to utilize this behavior, we additionally measured head orientation by means of a magnetic sensor in a similar scenario. By using head orientation as an additional feature, we observed significant gains in both recall and precision of pointing gestures. Moreover, the percentage of correctly identified pointing targets improved significantly from 65% to 83%. For estimating the pointing direction, we comparatively used three approaches: 1) The line of sight between head and hand, 2) the forearm orientation, and 3) the head orientation.