Abstract
MPEGโ4 supports the definition, encoding, transmission, and animation of 3โD head and body models. These features can be used for a variety of different applications ranging from low bitโrate video coding to character and avatar animation. In this article, an entire system for the analysis of facial expressions from image sequences and their synthesis is presented. New methods for the estimation of MPEGโ4 facial animation parameters as well as scene illumination are proposed. Experiments for different applications demonstrate the potential of using facial animation techniques in video analysis and synthesis. A modelโbased codec is presented that is able to encode headโandโshoulder video sequences at bitโrates of about 1 kbit/s. Besides the low bitโrate, many enhancements and scene modifications can be easily applied, like scene lighting changes or cloning of expressions for character animation. But also for the encoding of arbitrary sequences, 3โD knowledge can help to increase the coding efficiency. With our modelโaided codec, bitโrate reductions of up to 45% at the same quality can be achieved in comparison to standard hybrid video codecs. ยฉ 2004 Wiley Periodicals, Inc. Int J Imaging Syst Technol 13, 245โ256, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ima.10072