A System for Speech and 3D Facial Image Acquisition, Modeling and Analysis

Prof. Dr. Elmar Nöth, University of Erlangen-Nuremberg

Date: 16/03/2012
Time: 14:15-15:45
Room: H-F 114

A system for speech and 3D facial image acquisition, analysis and display is presented. The system is intended for patients with speech disorders and impediments of facial gestures, e.g., people with Parkinson's disease or facial palsy. The facial image is captured in parallel by a Time-Of-Flight (ToF) camera and a web camera or a Kinect camera (structured light and web camera). The images are merged to display a moving 3D face to the user. Additionally the face is analyzed in 2D and 3D (e.g. to detect face movement or movement abilities of special regions of the face). The speech signal is characterized using speaker recognition technology and a speech recognition system.