Arabic Lip-reading System: A Combination of Hypercolumn Neural Network Model with Hidden Markov Model

A. El.Sagheer, N. Tsuruta, and R.-I. Taniguchi (Japan)


Lip-reading system, visual features extraction, hypercolumn neural network model, and Hidden markov model.


In recent year, lip-reading systems have received much attention, since it plays an important role in human communication with computer especially for hearing impaired or elderly people. In this paper, we introduce a new visual feature representation combines the Hypercolumn Neural Network model (HCM) with Hidden Markov Model (HMM) to achieve a complete lip-reading system. To check our system performance we introduce the Arabic language to it. According to our knowledge, this is the first time that a visual speech recognition system is applied for Arabic language. Experiments include different Arabic sentences gathered from different native speakers (Male & Female).

Important Links:

Go Back