A Comprehensive Dataset for Machine-Learning-based Lip-Reading Algorithm
Jin Ting, Song Chai, Hongyang Huang, Taoling Tian
Abstract
Lip-reading technology captures the content of the speaker by analyzing the characteristics of the mouth movement. It has a wide application prospect in the fields of daily life, security and so on. The training of the lip-reading model relies on a large amount of data, and the construction of the lip-reading dataset is the first step of lip-reading. The quality of the dataset greatly affects the work of the whole lip-reading system. Therefore, this paper carry out research on the construction of lip-reading dataset. First of all, frames are extracted from original videos by using the Scikit-Video. Then face detection is performed by applying dlib. Lip images are captured by processing the feature points to achieve lip cropping. Finally, data augmentation is performed to enlarged the dataset. The resulting dataset has 33 speakers, each with 7,000 pictures of their lips.