June 14, 2015
June 14, 2015
June 17, 2015
Computers in Education
26.386.1 - 26.386.10
COMPUTER VISION AIDED LIP MOVEMENT CORRECTION TO IMPROVE ENGLISH PRONUNCIATION This paper explored the possibility of improving the English pronunciation for non-Englishspeakers by correcting their mouth-lip movement through visual feedback methods. Compared towriting and reading, speaking is more difficult for non-English speakers since mostly they havefewer opportunities to speak English in their own country. While people are learning English pronunciation, for some words, the mouth-lip movement (e.g.,opening size and duration) is important and it could affect the final pronunciation. We developeda visual pronunciation-training prototype to provide visual feedback for people to correct theirmouth movement while speaking English words. Using computer vision technology, the systemrecords and extracts the mouth shape movement of standard pronunciation into moving contours.When a user practice speaking an English word with this application, his/her mouth movementwill be recorded. The standard mouth moving contours are overlaid on top of the user’s mouthvideo. The user can then correct his/her own mouth movement by comparing the difference ofhis/her mouth with the standard mouth movement. We conducted user evaluation to test if this visual feedback approach can really improve Englishpronunciation by using the application. We recruited 22 international students with pronunciationproblems. Pretest-posttest control group within subjects design was used to evaluate the method.In the design, there were two groups: a control group, which was using standard pronunciationvideo to help pronunciation training and an experimental group, which was using the prototypesystem to help pronunciation training. Subjects in each group took a pretest and a posttest. Inpre- and posttests, the subject was tested on the same test items. The effect of the treatment canbe measured by comparing the results from the pre- and posttests. The difference betweennormal video training and visual pronunciation training can be quantitatively measured bycomparing the results from the control and experimental groups. The evaluation results show thatthe computer vision aided lip movement correction method is effective in improving Englishpronunciation on a small portion of English words, but could not be generalized to all Englishwords.
Wei, S., & Chen, Y., & McGraw, T., & Ginther, A. (2015, June), Computer-Vision-Aided Lip Movement Correction to Improve English Pronunciation Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.23725
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2015 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015