Computers still aren’t smart enough to read lips, and won’t be for a while

Reading lips is a skill usually reserved for fictional spies or the hearing impaired, but researchers have spent years trying to gift the talent to computers, too. A device capable of automated lip-reading would certainly be a game changer, raising questions of personal privacy while simultaneously creating new opportunities in the accessibility and security industries. Don’t get too nervous (or excited) though, Ahmad Hassanat, a researcher at Mu’Tah University in Jordan, says we have a long way to go before machine eyes can tell what we’re saying.

Back in the 16th century, a Spanish Benedictine monk called Pietro Ponce pioneered the seemingly magical art of lip reading. Although the technique probably predates him, Ponce was the first successful lip reading teacher. Then, as now, the technique was primarily used to help people with hearing difficulties interpret speech. But it is also used by others to eavesdrop on conversations. Indeed, various experiments show that our ability to interpret speech improves when we can see the moving lips of the speaker. In other words, almost everybody uses lip reading to a certain extent. That raises an interesting question. Can the process of lip reading be automated and performed by computer? And if so, how successful can this approach be and what kind of threat does it pose to privacy? Today, we get some answers thanks to the work of Ahmad Hassanat at Mu’tah University in Jordan. He outlines the challenges that researchers face in the field of automated lip reading, otherwise known as visual speech recognition. What is clear from his analysis is that if lip reading is going to be successfully automated, significant challenges still need to be overcome. The fundamental process of lip reading is to recognize a sequence of shapes formed by the mouth and then match it to a specific word or sequence of words.