Researchers at the University of Massachusetts Lowell have demonstrated an interesting technique for using Google Glass to detect phone PINs with 83% accuracy from across a room, even when the screen wasn’t visible. The technique used applies an image-recognition algorithm that doesn’t need direct sight of the screen. Instead, it uses a reference image of the target device to detect the angle at which it’s being held, then tracks the shadows from finger taps to detect which on-screen keys are being pressed. “I think of this as a kind of alert about Google Glass, smartwatches, all these devices,” says Xinwen Fu, a computer science professor at UMass Lowell who plans to present the findings with his students at the Black Hat security conference in August. “If someone can take a video of you typing on the screen, you lose everything.”
The odds are you can’t make out the PIN of that guy with the sun glaring obliquely off his iPad’s screen across the coffee shop. But if he’s wearing Google Glass or a smartwatch, he probably can see yours. Researchers at the University of Massachusetts Lowell found they could use video from wearables like Google Glass and the Samsung smartwatch to surreptitiously pick up four-digit PIN codes typed onto an iPad from almost 10 feet away—and from nearly 150 feet with a high-def camcorder. Their software, which used a custom-coded video recognition algorithm that tracks the shadows from finger taps, could spot the codes even when the video didn’t capture any images on the target devices’ displays. “I think of this as a kind of alert about Google Glass, smartwatches, all these devices,” says Xinwen Fu, a computer science professor at UMass Lowell who plans to present the findings with his students at the Black Hat security conference in August. “If someone can take a video of you typing on the screen, you lose everything.” Fu and his students tested a variety of video-enabled devices including Glass, an iPhone 5 camera, and a $72 Logitech webcam. They used Glass to spot a four-digit PIN from three meters away with 83 percent accuracy—and greater than 90 percent with some manual correction of errors. Webcam video revealed the code 92 percent of the time. And the iPhone’s sharper camera caught the code in every case. The researchers have tested the Samsung smartwatch just a few times, but it caught the target PIN about as often as Glass.
