MIT may have solved one of photography’s most annoying problems

You may not give much though to that faint, blurry reflection of yourself that you see when you look through a window, in fact, you probably just subconsciously ignore it, but it’s hard not to notice that reflection when its ruining a picture. Many pictures have been ruined by being taken through a window and having a blurry reflection obscuring what you’re trying to take a picture of. That’s where some MIT researchers think they can be of assistance, as they have developed a new algorithm that can automatically remove those reflections. 

Because of those pesky reflections, windows are great for blurry selfies but not so great for capturing what’s beyond the glass. With a new algorithm from MIT researchers, though, a through-the-window shot will be just that: a photo of what you see through the window, without a partial reflection of the photographer. When a window sends a reflection back to a camera lens, it’s actually sending two slightly different reflections, one from one side of the glass and another from the other side (this is more apparent with double-paned windows). It’s difficult for an algorithm to recognize a single reflection, since it can be compared only to the rest of the image. But with two reflections, the algorithm can pick out which pixels come from beyond the window and which are reflected. Besides finding double reflections, the researchers’ technique also involves looking for statistically anomalous pixels—those that don’t seem to be a part of the overall image. The researchers fed an algorithm 50,000 images to teach it how to differentiate pixels that are meant to be there from pixels that aren’t. When the “trained” algorithm divides an image into an 8×8 pixel square, it can reliably distinguish between reflection and overall image.