Facebook hasn’t got a lot of good press recently but their latest AI research paper is eye-opening.
Engineers at Facebook have turned to AI techniques to learn new ways to manipulate photos to make you look better.
Previous image processing techniques have focused on removing red-eye or lens flare but in a new paper the tech giant has taken aim at photo-wrecking blink.
The new process is able to ‘open’ closed eyes in a remarkably convincing manner.
Companies like Adobe have pioneered ‘context-aware’ fill and erase in products like Photoshop but the new algorithm does something a bit different. Rather than filling the space with what it ‘thinks’ should be there it Facebook has created a network that is trained to both recognize and replicate convincing open eyes.
Eyes have always been difficult for machines to replicate – they are complex and highly variable. Worse still humans are hard-wired to be particularly attentive to a persons eyes. We can easily tell when they are off.
Facebook nicknamed its network the Generative Adversarial Network. The network is a machine learning system. Facebook tasked it with fooling itself into thinking the eyes it created are real.
The system works by having one part of itself learning to recognise faces while the other repeatedly creates images that ask for feedback. The better the feedback the more the images gradually grow in realism.
The engineers were able to help the process by including an example of the person with their eyes open. This helped the machine learn the shape and colour of the eye.
Learn how machines learn
The results are impressive. There are still some examples of odd artefacts in the paper but Facebook looks to have cracked eye-colour mistmatches and removed obvious stitching from the images. In a few years time, the blink might not be the photo-ruining force it once was.
Published as Eye In-Painting with Exemplar Generative Adversarial Networks – Computer Vision and Pattern Recognition (CVPR) by Brian Dolhansky, Cristian Canton Ferrer at research.fb.com
