Machine Learning researchers have been working for a long time on faces. Most of the work which I read is related to either face detection or face recognition. Why so much focus on only these two problems? I think the answer is quite straightforward. First, these two problems are very important from the security point of view. Second, we know that we still have a lot to improve. Talking about just the face recognition, we also can’t disagree that it has grown fantastically right from semi-automated computer-based facial recognition made during the mid-1960s by Woodrow Wilson Bledsoe to faceid feature of iPhone X.

Left: rt.com, Right: WIKIPEDIA COMMONS

Recently, I conducted a hackathon, where the task was to develop a real-time face recognition system for an engineering school. When the students came to me for a hint, my answer was to use cloud services as it would be an easy and fast way to hack the problem. Amazon and Microsoft are market leaders in cloud computing with Amazon Web Services and Microsoft Azure as their subsidiaries respectively. While the Microsoft has Computer Vision under Microsoft Cognitive Services to work on images and videos, Amazon has AWS Rekognition. Facial recognition of AWS Rekognition service is capable of recognizing 100 different faces from a single image. They run some incredibly powerful algorithms trained by some very powerful machines.

Image: Screenshot from SendVids

There are several other examples that show that we have become pretty good with face recognition. Due to this. we now see a lot of new type of machine learning problem being focused upon. One such technique that made its name a lot into the news was: face swapping, but this time, it was not because of good reasons. There’s a video of Gal Gadot having sex with her stepbrother on the internet. But it’s not really Gadot’s body, and it’s barely her own face. It’s an approximation, face-swapped to look like she’s performing it. Deepfake is the name of the Reddit account that posted this video. He was using simple Keras and Tensorflow backend with data from Google image search and Youtube.

Face2Face is one more such fantastic project that does the real-time facial reenactment of a monocular target video sequence. The goal of the project was to animate the facial expressions of the target video by a source actor and re-render the manipulated output video in a photo-realistic fashion. You can see the video of this project on youtube: https://www.youtube.com/watch?reload=9&v=ohmajJTcpNk and can read the paper from http://niessnerlab.org/projects/thies2016face.html.

Talking about fake, another very popular research has been: Progressive Growing of GANs for Improved Quality, Stability, and Variation. GAN is “generative adversarial network,” and it’s a system that uses two neural networks — one generates things and the other evaluates them. These algorithms can generate artificial “photos” that look real to humans. For its project, NVIDIA found that training the neural network using low-resolution photos of real celebrities and then ramping up to high-res photos helped to both speed up and stabilize the “learning” process, allowing the AI to create “images of unprecedented quality.” To demonstrate the system, NVIDIA trained the neural network using the CelebA HQ database of famous faces. Once it had been trained on these real faces, the AI was able to begin generating photo-realistic photos of fake people.

Although, it is stunning but is it just what we are going to use deep learning for on faces? I believe no. There are hundreds of other problems that can be solved using machine learning over faces. I see its application in a variety of fields including healthcare and education. One such amazing field is psychology. Let’s talk about it in rest of the blog.

My area of work has been: the psychology of facial attractiveness and machine learning.  Understanding how we perceive attractiveness has been one of the challenging tasks for both psychologist and neurologist. There are several factors that govern how we see facial beauty. Anthony C. Little who is a professor at the University of Bath, Department of Psychology has done a wonderful work on decoding it. He is part of a more awesome lab “The Face Research Lab” run by Ben Jones and Lisa DeBruine and is based in the Institute of Neuroscience and Psychology at the University of Glasgow. Some of the findings of there research work are really surprising.

Facial attractiveness has been related to some common factors like facial symmetry, look alikeness of partners and facial averageness. But surprisingly several other factors also plan an important role like alcohol consumption, mensuration cycle, sleep etc.

I believe if understood correctly, we can use the findings of these researchers to generate attractive synthetic faces. These synthetic faces could than be custom designed using machine learning for the users, based on how they perceive facial attractiveness. I see these synthetic faces as being used for digital assistants like Alexa, Siri and Google Assist which are still just voices. Last year I wrote my Master’s thesis on this topic, trying to come up with a system that can generate such attractive synthetic faces. I got a distinction on my dissertation, however, this topic kept me fascinated by machine learning and kept me working on it even after completion of my Masters. I started with digital assistants but now I see the application of this integration of psychology and ML into Customer Relationship Management, Gaming, Dating Applications and what not.

Ex Machina, Blogspot

I just recently became part of a small venture that is trying to build a recommender system for dating application. This recommendation system will use the same idea of facial attractiveness to recommend dating partners to the users. I live with a vision that we might see something like Ava from the movie Ex Machina very soon. I believe face need more machine learning research than just face detection and recognition.

 

Vishal Yadav