An AI-Driven Robot Smiles Back

“When you’re smiling, the whole world smiles with you,” sang Louis Armstrong (and, you know, Fats Domino, Ella Fitzgerald, Frank Sinatra, etc.). “World,” here being a loose term here that roughly corresponds to “human beings” and excludes inanimate objects and robots. Some robots do smile, but they don’t smile with anyone. Though many respond to spoken commands, communication of the nonverbal type is lost on them.
 
But now, thanks to a group of roboticists at Columbia University, there is a robot that, when smiled at, will smile back. To do so, the robot learned, on its own, how to recognize a smile and how to make one.
 
“If I make different expressions to a robot, it should respond appropriately,” said Boyuan Chen, a researcher at the university’s Creative Machines Lab. He set out to make an emotively resonant robot that did just that.
 
For the purposes of developing robotic facial articulations, the Creative Machines Lab has created a robot, named Eva. In its skull, Eva has a handful of motors and cables that push and pull on its plastic face to roughly imitate the myriad expressions made by humans with their 42 facial muscles. With its few actuators, Eva is capable of making most human expressions. “The goal was not, in terms of hardware, to mimic the mechanical design of human faces,” said Chen. The goal was to have a robot recognize an expression and to mimic that on its own.

Editor's Pick: Service with a Robot Smile
 
To do that, Eva goes through two stages. In the first, she looks at a human face and extracts a few crucial landmarks. “Then, with this as input, the robot basically imagines what it would look like if it were making the same expression,” said Chen. “Then it will output the movements to make that self-image.”
 

Self-Derived Learning

 
Chen derived this algorithm from his own experience as a pianist. He has played for 23 years now, performing—and practicing—all over the world. “One of the most important things for me is that there has to be a mirror—through the mirror I can see my body gestures and I can see my finger movements,” he said. “I imagine what I will look like before I go on stage.”
 
For the robot to imagine what it will look like it, it first had to learn its own expressions. Rather than teach the robot by showing it faces and telling it when it has accurately imitated them and when it hasn’t—supervised learning—Chen and his colleagues turned to self-derived learning. In essence, they left Eva alone to learn from herself. “It learns just by performing” he said. “Just like babies making different facial expressions, the robot randomly moves its muscles—it’s motor babbling.” In fact, during the entire training process, the robot never sees a human face.

Watch Related Video: Advances in Social Domestic Robots

The facial automaton’s bank of expressions is derived from data it created on its own. So when it sees a flesh and blood human smile, and recognizes those landmarks, it draws on its own experience to reproduce it. “Basically, it will say, ‘Ok, I’m going to make the expression that I made some time ago. How do I know I’m correct? Because I’ve made that expression already—I can repeat what I have done by myself,’” explained Chen.
 

Robot-Human Interaction

Eva’s imitative smile is part of a larger effort on the part of the lab to make robot/human interactions more natural and less awkward. “One day I went to a hotel that had robots,” said Chen, “When I wanted to interact and take a photo with the robot it basically said, ‘I’m working, get out of my way.’ It shouldn’t be like that. It should appreciate interacting with humans.”

Most Popular: 10 Humanoid Robots of 2020
 
The mimicking robot has many possible applications, putting people at ease in museums, educational settings, or anywhere where a robot might help with navigation. But an obvious next step is to get the robot to go beyond imitation with its responses. Humans, after all, don’t just smile when smiled at.
 
They can laugh at sarcasm, beam sympathy in response to sadness, go agape at anger. “That’s one of the things we are working on now,” said Chen. “We are going to require the robot to learn physical social norms through human interactions. And we want to learn this process such that it can generalize to any scenario or situation.”
 
Michael Abrams is a science and technology writer based in Westfield, NJ.