Rubi the Robot Heads Back to School

Two years after first reporting on the development of Rubi the robot, a teacher’s aide droid, the Dana Foundation checks back in with one of the creators to learn about the project’s progress.

Last week was the first day of school for Rubi, the social robot being tested as a teacher’s aide in a San Diego early-childhood center, and Rubi’s chief developer was as nervous as a parent. After all, Rubi’s predecessor lasted only a couple of hours before the 18- to 24-month-olds at the center—apparently mistaking Rubi for a super-size Mrs. Potato Head—yanked off its sophisticated robotic arm and set the project back by months.

To solve the problem, researchers equipped Rubi with sensors that detected when a child might be getting too rough and triggered the robot to cry, digital tears and all. Seeing this, the children backed off, just as they typically do in interactions with one another. For Javier Movellan, Ph.D., and his team at the Machine Perception Laboratory at the University of California, San Diego, it was an illustration of how important social cues are when learning how to play well with others.

“The brain is wired to respond to social stimuli,” Movellan said. “We know that the social and emotional aspects of learning are very important. This is particularly true in children but applies to adults as well.”

Surviving in the Real World of a Day Care Center

More practically, the incident underscored why any droid dropped into the unpredictable environment of a day care center had better be able to adapt to the conditions at hand. “That was a big lesson,” Movellan said. “If Rubi was to do its job, it had to survive the children. It needed to respond to the real world.”

Funded by the National Science Foundation, the project, which began in 2004, aims to optimize the Rubi prototype for use in real-time, real-world educational environments and eventually develop a network of up to ten Rubi robots that can act as teachers’ assistants. The imperative is to develop a workable, socially interactive robot that is easy to replicate, a kind of “open-source” droid, in Movellan’s description.

Movellan’s long-term vision is to improve early-childhood education by enabling teachers to incorporate the robots in lesson plans to help children learn basic skills—not just the three Rs but socially appropriate behavior as well—and to open new windows for learning by digitally connecting classrooms across town or around the world. For example, Movellan is collaborating with a research group in Japan to develop a cross-cultural exchange program that lets children and teachers in California communicate with their counterparts in Kyoto via the computer screen built into Rubi’s belly. “Kind of like super-Skype,” said Movellan.

Its early dismemberment aside, Rubi not only survives in the preschool classroom, but enriches learning. Results presented in February at the annual meeting of the American Association for the Advancement of Society showed that Rubi teaches children vocabulary words effectively and efficiently, and even imparts a rudimentary knowledge of Spanish. Moreover, Rubi can exist in the classroom autonomously and positively affect social interactions, both between the children and the robot and among the children themselves.

A Face Steve Jobs Would Have Loved

The latest version of Rubi uses sophisticated software to recognize facial expressions and detect moods in others, then respond with digitally concocted smiles or frowns, accolades or tears. Rubi shows “emotions” through cartoonish animation on a face built around an iPad. Movellan worked with a team of computational scientists, including Dana Alliance member Terry Sejnowski, Ph.D., of the Salk Institute, to develop the software “brains” behind Rubi’s mood-detection capabilities. (For more, read a Q&A with Sejnowski.)

Rubi the Robot happy Rubi the Robot sad       The new faces of Rubi the robot (Photos courtesy of Javier Movellan.)

“The software we developed for recognizing facial expressions is very much based on what is known about early information processing in the brain, especially in area V1, the primary visual cortex,” said Movellan. The team applied mathematical models gleaned from studies of how neurons in area V1 detect and process images to create sophisticated computer simulations of neural processing. “We have built a neural network on the order of 10 million neurons dedicated to finding faces and recognizing facial expressions, all based on what we know from the neuroscience of the early visual cortex.”

To see Rubi’s range of facial expressions and movements, watch this video.

Brenda Patoine, science writer

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: