Skip to content

Heather Knight and Social Robotics: The Intersection of Storytelling and Technology

February 11, 2011

Heather Knight and a Robot Performer (credit: Kris Krug)

Heather Knight is a roboticist and PhD student at the Carnegie Mellon Robotics Institute who is breaking new ground in the field of “Social Robotics”. Along the way she’s earned undergraduate and graduate degrees at MIT (in Electrical Engineering and Computer Science) where she worked on Cynthia Breazeal‘s Huggable project (a new type of robotic companion being developed at the MIT Media Lab for healthcare, education, and social communication applications), founded a robot theater company (Marilyn Monrobot), worked in NASA’s Jet Propulsion Laboratory and even helped create OK Go’s award-winning YouTube sensation “This Too Shall Pass” (the Rube Goldberg machine version). [UPDATE: Heather was named to Forbes Top 30 Under 30 in the Science category: read more…] recently had the privilege of talking to Ms. Knight about the world of social robots and how her interest in technology and creative arts have intersected to create something new. When did you first get interested in technology?

Heather Knight: My first real enrapture with technology was on a high school science team. I remember one event where you had to practice working together to build a structure using whatever materials were provided on competition day, be it marshmallows, Legos, or sticks. To make it more challenging, one team member had to create a written description of how to build the object with the materials provided, while your partner waited in another room. Your partner then had to re-create the object using just the description. It’s amazing how easy it is to miscommunicate. I always liked making things with my hands and became interested in robots later, in college. When did you decide to pursue engineering in college and ultimately post graduate degrees?

Knight: I was trying to decide whether to become a writer or an engineer, because I always liked storytelling, I was a big reader. I didn’t really know what engineering was yet, but my dad was an engineer and he was really cool.

I took a year off during college to travel across Europe and when I came back I switched majors from mechanical to electrical engineering – exploring the world and different cultures has been such an important part of my life. Between my undergraduate and masters I worked in Paris for a robotics company for a little while, and took some acting classes just for fun. That was my first exposure to entertainment, but I realized I was really terrible at remembering lines, so now I leave the acting to my robots. What are some of the more interesting aspects of seeing your robots in action?

Knight: One of the first things I observed is how open people are to robots – the whole room comes alive when you have this physical object on stage – people connect with robots on a plane that is very distinct from how they connect with a two dimensional computer screen. Animated machines bring the experience into the physical world.

There are many exciting applications in the world of social robots, especially where robots and people are working together. For example, in a medical situation you could have a robot that acts as a companion, helping the person feel entertained or comfortable and reminding  them when to take their medicine, exercise, or just making exercise more fun. At this point the robot should not be responsible for the more serious aspects of care, but rather they act as an assistant making the job of the doctor more effective. I like the idea of people using robots to connect with other people. What are some of the key things you’ve learned in your social robot experiments, like the demonstration you gave at TED?

Knight: The TED demonstration was an early and exciting experiment that has opened a lot of doors. It showed that it is possible to personalize the robot experience with a little bit of help from humans. Next, I want to explore not only more complex audience sensing, but the robot’s expressions – different ways of looking at body language, the the interplay of personalities, character, credibility.

These sort of research questions are in great demand for the design of robots in our everyday lives. The algorithms we develop could make the difference between whether complex consumer robots become commercially viable. Do you want to work with the robot – do you like it?  What is the role of personality? Think about the ways in which you choose to hire a babysitter or have a gardener come back.  Social intelligence, dedication and personality will be key factors in the decision, and will also change what kind of applications for which it will be possible to use robots. How far away are we from robots truly understanding what we are trying to say? What we mean?

Knight: Very far away, but that’s the magic thing of working in this arena. First of all, it’s completely new, there’s a handful of people that have been doing work in this area for the past ten years, but that means it’s only just getting started. The second reason why the field is great is that you get a better understanding and appreciation of the complexity of being human, and how special it is that we can do what we do, how we are. In the case of computer vision – being able to detect objects within an image – people started working on that problem in the 1950′s. Their estimation of the complexity was so understated that two grad students were given a summer to work out a list of problems with computer vision. Turns out some of the items on that listare still being worked on today, now sixty years later. Obviously it points to the fact that we didn’t appreciate the complexity. A lot of the things that we do really naturally, like seeing a moving object, none of those things are obvious to a machine. In the process of trying to reverse engineer even really simple aspects of our behavior, we end up learning a tremendous amount about ourselves.

And I think social behavior is even more complex than vision recognition. Is social robotics a challenge of processing power or do we need to think about the problem in a different way?

Knight: It’s a completely new thing – we don’t know. One community that is trying to reverse engineer social behavior is researchers that work with autism and social disabilities. The people that coach children on how to integrate back into society, or coach people that are unable to detect emotions, social queues, to come up with coping strategies. These researchers have to think about how social behavior is structured – and come up with rules.

A second community is actors – actors think about how to structure a character, how to structure an interaction. When an actor reads a script they don’t wait until the end of someone’s line before reacting. Your emotions start reacting before you start saying your line. For example, if someone says, “You’re a horrible person you did this and this…”, you’re not going to smile serenely until the end of what they’re saying, rather, your entire physiology changes while you are listening. What should middle and high school students do to explore an interest in technology?

Knight: One thing is to start taking things apart. It’s amazing what you can learn from something you like playing with by completely destroying it! Anything that’s hands on, that’s creative, can help you. One example is the Maker community.

But I still believe that storytelling, integrating technology and storytelling, can be extremely powerful. There’s a technology project at Carnegie Mellon called Alice where they trick people into learning computer programming by getting people to tell a story using programming tools. To get the story into the system you need to use these tools. I think that’s clever. What is next for you?

Knight: This summer I’m going to do a Robot Film Festival for the first time. People will be able to submit short films that making stories using real or fictional robots. These kinds of side projects are always exciting. [UPDATE: Fall 2011 – Ms. Knight did coordinate a Robot Film Festival:]. Last fall I did a Robot Census at Carnegie Mellon, which was a pretty interesting exploration. We made a form that was a parody of the US Census form, and collected some really interesting information. In the end, we recorded 547 robots on campus. One of them, a robo-receptionist named Tank, greets visitors at the Robotics Institute every morning.

Related articles: