Ellie is a computer simulation designed to engage real people, like the woman on the left, in meaningful conversation and take their measure. The computer system looks for subtle patterns in body language and vocal inflections that might be clues to underlying depression or other emotional distress.
Her hair is brown and tied back into a professional-looking ponytail. She wears a blue shirt, tan sweater and delicate gold chain. It’s the first time she has met the man sitting across from her, and she looks out at him, her eyes curious.
“So how are you doing today?” she asks cautiously, trying to build rapport.
“I’m doing well,” he answers. His eyes blink.
“That’s good,” she continues. “Where are you from originally?”
“I’m from L.A.,” he tells her, and this makes her smile slightly.
“Oh!” she says with surprise in her voice. “I’m from L.A. myself!”
She is from L.A. She was created in Los Angeles and “lives out her life” there on a computer screen in a lab at the University of Southern California. She’s not a real woman but a virtual one, created to talk to people who are struggling emotionally, and to take their measure in a way no human can. Her makers believe that her ability to do this will ultimately revolutionize the way mental health care is practiced in this country. Her name is Ellie.
There’s Power In A Well-Timed ‘Uh-Huh’
The project that resulted in Ellie began almost two years ago at USC’s Institute for Creative Technologies. Two scientists in particular are responsible for her existence: psychologist Albert “Skip” Rizzo and computer scientist Louis-Philippe Morency.
Rizzo and Morency spent months laboring over every element of Ellie’s presentation and interaction with patients, experimenting with a range of different personalities, outfits and vocal mannerisms.
“Everything has been thought of,” says Morency. For example, when patients talk, Ellie encourages them to continue talking with a well timed “uh-huh,” just as real people do.
“We have recorded more than 200 of these uh-huhs,” Morency says, “and these are so powerful. Because a simple ‘uh-huh’ and a silence — if they are done the right way — can be extremely powerful. So we spent a lot of time on these little details.”
But the most important thing about Ellie is not her skill at gently probing all of the people her scientist brings into the lab to talk to her. Her real value, the reason she was built at all, is her skill at taking and analyzing thousands of measurements of those people.
Under the wide screen where Ellie’s image sits, there are three devices. A video camera tracks facial expressions of the person sitting opposite. A movement sensor — Microsoft Kinect — tracks the person’s gestures, fidgeting and other movements. A microphone records every inflection and tone in his or her voice. The point, Rizzo explains, is to analyze in almost microscopic detail the way people talk and move — to read their body language.
“We can look at the position of the head, the eye gaze,” Rizzo says. Does the head tilt? Does it lean forward? Is it static and fixed?” In fact, Ellie tracks and analyzes around 60 different features — various body and facial movements, and different aspects of the voice.
Read More: Here