December 9, 2003

The Whoa Effect

by Andrew Stern · , 8:49 am

Last Saturday, from the bedroom window of our third story apartment, as we were admiring the results of the previous night’s snowstorm, the doorbell rang. Our new couch was due to be delivered that morning, but we thought, surely they would cancel. It seems we underestimated Boston furniture delivery truck drivers; minutes later I was relaxing in the living room on the new sofa with the December 1 issue of The New Yorker. After a brief nap to recover from my interrupted sleep earlier that morning — I always wake up before sunrise when I have lots of new events to process — I came across two articles, different but connected, that made me sit up and think.

The first one was predictably interesting — a John Seabrook article profiling Stan Winston, whose Hollywood special effects studio created numerous animatronic characters over the last few decades for movies such as Aliens, Terminator and Jurassic Park. His studio also built Teddy, the talking and walking “supertoy” in the recent movie A.I., which at times required five or six puppeteers. (Funny, just like with the dinosaurs in Jurassic Park, I had assumed all of the shots of Teddy were computer-generated.) The article goes on to describe how M.I.T. researcher Cynthia Breazeal, creator of the autonomous robotic face Kismet (who we briefly talked about in our recent interactive drama discussion), was called in during the movie’s production to consult on robotic technology, which led her to propose a collaboration with Stan Winston to build “a real Teddy, a Teddy with a brain”. Now they’re currently building a Gremlins / Ewok-like animatronic robot named Leonardo, that according to Seabrook is to be “the most life-like mechanical creature ever built, a state-of-the-art emotional machine.” Breazeal’s long term goal is to build “a sociable robot, something that lives with you and that you can have a meaningful emotional interaction with.”

No surprise, I’m fascinated by this project, and have similar goals myself, although I’m more interested in virtual on-screen characters, versus robotic characters. As impressed as I am by the ambition of Leonardo, and if forced to predict would guess that physical characters, in the long run, will create stronger bonds with people than screen-based ones, I have to wonder about the short run… It just seems so much easier to create screen-based virtual characters than robots. And as a product, software is far less expensive and more distributable than hardware. … But maybe I’m overestimating how compelling of an experience on-screen characters can offer, and underestimating physical ones? It’s true that physical characters such as Tamagotchi, Furby and AIBO often outperform (either in sales figures or press attention) virtual characters of equal or greater sophistication, such as Petz, Babyz, Creatures and Seaman.

(An aside — I briefly met Cynthia a year ago this month when I gave an invited talk at Sherry Turkle’s Technology and Self group, who do research on “relational objects”, such as virtual pets and the like. My informal presentation, “The Significant Other”, was about the idea of virtual friends as products. After seeing the early Facade demo video, Cynthia was surprised and disapproving that we were allowing our on-screen characters to get angry and upset with the Player.)

Towards the end of the Winston / Breazeal article, Marvin Minsky is quoted as saying, “My objection to Leonardo is, it’s just a trick. It doesn’t really have emotions. It just knows how to fool you into thinking it does. Cynthia’s an excellent engineer, but her work doesn’t explain how emotions work. Leonardo is just an improved version of that software wizard, F1, that Microsoft tried to get people to buy a few years ago. People went, ‘Oh, gee, that’s neat,’ for a couple of days, and then they got tired of it.” When Winston heard Minsky’s views he cried, “Shame on Marvin Minsky!” Breazeal was surprised too, saying, “Well, I don’t see Leonardo’s emotions as being a trick. … We’re not trying to capture the human-feeling side of emotions, but we are trying to capture the pragmatic side — communicating with others and behaving more intelligently.”

(Another aside, hopefully Michael won’t mind me telling it — when Michael was a grad student, at a group lunch he briefly chatted with Daniel Dennett, explaining to him the concept of believable agents, about applying AI techniques not for rational problem-solving but for creating compelling emotional and personality-rich characters, and Dennett’s only reply was, “Well, that Microsoft Bob sure sucked.”) (About Minsky, the funny thing is in recent years he gave at least one presentation to the game development community, so I thought he was more on-board with applications of AI for interactive entertainment; or maybe he is, just not too close to home.)

Paging to the end of the New Yorker issue, I read an article by Louis Menand, subtitled “The art of short fiction”, which reviewed of a new collection of John Updike’s first 103 or so short stories. Menand, relating Poe, describes the “effect” that short stories in general try to achieve: with as much efficiency as possible, to create “an outcome that is both startling and anticipated … what you might call (if you were a college student) a general sense of ‘Whoa’ … James Joyce called the effect an ‘epiphany’ … ‘the revelation of the whatness of a thing’ — a sudden apprehension of the way the world unmediatedly is.”

At the risk of comparing digital interactive forms to non-digital non-interactive ones, I wonder what kind of effect, if any, will end up being common among short interactive experiences. By short I mean an experience that lasts anywhere from say, five to twenty minutes, i.e. not your typical epic computer game or console game. Tetris or any other “small” game counts, and of course many works of ht lit and IF.

Would the “whoa”-epiphany effect described by Menand belong to those interactive works with a greater sense of closure? Some interactive forms, such as hypertext literature, as well as open-ended simulations, often do not offer the player closure. Do the proponents of hypertext literature think about this “whoa” effect, or perhaps even explicitly reject it? Or some other powerful but different effect? Of course with ht lit one can achieve those effects if one wants, but can it be done without closure?

In the format of interactive experience that I focus on — real-time interactive characters, whether as virtual pets, friends, or co-protagonists in short dramatic scenarios — a primary, fundamental effect I’ve been trying to achieve to date could be boiled down to a sense of presence, of aliveness. Whether or not the quality of Facade’s narrative succeeds or fails, I hope players come to feel that the characters have a mind and will of their own, are aware, and interact robustly. Note, just as Breazeal and Winston state, this does not necessarily mean truly intelligent.

When a real sense of presence is achieved, I believe it can create its own particular effect of “whoa” in the player. That said, at this point in the evolution of such works, I think we need to go beyond mere aliveness, and create experiences of this sort that offer players some real, meaningful agency. (And as I suggested the other day, some degree of true intelligence will probably be required to do so.) Delivering on that would really take that “whoa” feeling to the next level. I’m interested to think of ways to do that in shorter formats, versus the longer formats of today’s games. Perhaps if the effect is strong enough, even if only short in duration, people will pay money for it (say, $20). Or even better, they’ll pay money for it because it delivers such an effect in a short experience; after all, who has the time to play long games?

Creative Commons License The text of this blog entry is licensed under a Creative Commons License.