May 24, 2005
Toward Authentically Interactive Characters and Stories
Janet Murray asked for the answers I would have given to the questions I posed to Warren Spector, Neil Young and Tim Schafer at the recent GDC panel, Why Isn’t the Game Industry Making Interactive Stories? I found it useful for myself to write these out, to clarify my own thinking, and to hopefully get feedback from anyone interested.
I’ll try to be succinct and specific. These answers are informed by my experience over the past 13 years developing interactive characters and stories and closely following the industry and academic R&D in the field, helping me identify what I believe is important and what’s not. (Also I’m guessing these would be answers similar to what Michael would have said had he been given more time to participate in the actual panel discussion.) For some background on the panel, you may first want to read what the panelists said: 1 2 3.
Question 1: What do you consider the most important qualities and pleasures we *don’t* yet find in today’s interactive entertainment? And why are they needed?
Boiling it down, I see three major areas sorely deficient in today’s games, that if given substantial attention from game developers, e.g. 3+ focused years of R&D, I believe would lead to some true progress toward creating authentically interactive, much more satisfying characters and stories.
1. We need engineering techniques to manage multiple, various “chains of events”, that can intermix, that can be pursued in any order; ideally the game would be designed to allow the player to be the agent primarily in control of the order they are pursued, so as to react to what the player wants to make happen. These intermixable “chains of events” involve NPC’s, and can vary in scope and size. They can be small chains, lasting 10 seconds to a minute, such as short-term NPC behaviors to explore a room, to describe how they’re feeling, extended use of objects such as loading weapons or fixing drinks, to sob for a few moments, etc.; characters need to be able to do start these behaviors, get interrupted, come back to them, do multiple behaviors simultaneously, dealing with conflicts appropriately. … These event chains can be bigger in scope, such as conversations about specific topics, that last many minutes or longer; any one conversation needs to be able to go off in multiple directions; multiple conversations need to be able to be active, get interrupted, resumed, intelligently aborted if necessary, etc. Larger scope event chains include subplots, such as spending several minutes accomplishing a complex task such as driving across town and interrogating a suspect, or making dinner and proposing marriage, or digging through the ground to discover a secret tunnel and then uncovering treasure. The system needs to be able to keep track of and manage all of these event chains in a coherent, dramatically paced manner. These behaviors / conversations / subplots, while varying in scope, can have overlap in their underlying structure, and therefore can overlap in implementation technique. Their authoring needs to be a tractable production process, and avoid becoming a QA (testing) nightmare; testing becomes a major production issue as complexity of behavior increases.
2. Players need a far broader array of discourse acts that they can express — the things they can say and do, what Chris Crawford calls verbs. The defacto game controller interface, a joystick and a few buttons, isn’t expressive enough. (Requiring players to do ridiculously convoluted button combos to increase their options is just awful; I can’t believe players put up with that. Choosing from menus is a little better, but not much.) Natural language via a keyboard is probably the only viable interface for increasing the player’s expressive range; voice recognition in an emotional game situation isn’t here yet, and may not be for quite a while. The real technical challenge with increasing player expression via language is in making the system capable of understanding it all, and reacting in meaningful, context-sensitive ways — discourse management. Natural language understanding alone is a major technical challenge; pragmatic, robust solutions that are at least minimally capable will need to be developed. The required variety of reactions can be supported by implementing the intermixable chains of events above; more complex event chains go hand-in-hand with increased player expression and discourse management.
3. We need techniques for more expressive, procedural faces and bodies. The best existing game development, e.g. the Half-Life 2 character engine, is pushing in this direction somewhat, although it may not be as procedural yet as it needs to be for truly reactive and dynamic characters. While challenging to accomplish, of the three areas listed here it is the most well-understood and achievable in the short term.
Question 2: Why haven’t these qualities been achieved? What are the obstacles, how daunting? Are they so technically difficult that incremental progress is not possible? Are players happy enough with what they’ve got? Too great a divide between programmers and designers?
The primary reason these haven’t been accomplished is simple: these qualities are complex. It’s not because developers and players don’t want them, it’s because they’re complicated and onerous to implement, in terms of both design and technology. Game developers over the years have taken stabs at these qualities, but are forced to back off when the complexity begins to conflict with 12-to-18 month production cycles.
Unlike graphics and physics, the underlying “laws” of character behavior, narrative event chains and discourse management are not deeply understood or easily modelled. (Facial and body expression is more well understood, thanks to years of animation production; this book published in 1981 is one of the earlier bibles on the subject.)
When it comes down to it, these qualities are fundamentally about people, not about objects and environments. People are very complicated, of course — that’s why they’re so interesting! But although they’re complicated, implementing interesting virtual versions of them should not be intractable. It should be possible to make abstracted, dramatized, simplified but effective versions of people. This is at the heart of Chris Crawford’s new book, and why it’s worth reading. The Sims is a good beginning, but they’re too simple; to more deeply engage the player, virtual characters and stories need to become more personal, first person and directly interactive.
Ultimately we’re talking about creating characters, worlds and systems that have the flexibility and generative power of simulations, but are designed to temporally progress at a good pace, perhaps integrated with drama management, to keep the pace of the experience moving forward and avoiding play that devolves into repetitive, rote labor or long stretches of inaction — some of that dramatic compression Janet is talking about. One might call these “directed simulations”.
Sustained R&D and product experimentation is the required path to get there. In a future post I’m going to pitch an idea of how professional game developers and academic research groups could get together in a new way, to accomplish this kind of thing.
Are today’s players content? On the whole, no… I think many are running out of patience, something has got to happen. We’ve got this latest round of game consoles hitting the market over the next year or so, which will dazzle us with higher fidelity physical objects, environments, and shells of characters — nicely rendered minimally-interactive animatronic mannequins, and gorgeous, pea-brained monsters.
We’ve talked about the need to close the artist programmer divide.
Question 3: Give a realistic prescription for how to surmount these obstacles in the foreseeable future. Or, describe how this is not possible, or the wrong direction. Near-term technical and design milestones to shoot for? How can publishers become more experimental? Do we need to train a new generation of designer-programmers?
I’d wager this will require a big coordinated effort, not merely bits and pieces of progress here and there — like a mini-Human Genome project, or specifically, a Homme Virtuel project. Starting with, say the HL2 world engine, building the required AI from scratch would take 5+ years. Building on academic research results would be better, saving ~2 years of work, I’d guess. Recapping the conclusion of my Hard to Believe post from last December:
If someone said to me, Andrew, tell me what you need to put together a team to do this — a Manhattan project for believable characters — here is what I’d say I’d need, to create just one really, really good believable character:
* 3-4 talented, experienced behavior programmers — very hard to find
* 2-3 talented, experienced character animators
* 2-3 talented, experienced procedural animation programmers, with knowledge of existing techniques
* A behavior language such as ABL, or Zoesis’ tech, or something equivalent
* A producer to manage the team
* A creative director
* Office space
* 12 months to assemble the team
* 24 months of productionThis would probably cost about 3 million dollars.
Once the first character has been created, it would probably only take 25% of the original effort to make an additional character. And so on.
This would be difficult, relatively risky R&D.
Add in basic (not overly complex, but basic) natural language understanding and generation, and we’re talking 24-36 months of 3-4 talented, experienced engineers and writers, starting with the best of today’s NL technology, adding about another two mil to the budget. (The $5 million virtual man… we can build him… we have the technology…)
Realistically, could this happen? Sure. With the right team and the right business plan, I like to believe one could find investors interested in such a venture. (Existing game companies? Probably not.)
Over time, probably we’d see a cycle of try-and-fail attempts, hopefully that can build upon each other. Zoesis is most the most substantial first attempt I know of.
On designer-programmers — we’ve talked about the need to train a new generation of them.
Janet’s reverse question: What moments in people’s own gameplay have created intense story pleasure or have aroused story expectations and succeeded or failed in satisfying them in a memorable way?
I’ll just say that I’m hard to please in this area, with an allergic reaction to contrived interfaces and non-natural modes of interaction. I don’t have any personal success stories to report. When it comes to experiencing intense interactive story pleasure, I’m still a virgin. :-) Façade included, I’m afraid.
(Certainly as a kid I’ve had aroused story expectations, from the best C-64 games and from text-based IF, akin to what is described in chapter 16 of Richard Powers’ Plowing the Dark.)
—
I’m always interested to hear what specific, fundamental pleasurable qualities of stories you’d like to see in games and interactive entertainment, e.g. this from yesterday. Try to be specific, not overly hand-wavy if you can.
May 25th, 2005 at 3:37 pm
Natural language processing has been the goal of many an AI researcher for a decade or three, and has proven to be Really Damn Hard(tm). I think it’s still a worthwhile research goal, but isn’t it a bit insane to think that a single research team could solve that problem within 5 years, along with all of the other goals you’ve laid out? Or were you excluding that from your estimations?
Other than that, I’d love to see a research group take up these ideas and see where it leads. I just have a strong aversion to the all-or-nothing definition of the problem, when I don’t see a strong reason why your goals couldn’t be researched and applied individually to the current state of gaming.
(An example comes to mind, but I can’t remember the name of the group – but it was an academic research project which was creating a story engine within the context of a FPS with characters and events that adapted to the players’ actions, such as holding up a bank teller.)
May 25th, 2005 at 4:29 pm
Excellent points, now I can use all these reasons to convince my GF to let me buy an xbox 360. “But honey, this $400 is spent attempting to support the create a full-realized narrative structure. Now, I must kill it.”
>What moments in people’s own gameplay have created intense story >pleasure or have aroused story expectations and succeeded or failed >in satisfying them in a memorable way?
Without using a bit of overly-hyped video/sound effects, the storyline of Marathon was told through a computer terminal interface found throughout the game. Three seperate artificial intelligences with distinct personalities and writing styles gave different elements of the story. I recently played through a couple levels of it and was still astounded that a FPS had this complex a narrative.
The creators, Bungie, went on to create Halo, depending on who you talk to, was either a bust or the greatest development on FPS in a long time.
May 25th, 2005 at 4:43 pm
josh g,
Natural language processing has been the goal of many an AI researcher for a decade or three, and has proven to be Really Damn Hard™. I think it’s still a worthwhile research goal, but isn’t it a bit insane to think that a single research team could solve that problem within 5 years, along with all of the other goals you’ve laid out?
Natural language processing is a field of research with numerous successful results. It’s not the name of an unsolved problem that people have struggled fruitlessly toward for decades.
You’ll recall that Andrew (and Michael) have already solved the NLP problem for the specific case of Façade, although you may not have seen the system in action yet. I’ve been able to play a pre-release version, and the approach they’ve taken does work at least reasonably well for their purposes. Plenty of other specific applications (for instance, the speech interface at 1-800-USA-RAIL) work well enough, although of course, computers can’t understand all general human language, spoken and typed, in every context. Still, I’d trust Andrew to estimate the amount of research time you’d need for this.
May 25th, 2005 at 10:57 pm
I stand corrected, and sorry for the snarkiness. I am looking forward to seeing Façade in action, and I guess I haven’t seen enough good examples of recent successes.
I wasn’t thinking of NLP as a currently fruitless endeavor, just a difficult one to apply to very generic scenarios. The most successful examples I’ve seen only understand a very restricted domain of knowledge, which I was assuming makes things much easier. NLP as an interface to a system which is simulating not only world events, but NPC events with emotional content, seems like the scope of knowledge and language that the system would have to contain understanding of would be pretty large.
(I guess my unstated assumption is that a text interface has to be significantly more capable than, say, current IF parsers if this is going to break new ground as a revolutionary new direction in gaming. I don’t see either current gamers or casual-or-non-gamers putting down controllers and typing if they see “I only understood you as far as …” half the time.)
Anyway, I would be interested to know what approach was used in Façade, once it’s released. (Secretly, I wish I could do R&D work in NLP and games, and occasionally dream of experimenting with learning NLP systems applied to a game world. Don’t tell anyone though.)
May 26th, 2005 at 1:49 am
Josh, no worries, your skepticism is appropriate, especially in regards to the NLP goal. Note in my post I was careful to only suggest “minimally capable” and “basic (not overly complex, but basic)” NLP goals.
I am looking forward to seeing Façade in action, and I guess I haven’t seen enough good examples of recent successes.
Once it’s publicly released, you’ll have to judge for yourself how well Façade‘s NLP performs; Nick is giving it a glass-half-full evaluation, which is good of him. :-) We can say that anecdotal evidence from a decent amount of beta-testing suggests that our particular solution for our particular domain succeeds about 50% of the time, has partial success 25% of the time, and more or less fails 25% of the time. Over time we’re going to be doing some more formal user studies to get a better understanding of it.
Also, to date we’ve described our approach quite a bit, including both NLP and character behavior, in our existing collection of papers; however we’ve got a substantial new paper coming out next week at AIIDE, called “Structuring Content Within the Façade Interactive Drama Architecture”, which offers some new details on its authoring and implementation. We’ll put it online soon.
Finally, when Façade is released, we plan to offer a meaty “Behind the Façade” document that describes in relatively plain language the inner workings and design.
I don’t see either current gamers or casual-or-non-gamers putting down controllers and typing if they see “I only understood you as far as …” half the time.
Couldn’t agree more.
May 30th, 2005 at 3:09 am
Greetings:
For question 1, I think points 1 and 2 are inherently tied together. The issue here is the interactive feedback loop between the player and the virtual world. Point 2 is, in fact, primary, because the number and types of reactions and management options you need are directly proportional to the range of discursive acts the player can perform.
The question of interface here is not trivial. It is not sufficient to provide the player with the appropriate range of options, but also to ensure that the mechanism for making those choices is as intuitive (if you want to reach a large audience) and as transparent as possible. If it takes the player significantly longer to express their desired action than it does to reach that conclusion, you’re going to lose the audience.
This is where we come back to the NLP problem, because it’s not just a case of coming up with a sufficiently delimited subset of natural language, but rather of handling effectively the full range of natural language the player is going to attempt to use within the game experience. You can train the player to some extent (provided you can keep their interest long enough) and you can use some shortcuts, but the premise of generating player involvement through this interface is going to be violated by the extent to which you cut off their instinctive reactions and force them to work within (what will seem to them to be) arbitrary limits.
Beyond that, you have to look at whether this interface feedback loop and the timing of dramatic progress is going to compare favorably to the loops of more clearly game-focused games like Diablo II, the Sims, or Medal of Honor. Take a look at the timing on these games, and you’ll see that the timing of that loop is very, very short. The training time on the interface is very quick, the use of the interface is very quick, and the reward situations for effective use of the interface not only happen quickly, they cycle quickly.
Even assuming that you could get the NLP down and the management of event chains to create believable character-driven narrative sequences, would the average player get the sense of mastery, control, and reinforcement on anything like the schedule of these games? This is one of those points of disconnect, I think, between the academy and the industry. Industry folks look at this problem and see the player experience at the functional level as more important; academic folks look at this problem and see the content as more important. If you want to compete in the marketplace, I think this is one of those issues that needs to be addressed.
On point 3, you guys should look at BioWare. While the Valve guys did a good job of hyping their character technology, BioWare has been taking this stuff seriously for years, building from project to project, and working to keep an internal knowledge base active and evolving from both the content and technology angles. Not only are they closer in focus to what you’re advocating, they’re also using a methodology that’s fairly similar to what you’re proposing.
On question 2, I’ve said it too many times to be worth more than simply repeating: most players are happy enough with what they’re getting and the underlying technical problems are vast.
On question 3, I think you’re tremendously underestimating the challenges. Your model also doesn’t include producing actual content once the technical, design, and writing challenges have been solved. There’s nothing in there for mise-en-scene.
Ultimately, I agree with Warren Spector’s points at the talk, that as noble as the project may be, it’s fundamentally misguided and impractical to look at this as being a viable approach to achieving a commercial success in the games market. It’s an interesting academic exercise, and it may someday flourish in a micro-community, but it’s not what the gaming market on the whole is looking for.
I know you don’t believe me, that you can’t acknowledge the truth of this and still pursue your projects, but I pose this simple challenge: look at all the post-E3 coverage and find me even a handful of articles from outside the academy that have anyone complaining about the lack of character or the need for more interactive fiction.
Still tilting at windmills,
Eyejinx.
May 30th, 2005 at 8:06 am
Eyejinx, I must say that I am often incapable of handling effectively the full range of natural language other people are attempting to use within my life experience. Although my abilities of responding are limited, and those limits can doubtlessly be seen as arbitrary by others, I’ve never had the problem of not being accepted as a believable interactive character. I believe that this is because I’m aware of my limits, and can communicate this awareness. I don’t think that the fact that virtual interactive characters cannot simulate this behavior yet is a reason to conclude that it can’t be done.
One of the very nice features of implementing language games is that the function space and the content space are nearly identical – at times, I even have to introduce artificial differences to draw a boundary between computer language and natural language, so that the character can seperate what she “says” from what she “thinks” -, and the question of which is more important doesn’t arise here.
How many players are happy with how things are doesn’t concern me either, as long as I’m not happy. For a long time, most people seemed to be quite happy with the idea that they live in a flat world that is circled by the sun.
And technical challenges are there to be solved. Hardly anybody believed that motorized flight was possible, either.
May 30th, 2005 at 12:42 pm
Eyejinx, this is exactly the kind of critical feedback I was hoping for. I hope to have time to reply in the next day or two.
May 30th, 2005 at 5:30 pm
I know you don’t believe me, that you can’t acknowledge the truth of this and still pursue your projects, but I pose this simple challenge: look at all the post-E3 coverage and find me even a handful of articles from outside the academy that have anyone complaining about the lack of character or the need for more interactive fiction.
Doug Lowenstein, president of the Entertainment Software Association, gave this state of the industry speech at E3, in which he said:
We need games with better stories, more interesting and complex characters, games that keep you up in the middle of the night wrestling with whether you made the right ethical or moral choices, games that stay with you when you’re done with them, games that make you happy when you play them, and afterwards.
Many of today’s games are amazing creative and artistic accomplishments, and many are incredibly fun and absorbing. But we have only scratched the surface of what games can be. We cannot let the lure of onrushing technology blind us to the essence of what makes great entertainment. Great entertainment – whether books, film, or games, must engage us at some emotional level. If video games aspire to movie-like status, then games need to become topics of conversation at dinner parties and happy hours and they won’t ever achieve that if they are mainly the province of an elite few who speak their own language, congregate in chat rooms and LAN centers, and have an endless amount of time on their hands.
So there are at least some non-academic folk interested in deeper characters and stories.
Even assuming that you could get the NLP down and the management of event chains to create believable character-driven narrative sequences, would the average player get the sense of mastery, control, and reinforcement on anything like the schedule of these games?
This is a good point; progressions, by their nature, require both the passage of time and the player remembering their interaction history in order for the player to develop a model of how their interaction effects the progression, and thus to develop a sense of mastery and control (agency). In Facade, we attempted to design for both local agency (being able to provoke immediate responses from the characters) and global agency (having a sense of mastery and control over longer-term progressions). Developing this understanding definitely takes mutiple playthroughs and experimentation on the part of the player (since any one playthrough is only ~20 minutes, it’s not onerous to play Facade multiple times, and in fact is hopefully enjoyable). The character dialog tries to communicate the “score” (underlying changes in character emotions and relationships that drive the progressions forward), though we were not as successfull at this as we would like. Creating global agency, that is, providing reinforcement, control and mastery over longer-term progressions, is definitely one of the key design challenges.
May 30th, 2005 at 5:42 pm
Oh, and regarding Bioware, which Bioware games do a good job providing procedural faces and bodies? I’m only really familier with Never Winter Nights, which, though I find it a great modding environment, I don’t find powerful from the point of view of procedural bodies and faces.
In my Interactive Narrative class this year, students built story-based NWN mods in which objects, actions and places (levels) take on more symbolic meaning. Since the NWN authoring tools are really designed to create dungeon crawls, it’s an interesting exercise in working against the affordances of the tools. Danny Mueller, a master’s student who worked with me a couple of years ago, and who has a background in screenwriting, built an NWN mod based on one of his screenplays as a way to investigate catharsis in games, and particularly, as a way to investigate mise-en-scene in level design.
May 30th, 2005 at 6:43 pm
Knights of the Old Republic features close-up shots of the characters during any conversation. I don’t know how procedural the expressions were, or if it was selecting from a static pool of choices, but the NPCs were very well done in KotOR. (Both visually, and voice acting. KotOR 1 is worth playing just to experience the droid NPC HK-47, really.)
I haven’t seen Jade Empire yet, but if it uses a similar visual style during conversations, that would be a good place to see their current state-of-the-art.
June 1st, 2005 at 12:35 pm
Another great thread Andrew, thanks.
Scott Miller of 3DRealms just updated with a piece from Bob Bates talking about the hero’s journey. I like this bit:
“Carl Jung argued that mythical motifs are structural elements of the psyche. In fact he went further to argue that there are patterns that are biologically present in our brains. He gave these patterns the name “the collective unconscious.” Just as Pinker believes humans have the capacity for grammar wired into our brains, waiting for a particular language to come along and imprint itself upon us, Jung believed we have mythic structures built into us, waiting for a particular belief system to be imprinted upon us by the culture we grow up in.”
Sooo,
If we approach the creation of believable characters with this in mind, where they all have these ‘myth templates’ present in their ‘minds’, I could see stories popping out that fit the heroes journey. Each character, from their own point of view, can take the role of a hero, acting out their own story, and acting in supporting roles in the stories of other characters, one or more of which could presumably be controlled to some degree by a player.
I’d like to elaborate, but I’d prefer to hear anything you guys would like to say about this type of approach.
June 1st, 2005 at 3:07 pm
You’re right, Chris, many writers find Jungian archetypes useful. I wouldn’t want to commit to mythical motifs as “structural elements of the psyche”; as long as I can figure out the structural elements of the story, that’s enough for me :-). Dramatica uses an array of eight archetypes that I find useful as a jumping-off point. Fanfilm producer a href=”http://www.apocprod.com/”>Michael F. Haspil has compiled a good reading list on the “Hero’s Journey theme.
However, it should be noted that the currently available literature doesn’t deal with interactivity. My opinion is that I’ll have to give the client/player access to all the Character Elements the system uses, no matter whether I use Archetypes or Complex Characters. Archetypes can be useful for categorizing player behavior, but I wouldn’t want to tie the player to having to use certain archetypes to communicate in the story, at least not in the early stages of the interaction, where the player is likely to test any behavior that comes to mind.
June 3rd, 2005 at 11:27 am
A lot of the stuff that’s being asked for above is already possible in massively multiplayer online games. MMOGs could benefit from closer attention to story and long-range character development (along the lines of Bob Bates’s excellent essay), but a lot of the tools described up above are already in place in those worlds.
What moments in people’s own gameplay have created intense story pleasure or have aroused story expectations and succeeded or failed in satisfying them in a memorable way? My answer here
But in short, there’s a very good reason why the game industry isn’t making interactive stories: they’re too busy making games. As has been pointed out, there are people who want more story, better interaction, cooler character arcs. But there aren’t many who want “intermixable chains of events.” If you want to get people fired up about this stuff (and it’s well worth getting fired up about), tell them what they’re going to get out of the experience, then work back from there. They’re going to get to live the story, be the character, experience the highs and lows of the hero’s journey. They’re going to be faced with important choices at every turn, they’ll be able to see their character express what they themselves are feeling, they’ll identify with their onscreen self like never before as they first learn about their world and its dilemmas, venture forth to slay the monsters and solve the problems in their path, and finally struggle with the long journey home, always wondering whether there’s not something more satisfying that still lurks out in the wilderness for them to conquer.
I’m buying (or investing my $3 million) in that game. I’m not buying the game that offers me “intermixable chains of events.” It may seem like a snarky quibble, but I think it’s important to keep the end product in mind. The best games (the best examples of any art form, in fact) arise out of the creator’s passion, which stirs some passion in the gamer, the reader, or whoever’s on the other end of the experience. People react to passion, and making a great game that incorporates the very good ideas laid out above just isn’t going to be possible without it. Gamers are very passionate about their games, and if you really want to move these ideas forward, you’re going to have inspire the same kind of passion they already feel.
June 3rd, 2005 at 3:05 pm
That was cool. Thanks for saying that, Mark.
June 14th, 2005 at 1:41 pm
GTxA is having trouble with automatic trackbacks – so I’ll manually add these links (discovered via new features in our update to WordPress 1.5)
From Jeff On Games: Toward Interactive Stories
From Walkerings: Primarily Out of Control (expanded version of Mark W.’s comment above)
March 14th, 2006 at 4:01 pm
[…] erring to himself as a narratologist ;-) plus ex-developer Mark Barrett. Wallace (who has commented here at GTxA, skeptic […]
April 1st, 2006 at 5:20 am
[…] between game scholars. An axe I often grind is the need for what I’ve called “authentically interactive” characters a […]
February 27th, 2007 at 7:27 pm
[…] to his next game. Admittedly we’re among those garage developers that believe that more-than-evolutionary steps may be required […]