Grand Text Auto

March 17, 2008

EP 8.5: Façade

by Noah Wardrip-Fruin · , 6:34 am

I first met Andrew Stern and Michael Mateas at a 1999 symposium on “Narrative Intelligence” sponsored by the Association for the Advancement of Artificial Intelligence. The symposium was organized by Mateas and Phoebe Sengers, two of the final Oz PhD students. They managed to bring together a number of their mentors, colleagues, and friends with a wide range of people pursuing different facets of the intersection of narrative, character, and AI. The Zoesis team was present, showing off their most advanced demo: The Penguin Who Wouldn’t Swim. Bringsjord and Ferrucci discussed active development of Brutus. Stern described his company’s newest commercial product based on believable agent work: Babyz. Mateas and his collaborators premiered Terminal Time. It felt like the field was blossoming with new projects, pushing the state of the art to new levels.

But, at the same time, Mateas and Stern were pushing each other, behind the scenes, to take even bigger risks in the project they had begun to plan. They wanted to create the first real example, as uncompromising as possible, of the dream of interactive drama. As Stern would later say about the resulting project, “when you shoot for the stars, you might hit the moon; with Façade, we think we got into orbit. . . . we enjoyed being overly ambitious” (2005).

Façade (2005) begins with the player approaching the apartment door of Grace and Trip — old college friends, now married — who can be heard in a muffled argument as the player decides whether to knock. Over roughly 20 minutes of interaction, the promise of this first moment grows from tense chitchat to full-blown meltdown, and usually the dissolution of Grace and Trip’s marriage. The shape of the crisis is determined by the actions of the player, which can also (less commonly) lead Grace and Trip to a new level of mutual understanding, or result in the player’s ejection from the room.

All of this takes place in a simulated apartment, with the characters speaking dialogue and performing actions and emotions. The player can move around freely, picking up items, typing arbitrary text, hugging and kissing, and being generally invited to produce her own performance. Trip and Grace respond dynamically, following up on topics raised by the player, commenting on patterns of player statements and actions, perhaps reestablishing their own interrupted lines of conversation, treating the player differently as she seems to take sides with one or another of them, and so on. It is, in other words, utterly unlike the experience offered by other contemporary digital fictions — nearly as far from a standard dialogue tree as it is from an email novel. Façade doesn’t always work, but more often than not it does, and with this it demonstrated that the dream of interactive drama could be made a reality.

Façade also does more than provide an existence proof of interactive drama and respond to the problem (one of digital media’s “grand challenges”) posed by Brenda Laurel two decades before. Façade also illustrates the consequences of a group of choices relative the issues raised in this book. By doing this it clarifies a future agenda for research, sheds new light on the central concept of “agency,” and simultaneously demonstrates that a set of techniques are ready for use in authoring.

Façade beats

Work on Façade sent Stern and Mateas flying straight at a number of unsolved research questions, from natural-language understanding (so that players can type any text they wish) to believable behavior for non-player characters. But the key to their approach was the re-introduction of the concept of drama management. Their approach to this problem, in turn, departed from the bus station’s plot graph — instead pivoting on a reinterpretation of a concept from dramatic writing: the “beat.”

The typical conception of a beat is provided by Robert McKee: “an exchange of behavior in action/reaction. . . . these changing behaviors shape the turning of a scene” (1997, 37). Mateas and Stern cite a famous beat from Casablanca: “Rick: ‘Why’d you come back? To tell me why you ran out on me at the railway station?’ Ilsa: ‘Yes’ ” (2007, 191). But, as they explain, beats in Façade are much larger and more complex, in effect operating as miniature scene structures embedded within the larger drama, built to be intermixed with other elements.

A primary goal of the beats Stern and Mateas created for Façade is to embroil the player in a series of what they call “social games.” The first section of the drama has Trip and Grace setting up zero-sum affinity games, in which the player is invited to agree with one of them (and disagree with the other). For example, Grace may initiate a conversation about their redecorating, which she thinks is problematic but Trip likes (figure 1.7). Similarly, Trip may begin a conversation about their recent trip to Italy, which he thought of as a second honeymoon and Grace saw only as a weekend getaway. Even the harmless-seeming activity of choosing drinks has an associated disagreement beat, with Trip urging the player to join him in a fancy mixed drink while Grace pushes for something simple. Simultaneously, throughout these affinity games, Grace and Trip respond to player utterances in a series of hot-button games around topics such as sex, divorce, family background, and emotionally-laden objects around the apartment — which also serve to reveal more information about the characters and their stories.

Figure 1.7: In Façade, the player disagrees with Grace about the decorating (via typing text) while joining Trip in a fancy drink.

In the second section of the drama the beats are crafted to produce a therapy game, which has the potential to increase character realization. Finally, the drama comes to a dramatic conclusion. Throughout, the drama manager selects beats based on a desired curve (and believed current state) of the tension level, attempting to produce a neo-Aristotelian experience of slowly-rising tension leading to catharsis. Producing this experience requires a rather different approach to character behavior than seen in the Sims or Oz.

Façade characters

Like Sengers, Mateas and Stern were strongly influenced by the Hap conception of character behavior. Façade’s tool for authoring such behaviors, ABL, is based on Hap. But, at the same time, Façade’s authors took the auto-critique of Hap embodied in The Expressivator further.

Mateas had been convinced of the need for transition behaviors and meta-level controls by his study of Oz work and discussions with Sengers, while Stern had seen it for himself while working on PF Magic’s “Petz.” The first of the Petz projects, Dogz (Resner, 1995), was a computer-based system for building relationships with virtual animals that arrived before Bandai’s keychain Tamagotchi (1996) and Tiger’s plush Furby (1998) — and long before Nintendo’s Nintendogs (2005). The original AI, created by Ben Resner, was of a traditional finite-state machine design. When Stern took over the AI direction, starting with building the Catz (Stern, 1996) system in early 1996, he was working under the direct influence of the Oz Project’s technical papers. By Petz 3 (Stern, Frank, Resner, Fulop, Harrington, and Shambroom, 1998) and Babyz (Stern, Frank, Resner, and Fulop, 1999) Stern’s architecture of goals, behaviors, and emotional states was relatively sophisticated, and included a type of transition behavior with aims similar to those Sengers pursued simultaneously with The Expressivator. As Stern puts it:

Well, the concept of transitions between behaviors is an inherent requirement for fluidity and lifelikeness for any character, I think. There were Petz/Babyz versions of those transitions — little behaviors like “looking around for what do next” or “looking to the player for reassurance, for a pet/tickle” after finishing one behavior (e.g. eating) and going on to another behavior (e.g. playing). There were little ad hoc behaviors like that scattered throughout Petz and Babyz. (2007)

Figure 1.8: PF Magic’s Petz and Babyz used procedural animation (along the lines of Improv) and a behavior-based character architecture (along the lines of the Oz Project) to create digital media entertainment about characters and relationships, rather than physical dexterity or puzzle solving.

The Expressivator, as a set of extensions to Hap, had demonstrated an approach to building an architecture for such transition behaviors — something more principled than an ad hoc collection. In building ABL for Façade, Mateas and Stern took this another step, building transition behaviors, and meta-level controls more generally, into the core of the system itself. But Façade’s beat-oriented authoring also required a deeper change. While Hap, The Expressivator, and the Petz projects all treated goals and behaviors as aspects of individual, independent characters, Façade’s characters can share goals and behaviors.

This was a surprisingly radical notion in the domain of believable characters.⁹ Woggles and Petz played together, and the Industrial Graveyard’s lamp and overseer interacted more grimly — but always by “sensing” and “reacting” to one another. The authors of each system had to work out software solutions to the problems of joint behavior, all hobbled by the AI assumption that they were creating entities that needed to act independently. But, in fact, each author was creating a system for performance of behaviors, whether for one character or more, a fact that became inescapable as Façade’s design called for fast-paced exchanges of dialogue (rather than looser interactions such as the Woggles’ “follow the leader” or Babyz throwing toys at each other). One can see this clearly by looking at Façade’s script.

Façade’s script

Façade’s characters perform together based on what Stern and Mateas call “joint dialogue behaviors” (JDBs). Each beat is a structured collection of 10 to 100 JDBs. Each JDB, in turn, is set of one to five lines of dialogue between Grace and Trip, which consists not simply of recordings of the words but also 50 to 200 lines of ABL code. This code supports the various ways the JDB can be performed as well as the ways that JDB alters Façade’s internal counters for the affinity game, character self-realization, overall tension, or other elements of story and character state. Part of the reason that beats contain so many JDBs is related to this. Most JDBs are actually authored with three to five alternate versions, with multiple takes of the recordings for each, so that an appropriate one can be selected based on current measures of affinity and tension (and repetitions between playthroughs can be lessened).

Another reason that beats contain a large number of JDBs is in order to support special meta-level controls (called “handlers”) that provide appropriate reactions to player statements and actions in the context of the beat. Each beat has a default way it will play out in the absence of interaction, but Mateas and Stern are hoping for another outcome — given that the point of Façade is to pursue interactive drama.

So, for example, the JDBs in the beat about fixing drinks include a basic sequence. This sequence is expressed as four joint goals, during which both the speaking and non-speaking character perform appropriate behaviors. These beat goals are: Trip bringing up the topic of drinks (a transition in to the beat), Trip making a drink suggestion (while bragging), Grace making a counter-suggestion (and mocking the behavior of Trip, who reacts), and a final response by each of them.

In the basic sequence, the characters wait between each goal for a response from the player. The player can also interrupt. The handlers for responding to the player can alter and rearrange the performance of the next goals, mix in reactions about the topic of the beat, mix in reactions about other topics, or even cause the beat to abort. So, for example, if the player immediately asks for a beer when the topic of drinks first comes up, Trip will suggest a fancy drink in a different way and Grace’s counter-suggestion may specifically reference the fact that the player asked for a simple beer. As the player expresses affinity for the positions of the two characters during the beat, their performances can shift to reflect the new landscape of the affinity game.

Of course, the player may also take actions that aren’t about drinks. The player may bring up a topic like divorce or therapy, or may refer to one of the objects in the room (by name, or by picking it up). This could mix in a JDB about one of the hot button games, at its current level of development. Or, for topics on which Grace and Trip have nothing to say (or when the system does not understand the player) Façade could mix in a JDB that “deflects” the player action either based on local context or using one of a set of non-responses that can appear throughout a playthrough. Each beat goal has specially-written dialogue used to re-establish context (that is, to make an audience-readable transition) when returning from a mix-in.

Beats are also key to Façade’s approach to the incredibly difficult problem of natural-language understanding. No computer system can actually “understand” arbitrary human language — or even, less ambitiously, consistently map human language statements to a logical model. What Façade does, instead, is attempt to map each player statement to a limited set of “discourse acts,” such as agreement, thanking, referring to a topic, and so on. Then, each of these discourse acts is handled differently (by a set of “reaction proposers”) depending on the current context. For example, after Trip proposes a fancy drink, agreeing with him and thanking him result onto the same reactions in the Façade system. However, when Grace criticizes the decorating, it doesn’t make sense to have agreement and thanking produce the same response.

Finally, all of Trip and Grace’s joint behavior, in turn, is mixed with individual behaviors. Trip may go on fixing drinks over the boundaries of several beats and mix-ins. Consuming drinks, playing with a magic eight-ball, and other individual behaviors take place in parallel with joint beats and mix-ins. Each JDB is written so that it can be performed when the characters are in different physical locations in the room, carrying out other individual behaviors, and so on. There is a profound result that arises from these individual behaviors, the reactions of handlers, and the prevalence of mix-ins. While character behavior in Façade is largely driven by a script, it doesn’t feel as though it takes place in a series of scripted segments, but rather as a continuity.

Notes

⁹Though similar ideas had been pursued in domains such as battlefield simulation, e.g., (Tambe, 1997).