March 18, 2008
EP 8.6: Learning from Façade
The surface experience produced by the Façade’s processes and data is shaped by a series of choices that have clear impacts in terms of the Eliza and Tale-Spin effects. The results are instructive.
Speaking to Façade
As with Eliza/Doctor, the freeform textual interaction of Façade invites play. Unlike Eliza/Doctor, the Façade system actually models aspects of the current context and ongoing conversation, so there’s more potential reward for this play. Façade’s audience members may interact in their own styles, whether to influence Trip and Grace, to express a particular idea of the player character’s identity, or to find the edges of the system.
The player aims of individual expression and system edge-finding, which sometimes overlap, can produce particularly memorable Façade transcripts. For example, one person posting on the Idle Thumbs forums decided to play in an unusual way: taking appropriate physical actions (sitting on the couch, answering the phone) but only saying one thing:
PHONE ** RING **
(GONZALO picks up the phone.)
PHONE ** RING **
PHONE Hello? Hello?
TRIP Gonzalo, no! Don’t do that.
GONZALO Brains!!
PHONE Who is this? Travis, is that you? This is your mother.
TRIP Uhh, that’s my mother, I can hear her loud voice.
GONZALO Brains!!!!!!
PHONE I don’t recognize your voice…
PHONE I – I think I have the wrong number… ** click **
GONZALO Brains?
(GONZALO puts down the phone.)
(Mailman, 2005)
As the poster concludes, “Playing as a zombie is fun.” Other players on the same forum share transcripts in which they play someone who has just been shot, an alien seeking tissue samples, Darth Vader (who screams “Noooooooooooo!” after being ejected from the apartment), and Grace’s secret lover. Of course, there are also many transcripts featuring player characters somewhat closer to Façade’s expectations, but those that push the system are particularly revealing in terms of the Eliza effect. For example, the aspect of Façade that makes the zombie transcript funny is also a clear example of how Eliza effect breakdowns reveal something of the shape of the underlying system. In this case, one sees the way Façade’s drama continues executing its beat goals, for the most part, even in the absence of intelligible player behavior.
In using a free-form natural language interface, Façade makes a certain tradeoff. Players are invited to perform more richly than in many digital fictions, which often limit interaction to the selection between a set of discrete choices (with everything else producing an error message).10 On the other hand, the actions of the Façade system actually reduce each player utterance to one of a set of discrete discourse acts — and not always successfully. This creates a serious mismatch. Personally, every time that I have played Façade in a manner I considered “according to expectations” a statement I thought of as perfectly normal produced a reaction so unexpected that I found myself thinking about the shape of the underlying system.
In fact, Façade’s authors estimate that natural language understanding failure takes place about 30% of the time. As a player, my subjective feeling is that failures are less frequent, but this is due to the clever ways that Façade deflects and recovers in many circumstances. And, as Mateas and Stern write, “This tradeoff was intentional, since we wanted to better understand the new pleasures that natural language can offer when it succeeds” (2007, 206). One can see, here, that natural-language interaction offers powerful potential to the audience — but also, by digital media authors, must be regarded as an area of active research (rather than a mature technology). It also points to other research areas, such as interfaces that might allow for players to directly express Façade’s discourse acts, removing unreliable natural language understanding from the equation.11
Hearing from Façade
Interestingly, Façade also presents a similar tradeoff in the area of the Tale-Spin effect. However, this is one in which it follows the model of many contemporary games, rather than departing from it. This is the choice to use voice acting, so that Grace and Trip’s statements are performed by playing a sequence of pre-recorded sound files to which their animations are synchronized.
Just as free-form textual input allows players to be expressive, the human expression embodied in Façade’s strong voice acting is part of what makes the experience so effective. The nuances of line performance make things by turns funnier and more uncomfortable — and, overall, more engaging. On the other hand, Façade also required significant authoring effort to appropriately funnel a vast number of possible system states into a much smaller number of possible pre-recorded utterances.
If the system had used plain text as output, many more system states could have been experienced meaningfully on the work’s surface. This would have been true for two reasons. First, the mixing of pre-written segments could have been more fine grained. Façade already sounds strange when it mixes recordings of the player character’s name into lines that were specially recorded for the purpose, and going further in this direction would have greatly reduced what the experience gains from its voice acting. Second, plain text output would have opened the door to another interesting research area: the construction of systems like the “literary augmented grammars” of Brutus. These have the potential to adjust the nuance of text, by applying hand-authored rules, depending on the current system state. However, for performance-oriented systems (such as Façade and many modern computer games) this, too, is best regarded as an area of research. Though speech generation systems are rapidly improving, for many purposes voice acting will remain the more powerful approach. However, for systems such as massively-multiplayer online games, in which most non-player character dialogue takes place through text, such research could have immediate application.
Agency in Façade
Finally, there is also an element of Façade that communicates clearly to players and, when natural language understanding succeeds, operates much as they expect. This is the combination of drama management, beat goals, joint behaviors, handlers, and mix-ins — what I have called Façade’s “script.” This combination creates a performed story that progresses in a way the audience understands, during which players can direct conversation toward a range of topics, and which can take a variety of shapes that culminate in an appropriate ending.
This experience is not that of the SimCity effect, because the nature of Façade as a system is not in the foreground. Rather, as Mateas and Stern point out, it is an example of an experience in interactive media commonly termed “agency” or “intention.” Façade’s authors point to Janet Murray’s formulation of the concept, which has been particularly influential in academic circles: “Agency is the satisfying power to take meaningful action and see the results of our decisions and choices” (1997, 126). In the field of game design the idea is often associated with Doug Church, who writes of “allowing and encouraging players to do things intentionally” — understanding the game world well enough to make and execute a plan of action, then seeing a clear reaction from the game world (1999). However, it is also worth noting that a version of this concept can be seen from the earliest full-length writing on digital fictions of which I am aware: the PhD dissertation of Mary Ann Buckles. She describes this in relation to the psychological concept of “effectance” as “the desire for competence and feeling effective in dealing with the surrounding environment” (1985, 37). She discusses how the world of the early interactive fiction game Adventure works to build this experience in its audience.
The SimCity effect is one way to build the experience of agency. Façade’s script demonstrates another. Mateas argues that, in general, “A player will experience agency when there is a balance between the material and formal constraints” (2004, 125). The formal constraints are present in the shape of the fictional world, which motivate some actions and not others. The material constraints, on the other hand, are the resources available for action. Many contemporary games, for children and adults, create a sense of agency by presenting a fictional world in which one would want to move across space (e.g., to rescue a princess, to find necessary information) and fight enemies (e.g., who are mean animals, who are zombies/aliens/robots/Nazis) and providing exactly the tools necessary (well-developed mechanics for spatial movement and combat).
Given this, achieving agency is relatively well understood for game worlds with simple fictional worlds and simple available actions, because it is easy for players to understand and act within the bounds of the system.12 The SimCity effect describes a related route to agency, for more complex systems. What Façade provides, instead — in its script — is an example of a route to agency that functions with less necessity for player understanding of the system. Rather than needing to replace the audience’s initial Eliza effect notion of drama with one closer to the model of the system, players who avoid natural language understanding errors can experience agency in Façade’s world while continuing to operate largely based on mental models drawn from theater, media, and human interaction.13 We might say that this is the true dream of interactive drama.
Unfortunately, this also means that the dream remains somewhat elusive. The Façade approach, while powerful, suffers from unavoidable errors in natural language understanding. An alternative, such as players directly expressing discourse acts (rather than typing arbitrary text) would require teaching the audience to understand the system, reintroducing the importance of the SimCity effect.
Beyond Façade
In addition to research in natural language understanding and text generation, Façade also points to further work in a number of other areas. For example, in developing the approach embodied in Façade’s script, and the large amount of material necessary to support it, Stern and Mateas also developed a set of code templates that they re-used repeatedly. As they have speculated, the design elements formalized in these templates could become the primitives in higher-level authoring languages for digital fictions, or even in graphical or AI-assisted authoring tools.
In the meantime, Façade has demonstrated the power of the ideas formulated in its script — which should, in time, find their way into other digital media projects. Further, as authors adopt the techniques pioneered by Façade, their strengths and limitations will become yet more apparent. New research will explore new possibilities. As this happens, hopefully, mainstream gaming will abandon the dialogue tree, thereby increasing the potential richness of existing genres and opening the possibilities for new ones.
Further, while these relatively well-developed techniques — the cumulative result of more than a decade of technology and design research — may be adopted by mainstream digital media, researchers are likely to turn to the next challenge. Experiences like Façade will remain limited by the sheer amount of material that must be authored. Future research is likely to consider not only how to produce a wider variety of sentences based on authored rules (as in literary augmented grammars) but also how to automatically assemble structures more akin to Façade’s JDBs, or even Façade’s larger beats, as the current situation dictates. Until this succeeds, Façade -like experiences will require large amounts of authoring for every minute of dramatic experience.
More generally, Façade provides an important example of how authors can engage the history of AI. It adopts powerful tools that grow from an AI legacy. At the same time, it disposes of limiting concepts that are the baggage of that legacy — especially the insistence on seeing characters as “autonomous agents” who sense and react to each other, rather than as coordinated performers. From this perspective, for those willing to do the conceptual work, the history of AI provides an exciting source of future directions.
Finally, Façade manages to create an emotionally engaging, if often uncomfortable, interactive experience about the relationships between people. It manages this not simply through the operations of its script and characters, but also because it is designed to capitalize on the power of fiction’s most powerful tool: language. In this it provides a stark contrast to the history from which it flows, from Tale-Spin to the Woggles. But in this way it connects to another, parallel history — considered in the next chapter.
Notes
10Many make the small selection of available actions visible on the work’s surface. However, some, like text-based interactive fictions, make the search for available actions part of the experience.
11This is something that Steven Dow and Blair MacIntyre have explored in creating the “wizard” interface for their augmented reality version of Façade.
12Though design failures happen commonly, as when actions that can be used in one context cannot be used in another, for reasons unexplained by the fictional world, reducing the player’s ability to formulate and execute intentions.
13Though playing Façade also requires some basic 3D navigation literacy.
March 18th, 2008 at 8:34 am
I’m not sure if this is the end of the Façade material or not, but I find myself wishing for a more critical treatment of the work. There are many challenges to issue against it, and I’d guess that most of them have been said only online, strewn all over the web. They deserve condensation and discussion in print. I’m thinking of things like accusations that the natural language interface is an affectation, or Mateas and Stern’s insistence on recreating actors in the black box theater rather than abstracting as games like SimCity do, or the fact that Façade was born partly out of a sort of “dare” to do “real” interactive drama, or the very obsession with Aristotelian drama, possibly started but assuredly egged on by Murray, despite the 20th century’s innovation in a multitude of different storytelling techniques, or the apparent mismatch between AI innovation and user experience, both in terms of the public’s and critical reception of the work.
(sidenote: an interesting challenge for this peer review project is the reader’s inability to “look ahead” to see if topics that come to mind are coming up later. The surprise of serialization works for fiction, but I’m scratching my head about whether or not it does for nonfiction, at least nonfiction of this sort).
March 18th, 2008 at 8:56 am
Ian, yes, this is the end of the Facade material. Certainly some of the issues you raise come up in the book as it stands (e.g., in this and the two previous sections I set up the “grand challenge” of Laurel-style interactive drama). But it sounds like you’re proposing a catalog of criticisms of Facade’s project, which I’m probably not the person to write.
I don’t actually want to have this chapter include the definitive piece on Facade — I want to use Facade as an example toward the overall arguments of my book. Are there parts of what you propose that you think are particularly missing from this book, rather than things you would like to see someone condense and put into print?
March 18th, 2008 at 9:07 am
I’m not really looking for a catalog of criticisms (the ones above were examples, not requirements). But isn’t one of your arguments about the ups and downs of the various “effects” for expressive processing? It seems that Eliza got a critical treatment that Facade is not getting here.
Maybe I’m just too familiar with the examples and you’re right that simply introducing this one is sufficient. Maybe I am (unfairly) asking for criticism of a sort out of place in this book. Maybe I’m mis-anticipating how the throughline of your book and how it will play out.
Once again, I feel like I have to admit that this online reading method is just not working for me, because I can’t see the whole picture enough to make substantive comments on a daily post, save for very specific, even pedantic ones.
March 18th, 2008 at 1:06 pm
Calling the idea of autonomous agents “baggage” and “limiting concept” is a little harsh!
I think you are setting up a false dichotomy here: seeing agents as autonomous and seeing them as coordinated performers are two compatible views – they are not exclusive. In fact, in our current project, we are treating agents as *both*. It is certainly true that agents need to be participating in coordinated practices for dramatic situations to arise – but they also need to be unique autonomous individuals so that we get emergent *variations* on situations.
The most common criticism of Facade from game industry folks is that the bang-for-buck (amount of dramatic content gained for the amount of authoring required) is too low. This low ratio is, to my mind, a direct result of not treating the agents as autonomous individuals. Unique autonomous individuals operating within the guidelines of a script gives emergent variation within a dramatic situation – just what is needed to raise the bang-for-buck ratio.
(I don’t want these remarks to be taken as criticism. I am a big fan of Facade – it is a towering achievement).
March 18th, 2008 at 2:50 pm
Well, we certainly welcome an extensive flogging (er, critical reading) of Facade, whether it’s in this book or elsewhere.
Serious critical treatment is something games rarely receive; such treatment would benefit both developers and players.
I thank Noah for what he’s put together here, especially the way he relates different systems to one another.
March 18th, 2008 at 3:28 pm
I agree with Richard that strongly autonomous characters shouldn’t be painted as a limited concept per se; it’s an approach that has its strengths and weaknesses. I think we wrote a good treatment of this issue in section 7 of our 2004 ABL paper (pdf), where we compare the “many-mind” approach (i.e., strongly autonomous characters), the “one-mind” approach (e.g. Lebowitz’s Universe), and what we call “a variably coupled multi-mind” in which agents can simultaneously be performing joint and well as autonomous behavior. The Facade architecture supports the multi-mind approach, as Richard reports his current project does as well. (However, in the end, ratio-wise, Grace and Trip perform joint behaviors far more often than autonomous ones; we’d like to see this be more balanced in future projects.)
Richard’s point about authoring bang-for-the-buck is valid, although I disagree that it is because the characters aren’t strongly autonomous. I think the public has some misconception about how much authoring actually went into Facade — a misconception that is our fault, for “bragging” that Facade took two people five years to make, presumably totalling ten person-years of work. Actually we did other things while developing Facade; for example I worked full-time at Zoesis for over a year, and Michael was full-time at Georgia Tech for over 2 years. After the project we sat down and tallied the Facade work up to be six person-years of work. More importantly though, about 75% of those six person-years was spent architecting, building infrastructure, and developing authoring idioms (i.e., R&D); therefore less than two person-years was spent on actual authoring of beats, which includes behavior programming, animation and testing. In other words, with this architecture in hand, two trained people could crank out another interactive drama equivalent to Facade in less than a year. So the authoring bang-for-the-buck ratio is probably a little better than is normally assumed.
(Note, your typical 2-year game production, e.g. Weapon Death Spiral 4000, with its 50 hours of grinding gameplay, requires 10+ person-years of authoring of levels, animation, AI, testing…)
Anyhow I think the issue at hand here is, for any interactive character/story system, what is its potential for generativity. Richard’s mention of emergence brings me back to question I asked the other week — where would you draw the line between story “emergence” from sandbox style games, and story “generation”? Does “emergent” story fall out of a slightly flexible system, and “generated” story from a highly flexible one, referring to the terms of your discussion of Universe?
March 18th, 2008 at 3:53 pm
I question the choice of the term “script” as the umbrella term for the combination of drama management, beat goals, joint behaviors, handlers, and mix-ins. (You first used the term in paragraphs 17 and 23 of section 8.5.)
The term “script” has a lot of linearity baggage to it, not just in the context of drama, but also in game development.
I’m trying to think of another single umbrella term… Maybe “system”, but that doesn’t allude to the narrative drive that “script” does. Maybe “plan”, since ABL is a reactive planner, and “plan” doesn’t have the linearity baggage.
“Narrative intelligence” seems accurate, but it’s cumbersome.
Or maybe I’m not totally getting why you chose “script”.
Michael, any thoughts?
March 18th, 2008 at 8:34 pm
Well, I think of the “script” as the body of writing that drives the experience. In this case, it’s writing of dialogue, behavior, and broader logic — and the results aren’t linear in the way a normal script is — so maybe it isn’t the right term. But I can’t think of a better one.
Maybe the solution is to add another word? For example, I could call it a “dynamic script” or something of the sort.
March 18th, 2008 at 8:42 pm
Ian, you’re absolutely right that Facade comes off differently from Eliza. But I think the reason for that is that I’m focusing on the relationship between underlying process and audience experience. For Eliza that relationship results in a relatively limited boom/bust. For Facade it’s a lot more complex. I think I try to point to the problematic aspects (e.g., NLU failure), the successful aspects (e.g., the script structure), and what I might call “revealing” aspects (e.g., how the experience keeps going even in the absence of playing along, as with the zombie transcript). Or, at least that’s what I think I’m doing. Does it read like I’m giving a pass?
Andrew, I’m glad you think the drawing of relationships between systems is working. I know you’ve been quite critical of Facade in some postmortem-style discussions. So maybe I can ask you, even though you’re obviously biased, the same question: Does this seem like a one-sided discussion of the project?
March 18th, 2008 at 8:49 pm
Richard, you’re completely right. This reads in a way that, actually, doesn’t make sense. What I mean to say is that Facade disposes of a sort of ideology of constructing digital characters as compartmentalized from one another and the broader system — an ideology that was often compromised in practice, but with a sort of embarrassment about it.
I need to rewrite this to make clear that the uses of autonomy and coordination should really be design decisions, rather than something we assume there’s a “right” way to do from the outset.
And, of course, I should probably also note that people were, before Facade, perfectly happy to do character coordination in projects that grew out of an animation tradition (e.g., Improv, if memory serves) rather than an AI tradition.
March 18th, 2008 at 8:57 pm
Andrew, in the discussion of Universe I didn’t mean to draw a distinction between the ideas of “generated” and “emergent” in my discussion of system flexibility. Instead, I was basically previewing some of my discussion of Brutus, which turns out (in my examination, at any rate) to mostly be an elaborate way of breaking up an entirely predictable, hand-coded story into pieces for the system to reassemble into the same shape every time.
I think it might be interesting to draw another distinction, between sorts of systems that are flexible enough to assemble stories into different shapes, but I haven’t done that here. Do you think it’s important that I do so in the book? If so, I might have to add another example, or take one of my existing examples in a significantly different direction.
March 19th, 2008 at 8:52 am
I agree that the successes and failures of Façade are much more complex than those of Eliza. But I’m not sure why that’s an excuse not to discuss them in greater detail?
Maybe what I’m looking for is actually not related to Façade, but only pointed to by the queasiness I’ve been expressing here: I’m not finding enough signposts and revisits of the throughline of your main argument in the different chapters. What do you think?
March 19th, 2008 at 9:16 am
Ian, I’m not sure I understand. Yes, I could have written an entire chapter on Facade — or Minstrel, or The Sims, or Knights of the Old Republic, or Abelson’s ideology machine, or many other systems that currently are only discussed for the length of a section. Why is it particularly disturbing that I didn’t write a whole chapter on Facade?
As for the main argument, that’s an interesting issue. One part of my main argument — that we need to pay attention to how digital media systems operate, not just what they look like from the outside — I’m pretty much treating as established by this part of the manuscript. But I could return to it here. Another part of the argument — that the Eliza, Tale-Spin, and SimCity effects are useful for thinking about the relationship between system processes and audience experiences — is explicitly what I’m trying to connect to in this section. I guess it’s not working?
March 19th, 2008 at 10:01 am
Noah, yes I think it’s not working, at least not satisfactorily. I’m about to post a long comment in this regard on the new “meta” post, so maybe we can continue this there.
March 20th, 2008 at 5:57 am
I’m not sure if the analogy would be helpful or muddling, but it reminds me a bit of the back and forth in computer animation (the graphics stuff, not character-related per se) between faithful physical simulation and “cheating” with e.g. keyframes and artificial forces. For example, you could try as faithfully as possible to simulate what smoke dispersion looks like… but on the other hand you might want the smoke to curl just so at 53 seconds in, and the chances of setting things up so that just happens are slim. So instead you usually end up doing some combination of faithful internal simulation and “cheating” external imposition.
March 20th, 2008 at 11:40 pm
“Dynamic script” is kind of vague, and starts to imply something else than what you really mean here — as if it was a traditional dramatic script with variation/branches in it.
Maybe “dynamic content”?
I don’t know, there may not be a perfect solution here. You do define your use of the term “script”, so that helps…
Hey — how about “goodies” ? ;-)
March 20th, 2008 at 11:43 pm
No, your analysis has good critique in it; but as Ian suggests, there’s even further one could go with it, if desired. I don’t think you’ve done too little though.
March 20th, 2008 at 11:49 pm
I can’t think of a place where someone has tried to break down the distinction between “emergent” and “generative” — I think there are nuanced differences — and I thought your various analyses here could be an opportunity and a great place to do it.
But no, I don’t think it’s particularly necessary or terribly important to do so.
March 21st, 2008 at 9:35 pm
The more I think about it, the more I think there has got to be a better term than “script” here.
Something like “narrative content” might work. It has a similar meaning to “script”, but without implying linearity in structure.
If I may be so bold, I’ll take the liberty to illustrate how this term would look in your text, to help us get a feel for it. These are just suggestions, please ignore if inappropriate.
From section 8.5 paragraph 16:
“One can see this clearly by looking at Façade’s narrative content.”
8.5, Paragraph 23:
“While character behavior in Façade is largely driven by narrative content, it doesn’t feel as though it takes place in a series of scripted segments, but rather as a continuity.”
Section 8.6, paragraph 13:
The SimCity effect is one way to build the experience of agency. Façade’s narrative content demonstrates another.
Etc.
March 21st, 2008 at 10:20 pm
Hmm… I guess the issue is that I’m looking for a term that communicates “thing that specifies what characters say and do.” I think that “script” has some linearity baggage, but it comes pretty close to communicating the concept. “Narrative content” feels a lot more ambiguous to me. But maybe it’s not so important that people know what the term means before I explain it.
Pursing this different approach, did you and Michael have any internal terminology that might be useful here? I could just call it “mintly” — or something else that has no baggage — if I was adopting a term with some prior connection to Facade itself.
March 21st, 2008 at 10:28 pm
“Content” is the closest we came to an umbrella term for the authored pieces (drama management rules, beat goals, joint behaviors, handlers, and mix-ins).
For example, we titled a paper “Structuring Content in the Façade Interactive Drama Architecture”. By “content”, we mean the essentially same thing you do here as “script”.
March 22nd, 2008 at 3:57 pm
Andrew, you write that “content” “has a similar meaning to ‘script’, but without implying linearity in structure.” But “content” implies to me that there is no structure at all – structure seems separate from content to me. I find it a rather vague word that reminds me of Web development, and which I try to avoid in that context and others. On the other hand, I don’t have a better suggestion.
March 22nd, 2008 at 4:55 pm
True, “content” doesn’t really work well — but I suppose I’d prefer a bit of vagueness over a mischaracterization.
Maybe “structured content”?
Anyhow, since Noah does explicitly define what he means by “script”, if he leaves it as is, I can live with it.
March 23rd, 2008 at 12:21 pm
Maybe “procedural script” is an improvement?
March 23rd, 2008 at 1:38 pm
Yeah, that’s pretty good…