May 14, 2003

Expressive AI

by Michael Mateas · , 7:50 pm

Hi, I’m Michael Mateas. My particular interest in the topics of this blog is thinking about, and making, artificial intelligence-based interactive experiences. AI techniques enable interactive art to be more porous to human meanings, to generate responses that more deeply incorporate interaction. In my work I’m interested in developing AI techniques and architectures that enable new forms of interactive experience. I’m currently working with Andrew Stern on the interactive drama Façade, which he described in more detail in his first post. Previous work includes Terminal Time, an interactive video piece that constructs ideologically-biased documentary histories in response to audience feedback, and Office Plant #1, a desktop robot that responds to the social and emotional tone of the email you receive.

One strand of work I’m interested in is interactive narrative. The well-known conundrum, which has already been raised in the opening comments on this blog, is how to combine interaction, which seems to be about the freedom to do what you want, with narrative, which seems to be about tightly crafted sequences. I believe AI techniques will be vital in creating interactive story systems that can generate large numbers of tightly crafted sequences that incorporate player interaction. That is, such systems support the author in capturing their authorial intention at a more abstract level than handcrafted sequences – this more abstract description is then used to generate the concrete sequences that result during any one play of the story. For example, in Façade we have authored a collection of dramatic beats, individual dramatic moments or episodes, that can sequence together in a fairly large number of ways. Whenever a beat is chosen to be sequenced, the exact choice made depends on what’s happened in the story so far, including player interaction. The beats are described using a custom language to specify the various conditions under which it is appropriate to choose a beat. Each beat is itself a mini story machine, a collection of character behaviors that can incorporate player interaction within the beat to accomplish the dramatic moment of the beat in a number of ways. The behaviors are written in another custom language designed for authoring real-time, reactive characters. There is of course more to the Façade architecture than this, but it gives an example of what I mean by specifying a story at a more abstract level. Incidentally, Andrew and I in no way feel that we have “solved” the interactive story problem. After having spent almost four years building Façade, we are keenly aware of the many limitations of our approach, which I’m sure we’ll discuss on this blog. There are many approaches to capturing story structure at a more abstract level – we’re both excited to move forward with new projects that build on the lessons learned in Façade. (For more on Façade, see Architecture, Authorial Idioms and Early Observations of the Interactive Drama Façade).

Skeptics may feel that I’m saying that all the problems of interactive art have a technical solution – just use artificial intelligence and presto, you have art. By this way of thinking, art becomes an application area of AI. Every art project should start with the step “First, build a general purpose AI” (watch out, that first step is a doozy). But AI-based art and entertainment is not simply an “application” of AI. Combining an art practice and AI research results in an new agenda, one I call Expressive AI. Expressive AI raises AI research questions that would never come up unless doing AI research in the context of art making, and opens up new opportunities in interactive art that would never arise unless making art in the context of a research practice. I’m looking forward to talking more about Expressive AI on this blog. If you would like to read more now (probably much more than you want to read) see my dissertation, Interactive Drama, Art and Artificial Intelligence.