June 11, 2003
Sweden Trip Report (complete with drama management digression)
I just spent the last week in Stockholm as an invited opponent on a licentiate thesis in interactive drama. While there I was able to visit with a number of folks in the Swedish Institute for Computer Science, the IT University of Kista, and the mobility studio of the Interactive Institute.
A licentiate thesis is sort of a cross between a master’s degree and a Ph.D. proposal. Unlike the U.S. system, in which the questioning is done by a committee, in the Swedish system an external person is brought in to be an official Opponent. After the student’s presentation, the opponent gives a short presentation situating the student’s work in the field and then questions the student for 45 minutes or so. At the final Ph.D. defense, the opponent actually presents the work! The student is then given a few minutes to add any corrections to the presentation, then the opponent again questions the student for 45 minutes. The idea behind the opponent giving the presentation is that, in order to show that they are capable of asking good questions and being a worthy opponent, they should be able to present the student’s work in full detail. This makes sense, though it is obviously more work for the opponent. It was fun being an opponent on the licentiate thesis – I look forward to being an opponent on the full thesis.
The most interesting direction in Jarmo’s Laaksolahti’s thesis is his approach to drama management. Brenda Laurel, in her seminal thesis on interactive drama, introduced the abstract interactive drama architecture consisting of a simulation world containing autonomous characters, and a drama manager (she called it a Playwright) that observes the concrete activity in the simulation world and tweaks it to help make a story happen. The drama manager might manipulate the minds of the autonomous characters (e.g. giving them new goals, plans and motivations), change the physical world behind a player’s back (e.g. make a prop appear or disappear) or make an event happen (e.g. an explosion, rain, etc.). Building a drama manager involves figuring out how to represent the desired story, defining the set of drama manager moves with which the drama manager can manipulate the world, and defining a mechanism for computing a story policy. A story policy is a function that, given the history of activity in the story world so far, the representation of the desired story (or stories) and the collection of drama management moves, decides which move to make when. Some past approaches to drama management include:
- specifying an evaluation function that, given a total story sequence (at the scene level), returns a “goodness” rating. Given a partial, in progress story, game-tree search is used to project all possible story futures. The evaluations of the projected total stories are backed up (using average-max rather than mini-max) to assign ratings to the next possible drama manager moves (This method was developed in Peter Weyhrauch’s Oz Project thesis and introduced here).
- using a story-grammar-like rule system to specify the rules of “good” stories – the rule system filters the possible actions of the autonomous characters (e.g. ID Tension).
- distributing story knowledge among the story pieces (beats or scenes) and using this knowledge to maintain a probabilistic agenda of story moves (Andrew’s and my approach in Façade).
- the ever popular null drama manager, in which there is no drama manager, only a simulation world that the author tunes so as to make stories emerge (some problems with this approach are discussed on this thread).
- monolithic systems in which the story and characters are represented within a single framework, perhaps using planning to determine sequences of detailed character action necessary to make a story happen (Liquid Narrative, Erasmatron).
Jarmo’s approach is to represent the story as a finite state machine in which nodes correspond to scenes. Transitions are labeled with conditions over story state (including character state such as emotional state). The main new idea is to forward simulate the autonomous characters to determine if an undesired story state happens in the near future. If an undesired story state is likely to happen, the drama manager takes action to correct this state. What’s interesting about forward simulating the characters (running the autonomous characters faster than real time without effecting the world) is that the authorial work of specifying the characters is reused in drama management. The author spends a lot of effort to craft the personalities of the characters (typically in a reactive planning framework) such that their personalities manifest in their moment-by-moment interactions with the player. Then the author has to specify completely different knowledge at the story level to capture “goodness” of story (for the particular story world). It is interesting if the moment-by-moment knowledge encoded in the characters can somehow be reused at the story level for computing the drama management policy. Also, it makes intuitive sense that characters can help create a higher quality story experience by looking ahead at their own activity (planning is a special case of forward simulation). I and others have discussed this idea over the years, and Oz Project member Scott Neal Reilly even did some unpublished work in forward simulation of characters by combining Hap (a reactive planning framework for characters) with the Prodigy planner. But Jarmo’s work is the first published work I’ve seen pursing this idea specifically for drama management. While there are many conceptual difficulties to overcome, I look forward to seeing what Jarmo does with this idea.
During my trip I finally got to meet Jarmo’s advisor Kristina Höök, who I’ve known virtually for several years. Kristina has done interesting work in social navigation, and is currently interested in affective devices.
Jonas Söderberg at SICS has built magical devices for live role playing games. The magical devices include old spell books containing small computers and magic mirrors made out of flat-panel displays.
At the Mobility Studio of the Interactive Institute, I met with Liselott Brunnberg and Oskar Juhlin. They are currently working on portable devices for situated story generation and lightweight social interaction among bikers. In the story generation project, Lise is building a system that generates stories for children riding around in a car with their parents. As they drive around, the system incorporates the physical locations they pass in the car into the story. The biker project is grounded in an ethnographic study of social interaction among bikers on road trips. Oskar and I had a nice conversation about ethnography and design – I used to work as an ethnographer and concept designer at Intel Labs. We talked about how, in my current work in AI-based art, I’m hoping to bring ethnography back into my art practice by situating it relative to site-specific art, but where the site becomes a cultural rather than physical site.
I spent time hanging out with Loren Terveen, another opponent from the U.S. that Kristina had invited over for a different thesis. We had great fun together. One of the hilights of the trip was being taken herring fishing in the Baltic Sea by Kristina’s family. At this time of year the herring schools are so dense that you can pull three or four herring out at a time almost as fast as you drop the line in. Loren had the record in our boat by pulling out five at a time.
June 11th, 2003 at 5:46 pm
If an undesired story state is likely to happen, the drama manager takes action to correct this state.
In a projected future, I could imagine how to make the system be able to tell if “undesired story state” is happening, but it’s much less clear how the “take action to correct this state” is achieved… Did Jarmo yet suggest any ideas of how he plans to do that? That seems pretty hard to me.
June 12th, 2003 at 3:48 am
I hope I am not overstepping my authority by replying in person to this?
Dragging the story from an undesirable state to a desirable one is undoubtedly a difficult task. So far I have been thinking about it on a high level, but the nitty-gritty details remain to be worked out. The way I have been thinking about it so far is that you have a set of ‘effectors’ that are applied to drag the story from an undesirable to a desirable state. An effector could for instance be “introduce a new topic of discussion”, “give agent a new higher level goal to pursue” or “have a temper tantrum”. Guidance about which effector(s) that should be applied in a certain situation is given by preconditions associated with transitions to desirable states. For example if a character is required to have a certain emotional state for a desirable transition to take place effectors that cause such an emotional change in the character are chosen. However, I am not sure about what level that effectors should work on yet. Maybe they should work on individual agent goals, plans and intentions but it might be more appropriate if they work by instead selecting appropriate ‘beats’ to perform a la Facade (or something completely different). The future will tell.
June 12th, 2003 at 9:37 am
Since Jamo seems to be listening, I was wondering if you had gone through any arguments why an interactive experience would be _more_ compelling if it was structured more like a story?
There is certainly a case to be made that this would decrease the consistency of the world and possibly generally be a worse experience, even if all AI problems were solved. Comments?
June 12th, 2003 at 11:55 am
Jarmo, delighted that you posted a comment!
I’d just like to add my understanding of how you are currently thinking of selecting an effector.
In the case that the bad state is implicit (meaning that the finite state machine failed to accept the stream of story world changes), you search through the effectors looking for one that modifies the story world state so as to produce a stream of story world changes (a history) that the FSM can accept. Depending on the number of effectors and the complexity of abstract story world state you’re maintaining, this search could become expensive.
In the case that you’ve explicitly modeled the bad state in the FSM (a node marked as “bad”), then the transition from the bad state back to a good state provides information about which effector to use. Presumably there are many more bad than good story states, so you wouldn’t want to have to explictly model all of them in the FSM. By modeling a subset of them (and, through the transitions back to good states, providing effector hints), the author can control the combinatorial explosion of the search for an appropriate effector.
Is this understanding correct?
June 16th, 2003 at 3:49 pm
…why an interactive experience would be _more_ compelling if it was structured more like a story?
I don’t think that adding a dose of narrativity to games makes them _more_ compelling in general. It might even be the case that doing so makes the experience worse (although I am not entirely sure about what you mean by that). Many of the non-narrative games out there are great and I love playing them!
However, I do believe that games that already have a narrative structure, e.g. role-playing games or adventure games could benefit from an even stronger narrative structure, but also a more interactive structure. Players interaction with the game should influence the story more than it currently does.
In addition I believe that there is a need for alternatives to the action oriented games that are currently dominating the market. Games that focus on telling a story, have a slower pace and require less motor skills might be an alternative for players that are not in to action games for one reason or the other e.g. elderly.
June 16th, 2003 at 4:06 pm
In the case that the bad state is implicit …
Your understanding is correct. However, each effector will typically change one parameter/value and a state transition places conditions on many such values. My hope is that effectors that turns values in the right directon can be applied as soon as they are found. The longer the process can go on before it has to become reactive (i.e. the systems head start is eaten up) the more effectors can be applied. So in a sense it would be an any-time algorithm.
June 17th, 2003 at 4:44 pm
Jarmo,
Your work sounds interesting, and it’s really nice that you can come here and continue the discussion.
I have always felt that abstracting charcter actions into story actions is an extremely difficult process. Often it seems you are creating some kind of model of the characters in your story system. Using the character achiticture to “Fast Forward” the simulation to get information is one solution that avoids this particular problem and seems promising, especially when you are searching for problems in the short term.
One quick question: is there one FSM per character and one for the story, or is there just one FSM that controls the character and story progression?
Meanwhile, there is a potential computational explosion, which brings up a question that has plagued my thinking for quite some time…
Can interactive story telling can be done considering just the immediate future (i.e., look ahead just a few steps and make corrections) or does it actually require considering the distant future?
I’ll leave out the technical details of this question for now, but I’m curious of people’s opinions.
June 19th, 2003 at 5:29 am
is there one FSM per character and one for the story, or is there just one FSM that controls the character and story progression?
Currently there is a single FSM that controls story progression through the characters. The characters themselves are coded in the JAM agent language. I’ll take this opportunity to say that I am not sure that a FSM is the best representation (for my purposes), but it is simple and good enough to start with. An alternative would be to use e.g. activation networks. One benefit of using networks is that it presumably would be easier to find the next state to aim for i.e, the most activated node.
As for whether interactive storytelling requires looking into the distant future I think it depends on how much control you want over the story. If your aim is to somehow “optimize” the story or the players experience of it, I think you need to look further ahead into the future. If on the other hand you are satisfied with the user having a good , but not necessarily optimal, experience I think you can get away with a shorter lookahead. Good but not necessarily optimal experiences are what I am aiming for in my work :-). I think this fits well with Robert Rosens definition of anticipatory systems (which I am very much influenced by) and his notion of how they may be controlled. Instead of constantly trying to find the best trajectory through a state space a correction mechanism kicks in only when the system predicts that something will go wrong. However I guess that the frequency with which this will happen depends on your definition of “wrongness”. I think I am just adding to the confusion.