October 30, 2007
Last year I posted a series of thoughts about two story generation systems: Minstrel and Universe (1 2 3 4 5). I had some critical things to say about the Minstrel system, but they were based on my reading — I hadn’t yet been in contact with the system’s author, Scott Turner. This month I finally connected with Scott, and yesterday he sent me the following thoughtful response to the issues raised by our previous discussion on Grand Text Auto.
I’m particularly happy about this because Scott has graciously offered to try to respond to any further questions in the comments for this thread. Also, I’ll be paying close attention to the conversation, given I’m writing about Minstrel in my forthcoming book. Below are Scott’s thoughts.
I haven’t worked in AI for many years, but I was delighted when Noah contacted me and I had a chance to read the discussion on this blog of my dissertation work. At the time I did this work there was no Internet as we know it today, and in some sense I worked virtually in isolation. No one else was working on computer storytelling, creativity or related subjects such as interactive fiction. The best that I could hope for in the way of a community of interest was occasionally meeting up with folks like Michael Lebowitz at a conference. I can’t help but think that if I were doing my work today, the feedback I could get through the Internet would greatly improve my results. The Internet is truly wonderful in the way it can bridge space and economics to bring together similar interests in ways that could never happen in the physical world!
After Noah pointed me towards this blog I read through the discussion of Minstrel and found it very thought provoking. I thought I’d take a few minutes to share some insight into how Minstrel came to be and discuss some of the issues that Noah raised.
At the time I began my work on Minstrel the only previous work on storytelling was Meehan’s Talespin. Talespin was intriguing because it was clearly an incomplete approach to storytelling1 and yet what it produced had many of the necessary elements of stories. Talespin’s stories were, if nothing else, internally consistent and about the sorts of things we expect in stories. What was missing was purpose. An author doesn’t just ramble a stream of character actions; he creates his story to try to achieve his own goals.2
Now, Talespin was essentially a planning engine3, so it seemed reasonable to build a better storytelling program by simply augmenting the Talespin model with a “meta”4 level of goals and plans representing what the author was trying to achieve with his storytelling. And, in fact, the first versions of Minstrel operated just this way.
One problem became immediately obvious with this approach: the stories weren’t original. They just regurgitated what Minstrel already knew. In truth, Talespin had this problem as well, but because Talespin had a fairly large dictionary of character goals, plans and actions it wasn’t as immediately obvious that it was only “shuffling the pieces around the board.” And while re-ordering knowledge might provide a low level of creativity, it clearly wasn’t sufficient for creating interesting stories.
Consequently my effort shifted from storytelling to creativity. Storytelling went from being an end in itself to being the domain in which Minstrel demonstrated creativity.5
Now, there are many fascinating questions about creativity, but one that intrigued me was how people could be creative across many different problem domains. (I’m speaking here of normal day-to-day creativity, not Thomas Edison-level creativity.) It seemed unlikely to me that people developed wholly different creative processes for different problem domains. Instead, I looked for ways in which creativity could be embedded in the low-level cognitive processes that underly all intelligence.
The result was Minstrel’s notion of “creative memory.” In Minstrel, if you try to recall something and draw a blank,6 the creative process kicks in and tries to imagine something appropriate. This notion has a beautiful elegance to it, because (whether one believes specifically in case-based reasoning or not) it is clear that memory underlies intelligence. So a creative memory — imagination — immediately adds creativity to all the upper levels of cognition.
There’s much more to be said (and argued) about creativity, but I will skip over that for the moment.
As I began to implement this model of creativity in Minstrel, I ran headlong into another difficult problem — knowledge and context.
Consider this (true) example of creativity:
A motorcyclist is motoring down a lonely road when his crankcase cracks. He pulls over in time to see all the oil drain from his engine. He has spare oil, but not welding equipment to fix his cracked crankcase. After some thought, he opens both ends of a can of beans, puts the cylinder on the road and builds a fire inside of it. He lets the fire burn for about a half-hour. Then he removes the can, uses a stick to scoop up the softened asphalt and patches the crack. He lets the patch dry, refills the oil, and limps into the next town for more permanent repairs.
Suppose we try to detail all the knowledge that the motorcyclist required to invent this solution to his problem: roads are made of asphalt, asphalt melts at a low temperature, asphalt is sticky when melted, asphalt adheres when dry, asphalt is impermeable to oil, food comes in cans that are cylindrical and made of metal, a can opener can remove the entire lid of a can, removing both ends of can forms a cylinder… A full listing of all the knowledge needed to understand/create this example would cover pages.
To put it another way, before we can be creative, we need all the knowledge and cognitive processes to be non-creative.7 Doug Lenat has had 40 people working twenty years on cataloging this sort of common-sense knowledge and hasn’t yet finished. So clearly I, the lonely graduate student, faced a serious problem.
The solution I chose was to limit Minstrel to a “toy domain”8, and because of the depth of knowledge Minstrel required, it was a very tiny domain indeed. So when Noah asks why Minstrel was “starved for data” the primary answer is that I didn’t have the manpower to represent and encode more data. I had to choose between coding up more example stories, or implementing the processes of storytelling and creativity. In addition, I was wary of the criticism I might receive if I gave Minstrel “interesting” data to start with. Certainly if I’d started Minstrel with a wide variety of unique and clever story scenes, it could have produced unique and clever inventions. But would the creativity lie in Minstrel or in me? So I chose instead to limit Minstrel to a small set of “uninteresting” data to make it clear that any creativity displayed was due to Minstrel.
Nonetheless, Minstrel was a brittle program. My contention is that if you give me a robust, non-creative program that demonstrates all the world knowledge, intelligence and problem-solving ability of the average 21 year old, I’ll be able to implement a robust creative program atop that. But I didn’t have that luxury. I had to build just exactly those parts of that robust intelligence I needed to demonstrate my thesis. Naturally, if you stray even slightly from those limits, things break.
Talking about Minstrel’s brittleness, Noah says:
But it is important to remember that this problem arose with the completed system (and not an incomplete one, as with the mis-spun tales of Tale-Spin reprinted by Aarseth, Murray, Bolter, and others).
This is, I think, a false distinction. In this sort of research, the programs we write aren’t products that are “complete” when the Ph.D. is awarded9. They are vehicles for implementing, experimenting with, and understanding theories of artificial intelligence. Meehan fixed many of his mis-spun tales along the way to the “final” version of Talespin, and I did the same with Minstrel. At any rate, I’m dubious (as was Meehan) about drawing anything more than entertainment from these mis-spun tales. As one of my committee members said10, “Don’t these just prove that you’re a bad programmer?”
Noah also repeats the comment from Rafael Pérez y Pérez and Mike Sharples:
[T]he reader can imagine a Knight who is sewing his socks and pricked himself by accident; in this case, because the action of sewing produced an injury to the Knight, Minstrel would treat sewing as a method to kill someone.
(p. 21, “Three Computer-Based Models of Story-Telling: BRUTUS,MINSTREL and MEXICA”)
Like some of the commentors on the previous thread, I actually find this an (unintentionally) wonderful example of creativity, and exactly the sort of thing Minstrel ought to be capable of creating. There’s an Irish folk song in which a woman imprisons her husband by sewing him into the bedsheets while he sleeps. Doesn’t that show exactly the same creative process (magnifying a small effect to create a large one)? The problem with Minstrel is that it would do a bad job of the magnifying. It would write something like:
The Knight killed himself by sewing on himself.
because it doesn’t have the knowledge and reasoning processes to create a more plausible “magnification” of the effect. But imagine a novel in which a terrorist in solitary confinement is given access to a sewing needle because “What could he do with a sewing needle?” He then manages to kill himself by plunging the needle into his carotid artery — seems like a fine plot device, yes? But all this really says about Minstrel is that it isn’t a complete reasoner — which I’d not dispute.
As to the larger question of whether it is even possible to capture the knowledge necessary for human-level storytelling, or whether “scruffy” AI is a bankrupt idea, I’ll not comment except to say that it takes the most highly-evolved learning machines on the planet at least 20 years to build that sort of knowledge base, and even then most of those do not become good authors. So expecting a complete solution from a program with a few grad-student-years of effort sets the bar unreasonably high.
Noah concludes by saying that whether Minstrel is an improvement over Talespin is “debatable.” That seems like hyperbole to me. I can’t imagine that anyone would seriously argue for a model of storytelling that didn’t include explicit author goals, or creativity, or the notion of boredom — all of which Minstrel brought to the table. There’s certainly plenty of room to criticize Minstrel, but I think it was a leap forward from Talespin and began to address some of the difficult (and fascinating) issues in both storytelling and creativity.
1. Indeed, the only way Talespin produced narratives that seemed at all like stories was to force the narrative into a supplied “storymold”.
2. It’s worth noting that Meehan understood this point very well.
3. My advisor once asked this question: “What are the fundamental differences between Talespin and blocks world planning?”
4. As we used to say in the UCLA AI Lab, “Anything you can do I can do meta.”
5. However, Minstrel was also capable of doing device invention using the same creativity techniques it used in storytelling, and lately I’ve been applying the same creativity techniques to music composition.
6. Or find the answer you’ve recalled “boring.”
7. Common-sense knowledge also serves as a back-end filter or “sanity check” on creativity, although Minstrel didn’t model this.
8. A very common solution to that problem in those days.
9. Indeed, Minstrel was more a tool set than a monolithic program. It was constantly being changed, updated and used in new ways as I explored different issues.
10. If I recall correctly. Perhaps this comment was made to Meehan and I’m suffering from “creative memory”.