February 18, 2004
Jabberwacky
We’re reading Sherry Turkle’s Life on the Screen in my Internet Writing & Society class, and discussing AI. While googling around trying to see if there was a working version of Depression 2.0 out there, I ran across Jabberwacky, a Web chatterbot that took 3rd place in the 2003 Loebner Prize.
The bot is different from earlier systems, such as the classic Eliza, in that it learns from the aggregate knowledege of its users: “It stores everything everyone has ever said, and finds the most appropriate thing to say using contextual pattern matching techniques. In speaking to you it uses only learnt material. With no hard-coded rules, it relies entirely on the principles of feedback. This is very different to the majority of chatbots, which are rule-bound and finite.” While it wouldn’t pass a Turing test, Jabberwacky returns some very interesting responses, and can respond in many different languages. I find the idea of a collectively authored bot very compelling — sort of like Eliza as a globally authored-and-corrected FAQ. The winner of the 2003 Loebner Prize, Jabberwock actually responds in a much more “human-like” way, even though its system is more like the standard “Eliza-style” AI, with a “brain file,” produced by the system’s creators and a good collection of stock phrases used to “fudge” when the system can’t come up with a good match. The 2004 Loebner Prize competition final round will take place in September in New York.
February 18th, 2004 at 8:12 pm
The actual winner was a bot called Jabberwock. Unfortunately the link to the bot seems to be down. However, the transcripts of the judges interrogation of the 2003 competition bots is active and very interesting.
February 19th, 2004 at 9:55 am
Last year Salon.com had a couple of interesting articles on the Loebner prize, Artificial Stupidity – Parts 1 and 2, describing the history of the Loebner competition, the mainstream AI community’s reaction to the competition, and discussing Richard Wallace, iconoclastic computer scientist, creator of AliceBot (a two time Loebner prize winner) and AIML (the AI markup language).
Scott, what sorts of discussions on AI are you having in your class? I’d be curious to hear more about how how you discuss AI in the context of electronic writing and your students reactions.
February 19th, 2004 at 11:43 am
Why can’t the language understanding of a bot be combined with Interactive Fiction? The thing I hate most about IF, is the limitations on its understanding of the user’s instructions. I would rather communicate my instructions to a text adventure as normal sentences rather than the specialized code normally used. Can’t something like this be used to give a greater playability to IF?
February 19th, 2004 at 12:23 pm
Definitely! That’s been a primary goal of Facade, which although animated, has some similarity in form to text-based IF (and many differences too). It’s been a lot of labor to get conversational NLP up and running, and it relies on some heavy duty existing technologies (e.g., the Jess rule language which runs under the Java virtual machine), so it’s not a huge surprise we haven’t seen it yet in existing IF. Michael and I hope that the technology and techniques we’ve developed in Facade can be integrated into future IF projects / engines, either by us or others.
Furthermore, on the topic of bots and the Loebner prize, the narrative-oriented techniques we’ve been developing (beats) could apply very well to making chatterbots less “chatty” and more motivated. Bots could have things they want to talk about, and pursue them over time, which should increase player’s perception of their intelligence.
February 19th, 2004 at 1:02 pm
Sounds great Andrew, I can’t wait to play it! When is the planned date of release?
This type of interaction is sorely missed in many game types from IF to CRPG. It seems that the technology to allow it has been in place for some time but nobody has applied it to games until now. It would definitely revolutionize the game world to have these bots driving gameplay by a more direct, language-based interaction. At the RPG Codex forums we had a similar discussion about a system to use game variables that drive dialogues along diferent paths in CRPGs. Traditionally dialogue has been provided in a tree-branch system which sometimes varies based on player appearance, reputation, etc. But maybe an approach based on AI similar to that used by Facade would allow for more elaborate interactions.
February 19th, 2004 at 2:37 pm
The planned date for release, an interesting question… Sometime in the summer.
When we first designed the natural language understanding architecture for Facade, I thought we’d come up with something that would work well enough for Facade, but not necessarily have any legs. But surprisingly, our rule language for mapping surface text into discourse acts turns out to support authoring surprisingly robust NLU rules (though rule authoring still requires signficant authorial effort). And our discourse management framework for deciding what to do with discourse acts in different discourse contexts turned out to be robust enough to let us get away with overly promiscuous rules (tend to recognize many discourse acts in a given utterance) while still producing sensible responses from the characters. I’d be curious to apply the discourse act approach to other game domains – I suspect that a discourse acts approach such as we used in Facade (focused on what language does or socially accomplishes – e.g. a flirt, an agree, etc.) will be more appropriate for many games than a semantics approach (focused on what language means – e.g. “The cat sat on the mat” refers to an instance of a certain species of mammal that is in a posture of inactivity, as well as maintaining a spatial relationship on-top-of to a textile object often used to cover a floor).
There’s a huge amount of work still to be done on creating appropriate natural language processing frameworks for games and interactive stories. Such frameworks need to balance the need for authorial control with deeper processing that does more of the between-the-lines inferrencing needed to deal with language.
February 23rd, 2004 at 12:28 am
Since neither Michael nor Andrew mentioned it, shy guys that they are ;), I’ll remind everyone that Facade is an IGF Finalist this year, and anyone will be able to play it at the GDC exhibit hall next month.
(disclaimer: I am an IGF judge)
February 23rd, 2004 at 10:30 pm
Michael — on my student’s reactions and the place of AI in the class — The Internet Writing & Society class is essentially a course that could be called “Topics in Network Textuality.” It’s not a new media writing class per se, more a course in the social aspects of the network. Class discussions of issues like identity, artificial intelligence, materiality, network ubiquity, mobile computing, legal issues and copyright are guided largely by the texts we’re reading: Unspun: Key Concepts for Understanding the WWW, edited by Thom Swiss, Turkle’s Life on the Screen, Rheingold, Hayles, and Lessig’s latest books. The students are writing position papers as blog posts in response to the readings, are constructing an Internet use survey, and writing a term paper on a topic of their choosing.
Generally my students were amused with but not too impressed by the bots. In their responses, many of them reported that although they imagined that an AI would be able to pass the Turing test before too long, they still wouldn’t turn to one for psychotherapy. We did discuss the programs that we looked it in comparison to the interactive fiction which some them had seen earlier in the Intro to New Media Studies course. One student observed that even though IF games have a more limited vocabulary, in some ways those made for a more convincing experience, because once he had learned the basic limits of IF vocabulary, he understood the how to interact with it, even if the types of exchanges that one could have (go west, pick up, drop, etc.) were linguistically unsophisticated (non-human-conversation-like). We also discussed the two different approaches in Jabberwacky and Jabberwock in the context of Searles’ Chinese Room metaphor — talking about the programs not as “actual intelligence” but as “programmed to appear intelligent.” There was general consensus that Jabberwock’s more sophisticated “fudges” — its ability to switch topics, stammer, use different kinds of speech acts to cover its limitations/inability to actually communicate — made it seem more “humanlike” than the others that we looked at. Of course, neither Eliza, Jabberwock nor Jabberwacky came close to convincing anyone in the room that there was a person on the other end of the interface. I of course mentioned Facade which, like the majority of your public, I’m eagerly looking forward to playing with.
February 20th, 2004 at 2:22 pm
Each year, a prize is given to the AI program that best represents itself as a sentient human over a communication channel. Many people remember / have talked to Eliza (way back when), but these conversation bots are a step…