Grand Text Auto

A bit of exciting project news… We’ve just cast two writer / director / performers to be the voices for Grace and Trip, the two computer characters in our interactive drama Facade. We have been fortunate to find two talented members of the Chicago-based experimental / interactive theater group, The Neo-Futurists, Chloe Johnston and Andy Bayiates.

For the past few years Chloe and Andy have been members of the Neo-Futurists’ long-running signature show, Too Much Light Makes The Baby Go Blind. If you’re ever in Chicago, be sure to see it, it’s an ever-changing, inventive, searingly honest and, of course, funny, sixty minutes of experimental theater. Chloe and Andy have also written, produced and performed other plays with the Neo-Futurists.

The post Responsive Narratives (and its comments) raises the question of whether there lies anything in between brute force authoring approaches and building a human-level AI. This question is important not just for interactive drama, but for Expressive AI (AI-based art and entertainment) in general. In a brute force authoring approach, the artist lovingly hand-crafts material (e.g. animation, text, images, etc.) for every possible context, for all possible interactions. In the AI-complete approach, the artist somehow describes their intention at a very high level (e.g. the high-level motivations of characters), and the system auto-magically grounds this high-level description with concrete representations for different contexts (e.g. generated animation, generated text, generated images) and for all possible interactions. There are good reasons to seek a middle ground besides the current technical impossibility of AI-complete approaches.

Surfing around I came across a new book to be published this Fall, called “Tomorrow’s Stories: How Responsive Narratives Will Change Storytelling”. The author is Andrew Glassner, perhaps best known as the creator and editor of the Graphics Gems book series on computer graphics. According to his bio, he’s also written a novel, directed a short animated film, and designed some interactive fiction / game prototypes. I heard Glassner opine on the “Future of Fiction” panel Noah co-organized at SIGGRAPH 2000, and remember him as thoughtful and articulate.

His forthcoming book “addresses the fundamentals of how and why successful games and stories work. It is the first book to give a reader a practical understanding of both the structure of story and the structure of participatory gaming. This knowledge helps the reader see why and how today’s models of interactive fiction succeed and fail, and provides a foundation for developing new storytelling art forms that harmoniously integrate interaction and narrative.”

He sees limitations in branching narrative / hypertext approaches. “These ideas, and their cousins, have been tried time and again in the marketplace but have yet to achieve mainstream success. There are some very good, specific reasons for this lack of success, and those reasons can be found by going back to the basics of what stories and games are, and how they work.”

Perhaps it’s obvious to say, but the arcade still had it all over any other public context when it comes to interface innovation. Except perhaps the more elaborate setups at science museums and in interactive art installations.

For example, it may not be obvious in my last post, but the interface for that game is a real drum, not the tap pads of DDR or of the Western-style drum games. The drum in this game can make a range of sounds as players hit it on the main surface, on the rim, with different degrees of pressure, and so on. The way you play the drum makes a difference both on the level of music and on the level of gameplay. That’s why I stopped to take a picture of it.

I also stopped to take a picture of this direct-manipulation coffee cup (the text on it reads “Drink me. Drive me.”). The cup is part of a network racing game of a pretty standard sort — except for the interface.

This fine story from CNN relates some findings from a study, done under the auspices of the Pew Internet & American Life Project by Steve Jones at UIC. Jones found that gaming didn’t gnaw away at work or social activities, but coexisted with them, as gamers multi-tasked; it appeared to be a social activity itself, and not as male-dominated as it has been made out to be.

… and where am I? Yes, Scott McCloud has completed a “mature” new comic called The Right Number, and placed a smidgen of it online for free. The rest can be bought for $0.25 (after you buy a $3 debit card.) It’s a interesting-looking piece which I’ll read as soon as I run out of free things on the Web to read.

I do appreciate McCloud putting his mouth where my money is; as an advocate of micropayments, who sees such a system as important for supporting independent creative types, he’s doing well to attempt to get this process going. Coming from micro-cost comic culture, the system does seem to make sense in some ways. But like Noah, I’m not going to get behind such a system until there are ways to provide free public access, the sort of access that libraries now provide for our antiquated book medium.

Wandering around Shibuya, and into the big Sega arcade, I found people playing a traditional Japanese version of the drumming version of Dance Dance Revolution.

Later, outside the train station, a group of kids were playing bongo drums. The crowd cleared a circle around them so another group of kids could take turns pop lock dancing to the beat. My first experience with hippy/hip-hop fusion. I’m liking being back in Tokyo.

Lately I’ve been wondering how authors of non-linear interactive experiences develop their designs. Specifically, before coding, what are the ways people describe, represent, write down, their designs? Creating a written description of a system gets particularly tricky when the system’s behavior is more complex than what a state machine or graph can represent.

My friend Martín Hadis has just written and coded up a new site on the life and works of Jorge Luis Borges. (Martín is coeditor of Borges Profesor, a collection of Borges’s lectures on English literature.) The new site is available in Spanish and English, offers not only bibliographical information but also a Web directory and suggestions on how to begin reading Borges’s stories and poems – and it should continue to grow in the future, hopefully to incorporate more information about Borges’s influence in the digital realm.

Second Life, “a rapidly growing and constantly changing 3D online society, shaped entirely by its residents”, officially started on Monday. Read the press release here. Free 5 day memberships are available; beyond that, it’s $15 / month.

I’ve yet to experience Second Life, but from what I’ve read, I’m pretty excited by this online world. For my taste, I’m much more interested in “real life” style online worlds, such as Second Life or There.com, versus fantasy online worlds such as Everquest. (Although, the screenshots of Second Life and There do seem a bit like “J.Crew” world to me. And, I have to admit, Star Wars Galaxies, which launches tomorrow, has some strong appeal. Hard to beat that brand I guess.)

The program for the 1st international digital games research conference, LEVEL UP, this November in the Netherlands, is now online. It’s quite a program.

Harold’s post inspired me to post on what I see as the modes or genres of AI-based art. The modes described here are not mutually exclusive; a single piece may simultaneously explore multiple modes or genres. I would love to hear any comments describing a new mode (with an example) or an alternative categorization scheme.

In typing up a recent comment to add to the neverending thread on the category “Games in Virtual Environments” I realized that I know very little about which traditional games are played in different cultures. Well, not very little, perhaps – but I don’t exactly know what I do and don’t know about this. Certainly, I’m aware that lots of people will know what chess is, and that all sorts of consumer products (games included) have made their way to other markets, but I think I have a much better sense of what literature and art is known cross-culturally than I do when it comes to non-computer games.

After a discussion with some friends about spelling bees (which seem to have originated in America), we realized that this sort of contest would be absolutely absurd in Japan, where words are spelled exactly as they are pronounced. In crossword puzzles there are plenty of national differences; sources indicate that we find “American, cryptic, quick, freeform, coded, French-style and clues-in-squares crosswords.” But it isn’t only within language games that we can find these sorts of variations.

As we have been discussing artist programmers and meaning machines on grandtextauto, I sent an email to Harold Cohen, creator of AARON, asking if he’d like to share his thoughts on the topic. To my delight, he wrote back with the following comments.

I wrote my first program early in 1969, at which point, I’m sure you must realize, the option of using an existing package as opposed to writing your own program didn’t exist — there weren’t any packages. If there had been I suspect I’d never have thought computing had anything to offer me.

That reflection leads me to one rather obvious comment; I don’t see anyone saying why they got involved in computing, what they wanted from it. And in the absence of any driving personal need, questions about whether one needs to program or not seem very arbitrary.

The Digital Storytelling Festival is underway in Arizona, blogged here and on Fray. Brenda Laurel presented a project called Backstory she and others are doing at the Art Center College of Design in Pasadena, where she is chair of the media design program. Mark Bernstein calls Backstory “a really ambitious effort … to harness the storytelling power of teens for social change.”

More reading on the topic of artist programmers in an essay, Are the days of the lone programmer numbered?

Some new books: The nature of computer games: play as semiosis, by David Meyers. Ernest Adams, a longtime contributor to Gamasutra, has teamed up with Andrew Rollings to publish a book on game design.

I just spent the last week in Stockholm as an invited opponent on a licentiate thesis in interactive drama. While there I was able to visit with a number of folks in the Swedish Institute for Computer Science, the IT University of Kista, and the mobility studio of the Interactive Institute.

As we get serious about studying new forms of art and new sorts of games, the question of how to draw categories for consideration arises. Defining categories isn’t just some tedious Scandanavian pastime; it’s also a way of figuring out why exactly we are more interested in some stuff than in some other stuff. Do we like certain types of literary works because they are presented on a computer, for instance, or because they require effort from the reader, who participates in determining what is read? (This is the implicit question asked in Cybertext and in some of Espen Aarseth’s earlier writing.)

Espen, Stephen Granade, and some others have been discussing the virtues of the category “Games in Virtual Environments,” as an alternative to “computer games,” here on Grand Text Auto. The “virtual environment” is a feature of interest to me (it’s part of my definition of interactive fiction, while “game” is not) and also, I think, of interest to Stuart. In fact, I do think there is something more interesting about richly world-simulating computer games than one sees in computerized versions of card and board games. But this idea for a category also raises some questions.

The Iowa Review Web this month features Stuart Moulthrop. In addition to the official publication of Pax (mentioned in this space earlier) the feature includes an interview with Stuart conducted by yours truly. It was this conversation with Stuart that really got me thinking about the notion of “instrumental texts” and I suspect it’ll prove similarly thought-provoking for many. Here’s an excerpt to get you started…

Noah Wardrip-Fruin: Talk with me about the idea of an “instrumental text”…

We’ve had an array of wonderful comments in the two previous posts on artist programmers. (I just added a lengthy comment with a bunch of new links.)

Another facet of this debate: What happens when artists and programmers collaborate? The issues are more than just the potential cultural divide of freaks vs. geeks, but also the (perhaps unpleasant) issue of artistic credit. I’ve heard more than one story of a team of people working on a new media art piece “led” by a “primary” artist, who effectively takes all of the credit for the piece, when those who did the actual programming deserve at least as much credit for the success of the work. Sound familiar?

Andrew raises the question of whether artists should program. The answer is yes. Here’s why.

Computers are not fundamentally about producing 2D visual imagery, video, or 3D models (everything taught in the typical into to electronic media classes).

Computers are not fundamentally about responding to the input of a user/player/interactor (computer-based interactive everything).

Computers are not fundamentally about controlling motors, lights, projectors, or other electro-mechanical systems (installation art, robotic sculpture).

Computers are not fundamentally about mediating signals from distant locations (telepresence).

I caught up with Michael in Atlanta on my way back from ACH/ALLC. Michael and I got to talking about (among other things) programs to generate stories, poems, and other creative texts. He mentioned that he benefits from looking at thoughtful symbolic architectures for this sort of thing, such as Minstrel, but really finds nothing of interest in the statistical approaches taken by systems like Gnoetry and Ray Kurzweil’s Cybernetic Poet.(Correct me if I’m misrepresenting you, Michael!) On the other hand, I do think there’s something to the way that these statistical systems manage to knit together a new sort of voice – something that makes the process of the generator interesting to consider – and I’ve written about this a bit in an article that is forthcoming in The Cybertext Yearbook 2003. Interestingly, a paper by Greg Lessard at ACH/ALLC described a limerick generator he had developed, called VINCI, so this topic is still a current one.

Our discussion led to questions about how, in this specific case of creative text generation, symbolic/rule-based AI can be integrated with statistical AI to achieve some of the advantages of each. …

In the Narrative as Virtual Reality comments thread, I’ve suggested rather strongly that those who plan to discuss hypertext should read the work of Ted Nelson — as both the term “hypertext” and the ideas it describes come from his writings.

Of course, such suggestions often lead to the question, “Where can I read Nelson’s writings?” Unfortunately, they aren’t found in the local chain bookstore, and perhaps not even in the local research library. So I’ve put together a few pointers.

In the comment thread of Narrative as Virtual Reality, Lisa asks, “Why would an author *want* to yield the authorial control of a piece to some sort of AI engine?”

That’s a really good and interesting question. To me the issue of who has control (or, similarly, agency) of an interactive artwork is primary. There seem to be at least 3 parties who could be in (or share) control of an interactive artwork: the original author, the user, and the work itself (e.g., the AI).

An issue we want to address on grandtextauto, as we discuss the practice of making computer-based art, literature, poetry, drama, etc., is the question of artists as, or needing to be, programmers. How can artists to learn to be programmers? Why aren’t more artists programmers? Can tools simplify or take the place of programming? What kind of programming languages are amenable to artists? At what point do artists need to be programmers? Isn’t it enough for artists just to collaborate with programmers? Are programmers artists?

This topic is too big to address in just a post or two – so we’ll be addressing it in blog posts over time. Please join in with comments, ideas, and opinions.

Melbourne, Victoria, Australia. May 19 – May 23.

DAC is my “home conference” and the only place I know of where art, literature, music, performance, and games for the computer come together with serious scholarly discussion of new media. I’ve presented work there since the second time they put it on, in Atlanta in 1999, and the conference has been host to an array of great discussions and ideas, as well as leading me rather immediately to collaborate with Noah and with William Gillespie. Thus, I was determined not to miss DAC this year, even if it meant traveling halfway around the world in a pet carrier. Fortunately, it wasn’t that bad.