October 18, 2007

More on GTxA the Show

by Andrew Stern · , 1:49 pm

I thought the opening of the Grand Text Auto group show at the Beall Art+Tech gallery went very well, especially when you consider how elaborate the three large installations were. All of the artworks worked, installations and otherwise! And they were physically arranged to fit nicely in a somewhat small space, without feeling overly cramped. Thanks again to all those who put so much time into organizing and setup. (I wasn’t one of them. ;-)

As I hoped would happen, I found it really interesting to experience our various literary and ludic works together in one place. I think they play off each other in both obvious and non-obvious ways. For example, you’ve got [giantJoystick], an overgrown re-creation of the Atari 2600 controller, which thirty years ago was probably the first exposure to digital media for all of us, side-by-side with a proto-Holodeck, AR Façade, as futuristic a working prototype of digital fiction I’ve ever played, a fully-working version of such perhaps we’ll see when we’re all old and gray. You’ve got Screen, a story constructed as a massive black rectangle with flying white text floating in the middle of the gallery, next to Implementation, a story constructed as a massive collection of small white rectangles and black text, mounted on the gallery walls, the walls outside the gallery, and the wide world beyond.

I also really like how the blog literally has a presence in the gallery, with gallery visitors’ guest entries showing up here on the blog, as well as being printed out and hung on the gallery walls.

The symposium was good too, although I felt just the start of what could be a bigger and better one. We each presented our current lines of work and concerns (the text of mine is at the end of this post), followed by some freewheeling discussion (blogged by Mark at WRT), responding to thoughts and questions from the audience. That was great, though in addition, in a future gathering, I’d love to have each us directly address and discuss a set of focused topics. (Noah brought a few seed topics for us, but we didn’t really get to them.) Some of us did manage to admit various feelings of guilt about this or that, so it was therapeutic in that way. ;-)

Here are a few comments on Augmented Reality Façade, which I played for the first time at the show. (Don’t miss Scott and his Unknown collaborators’ experience with it.)

I mentioned to Blair MacIntyre and Steven Dow of the GVU Center at Georgia Tech, where the AR was put into AR Façade, how remarkably suited the original Façade was for augmented reality adaptation! The drama takes place in a single location, with only two characters, and is primarily dialog driven, making minimal use of physical action, other than the player walking around. These were all design decisions we had made long ago, for reasons other than AR adaptation (see our initial 2000 paper for details), so it was really interesting to see how well they applied to AR.

For me, both the highlights and problems of AR Façade were in the interface. A highlight: being able to reach my arm out and put it around Grace’s shoulders, to hug her, was really cool. That was a big improvement over clicking on Grace to hug her in the original desktop verison. I thought Blair, Steve and the rest of the AR team did a good job merging the animation of Grace and Trip with the real-life environment; the head tracking worked pretty well, I didn’t feel any vertigo or motion-sickness.

The primary problem I found with AR Façade, it being a lab prototype making its first public showing with understandably rough edges, is in the delay between the player speaking her dialog and the dialog actually being transmitted to Grace and Trip. In AR Façade, a human “wizard” is hidden behind a curtain, and via a mic and surveillance camera is listening to the player speak and watching her perform. The wizard, in real-time, manually enters the text of what the player is saying to the Grace and Trip, as well as actions she takes, e.g., hugging. Much of the time though, the text was entered to the AI 5+ seconds after being spoken — the time it takes for the wizard to listen to it and type it in, plus any required simplification of the phrasing. Recall, the original AI (basically unaltered for AR Façade) can only handle about 8 words at a time, as frequently as every 5 seconds or so; of course, people often verbally speak faster and lengthier than that. (Also, at the time of the opening, the wizards were mostly newbie gallery docents just learning how to be wizards; over time they will surely get better at it.)

So, AR Façade suffers from a third layer of potential communication breakdown, the wizard intermediary, on top of Façade’s original two potential breakdowns that I outlined in a past post on NLU interfaces, namely 1) at times the NPCs may not literally understand your words due to parser/NLU limitations, and 2) when they do understand your words, at the particular moment when you spoke, they may be unable to respond or are only able to respond in a limited way, for a variety of reasons, such as content limitations.

Additionally, as a technology-based artwork situated in a gallery, it is cumbersome that AR Façade requires a human wizard to make it run, which arguably diminishes the AI appeal of the piece, even though the original AI is still in operation. (Voice recognition doesn’t work well enough to replace the wizard, especially for emotionally-rich speech.)

Still, all that said, AR Façade is a remarkable combination of technologies and designs, a real taste of what the Holodeck could be like. Like Façade, it is a research/art experiment, for which many people are willing to forgive the rough edges in order to experience a new form of digital fiction. And it’s free to play, until mid-December, so go check it and the rest of show out!

Finally, here is the text of my future directions presentation at the symposium. I had pre-written this as a blog post, in the spirit of the show being borne from the blog. (Moved to its own top level blog post. -ed)