February 24, 2005

Not/Non Feeling Content: The Unfairness of Programming

by Andrew Stern · , 10:13 am

Before Will Wright regales us with The Future of Content, I’d like to get in a few words, to “put down on blog” a few concerns I’ve been mulling over.

There is a serious problem, I believe, that doesn’t get enough attention: by and large, people are not enabled to create dynamic content for interactive experiences. Many people want to do it, I’ve noticed, but most are unable to. And, the path to allowing this to happen is not an easy one.

What does dynamic content mean? In an interactive medium, this means content that is alive, not dead; that reacts to and acts upon other content; content that is not immobile or frozen. Not mere skins, static models, layouts of walls and hallways, or chunks of linear text — but moving, behaving, reactive content: machines, automata, manipulators, generators, even intelligent characters and lifeforms.

First a few words on why users creating dynamic content matters, then some thoughts on solutions.

Some suggest that users creating content is the only feasible way to create rich virtual worlds for us to play in, that there is too much labor involved to put it all on the shoulders of developers. Games are getting more and more expensive to produce, and it’s only getting worse. The economics of game development is at stake.

I think that’s all true. But it’s not the most important reason this matters. The biggest reason that users need to be able to create dynamic content is that they want to, and should be enabled to. I sense this from many folks — players, writers and artists, non-programmer game developers — who when asked about it, often sort of have an air of resignation about this.

I don’t think it’s fair that programmers hold so much of the power to create dynamic, living content. Few of us are programmers; while that number will hopefully increase over time, thanks to curricula like these, it’s not clear to me that even at best the numbers will ever grow beyond a certain percentage of the population. Furthermore, it’s unbalanced gender-wise.

Let me throw in that personally, it’s not exactly in my interest to change this situation. I’m a designer/programmer, and will keep a higher status and paycheck if things stay just as they are.

But I think we risk a can-and-can’t-do type schism if we don’t work on this problem. The truth is, those who can program have an unfair advantage in interactive virtual worlds over those who can’t.

But aren’t unfair advantages such as this just the way it is? The same could be said for the real world, yes? But, ah, see, we have much more control over the laws of virtual worlds, and can do something about it. We can enable people with powers in virtual worlds that are difficult or impossible to achieve in the real world. More on this in a moment.

Some suggest the solution is to train more people to program. That’s a good direction worth pursuing. But the state of the art of programming is hard, no doubt about it. Well, it’s not too too hard to do a little bit of it, but to get to the level where you’re creating really dynamic stuff takes a certain aptitude for engineering, organization, and anal-retentiveness, not to mention years of training and practice, that too few seem to have.

We can and certainly should create better programming tools, to make it easier for people to code. That’s a good idea too, but at a certain point, I’m guessing many people just won’t want to, or will be unable to, deal with all the minute details and complex procedural thinking required.

Ultimately a more successful, even necessary approach will be to shift the burden onto the machine. To enable users to give high-level creative direction, and the computer does the hard work. That is, AI-assisted creation of dynamic content.

In its purest form, this would be programs that can write programs, guided by users. Programs that can recognize, understand and reason about abstract characteristics of complex structures — the characteristics of objects, machines, games, agents, lifeforms, characters, narratives — and can generate concrete new structures with variations on those abstract characteristics. There is research going on in this area, but from what I know about it, it’s in its early stages of course.

Less pie-in-the-sky, but still quite daunting, would be programs that can tweak existing human-made programs in limited but interesting ways, under the user’s direction. Perhaps this would be analogous to a person in real life using a wedding planner to create a custom ceremony, or a sports coach directing a team of players, or a customer requesting a special dish from a receptive chef, or filmmaker Mike Leigh collaborating with his actors.

Ideally, to be both easy enough for users to use, and expressive enough to be interesting, the directions from users would be in natural language. For example, I would describe in words what I want and how it should behave. The computer would create a first draft of it, I’d play around with it, and give additional details, qualifications and tweaks — an iterative process of refining my description of what I want.

Another approach would be to train the content in a natural way; imagine a system where you can create a conversational NPC by acting in character. You behave like what you want, acting it out, and the system picks up on and extrapolates from your direction, filling in the blanks in a common sense way.

In a sense, am I advocating “wizards” here? Perhaps, but not wizards in their existing form, which are pretty rigid, limited and scripted. In fact, I have to admit, existing AI assistants turn me off; I feel like they never really behave how I want, nor give me any where close to the amount of freedom of expression and control I need. But I believe it’s because they’re just not generative enough yet, nor can they yet understand expressive, high-level instruction, or complex detailed instruction.

By now you’re probably thinking, we already see the precursors of these ideas in today’s constructive sim games, and you’re right. Within game worlds, for example, you can create your own dynamic, operating cities, train systems, simple social groups, simple machines and automata. These are more “do-it-yourself” kits than AI assistants. Those who aren’t programmers (and those who are) can enjoy these and create rewarding dynamic content. The precursors to these include things like programming in MUDs (e.g. 1 2) and interactive fiction worlds (e.g. 1 2). We’ve discussed Gnoetry a few times on this blog — a human/computer collaborative poetry composition system.

Assuming for a moment that AI-assisted creation of dynamic content is a good direction to pursue, how do we get there? That’s a whole ‘nother post or ten to write, but in a nutshell: first, we need to create systems that can generate dynamic content from low-level direction from users/programmers, designed in anticipation of future hooks to be added for higher-level direction from non-programmers. This means creating virtual world platforms (e.g., a simulation) in which our dynamic content (machines, automata, lifeforms, characters) can exist and operate. Next, we hand-create canonical example pieces of content in those platforms, and write programs that can generate variations of these example pieces of content. Then, we build a natural interface that can take high-level, abstract direction from non-programmer users, which gets turned into partial / fuzzy low-level direction; the system needs to be able to do additional reasoning to fill in the blanks, possibly by asking additional questions to the user.

I’ll finish this thought piece by suggesting that in general, user created content, AI-assisted or not, will never completely take the place of content created by professional developers. Even in this age of “amateur” production, pros will always be in the role of making some of the best content, since they are trained, highly skilled craftspeople actually getting paid to make content full-time. Developers shouldn’t fear competion from users, and in fact should build AI assistants to enable them to join in the fun, and blur the boundaries a bit.