January 30, 2007
Media Machines
Earlier this month I posted an excerpt from my in-process manuscript — currently titled Expressive Processing — on the topic of process intensity. Interesting discussion ensued, I decided to post further excerpts, and I realized what excerpt I’d post next: the part of the book that comes just before the section on process intensity.
This is a central passage for the book. I’m laying out my basic perspective — what it is that draws me toward examining processes — and starting to work through the practice-oriented part of that perspective.
I appreciated the comments from people last time (they’ve already resulted in manuscript revisions) and I’ll be interested to hear any thoughts on the moves this excerpt makes.
Media Machines
A computer is a strange type of machine. While most machines were developed for particular purposes — washing machines, forklifts, movie projectors, typewriters — modern computers are designed specifically to be able to simulate the operations of many different types of machines, depending on the computer’s current instructions.
This is why a computer can simulate a movie projector: showing a set of image frames in quick succession. It’s also why a computer can act like a tape player: reading and amplifying a stream of sound data.
And it is for this same reason that computers can be instructed to act like previously-impossible types of machines. A computer can simulate a typewriter — getting input from the keyboard and arranging pixels on the screen to shape the corresponding letters — but it can also go far beyond a typewriter, offering many fonts, automatic spelling correction, painless movement of manuscript sections (through simulations of “cut” and “paste”), programmable transformations (such as “find and replace”), and even collaborative authoring by large, dispersed groups (as with projects like Wikipedia). This is what modern computers (more lengthily called “stored-program electronic digital computers”) are designed to make possible: the continual creation of new machines, opening new possibilities, through the definition of new sets of computational processes.
“Digital media” are the media enabled by this possibility. This includes web projects, like Wikipedia, and also all computer games. The first computer games were created on the earliest stored-program computers, and since then we have seen a major cultural impact from the fact that a computer can not only simulate a pinball machine but also act like game machines never seen before: a Tetris machine, a Doom machine, a SimCity machine, and more.
Personally, I am fascinated by the possibilities that digital media open for fiction. A blossoming of new models of character, story, and language is being enabled by computational processes. From computer games with epic structures to experimental interactive films, digital fictions are providing diverse experiences for a wide range of audiences. From ambitious artificial intelligence experiments to straightforward uses of weblogs and email, authors are creating digital fictions at a wide range of technical complexity. The field is already too vast to cover in a single book.
Luckily, quite a number of books have already been written about digital literature, and many more have been written about digital media more generally. However, almost all of these have focused on what the machines of digital media look like from the outside: their output. Sometimes the output is considered as an artifact, and interpreted as a literary scholar or art historian might. Sometimes the output is seen in relation to the audience and the wider culture, as an educator or sociologist might. And there are, of course, a variety of other perspectives. But, regardless of perspective, writings on digital media almost all ignore something crucial: the actual processes that make digital media work, the computational machines that make digital media possible.
On one hand, there is nothing wrong with this. Output-focused approaches have brought many valuable insights for those who seek to understand and create digital media. But, on the other hand, it leaves a big gap.
This book is my attempt to help bridge the gap. As far as I know, it is the first book focused on computational processes that comes from the perspective of media, games, and fiction (rather than software engineering or computer science). It is a first passage across the gap, and we will want to move much more weight across over time. But hopefully it demonstrates that there is something to be gained by being able to move between the gap’s two sides, being able to see the inside and outside of digital media’s machines.
At this point I do a certain amount of signposting, laying out some of the chapter organization — which doesn’t seem necessary to include here. Then the chapter continues with a new section…
Authoring Processes
Above, I said that the possibility of creating new simulated machines, of defining new computational behaviors, is the great opportunity that digital media offers. Seizing this opportunity requires a bit of a shift. It is common to think of the work of authoring, the work of creating media, as the work of writing text, composing images, arranging sound, and so on. But now one must think of authoring new processes as an important element of media creation.
In undertaking this shift, it may be helpful to think of the creation of a piece of digital media as being made up of data and process, with a somewhat fuzzy line between them. The data are mostly pre-created media elements (text, still images, video and animation, sound and music) and the sorts of things that are stored in spreadsheets (lists and tables of information, with varying degrees of structure).
The processes, on the other hand, are the working parts of the simulated machine. Some are dedicated to tasks with simple structures, such as displaying a series of video images on a screen. But many of digital media’s tasks are more complex in structure, requiring processes capable of performing in a range of different ways. Even a simple piece of digital media such as Pong has processes that define behaviors much more complex than showing a series of images in quick succession. The processes of Pong define and calculate simple rules of physics (how the ball bounces off the paddles and walls) and simple game rules (who receives each serve, how points are scored, and how winning is achieved) that, when well-tuned, can combine to create a compelling experience of gameplay — even in the face of remarkably primitive graphics.
Of course, the idea of creating media through the authoring of novel processes is not new. Tristan Tzara’s Dada cut-up technique was presented, in the wake of World War One, as a process for turning a chosen newspaper article into a poem. On a more technological level, the pioneers of early cinema had to develop novel processes (embodied in physical machinery) to capture and display their sets of image data. And, on a longer-term level, the creation of board and card games has always primarily been the development of process definitions, embodied in game rules, that determine how play moves forward.
In important ways the non-computational media processes mentioned above are like the processes of digital media: they are defined previously, but (at least in part) carried out during the time of audience experience. This is true as Tzara pulls a paper scrap from his sack, as the Zoetrope image flickers, as the poker hand goes through another round of betting, and as the image of a Pong ball bounces off the image of a Pong paddle. The processes of digital media are, however, separated from non-computational media processes by their potential numerousness, repetition, and complexity. For example, we might play a game of tennis using the rules of Pong — they’re simpler than the normal rules of tennis. But we wouldn’t want to play Pong as a board game, having to hand-execute all the processes involved even in its (extremely simplified) modeling of physics. It is the computer’s ability to carry out processes of significant magnitude (at least in part during the time of audience experience) that enables digital media that create a wide variety of possible experiences, respond to context, evolve over time, and interact with audiences.
From here the text continues into the already-excerpted section on process intensity, from there continues into Expressive AI, and then transitions to some of the reasons I want to focus on processes from a critical (rather than practice-oriented) perspective. After that I go on to discuss issues related to those in my recent receiver article. Any requests for what excerpt comes next?
January 30th, 2007 at 4:49 pm
This is great to read, Noah. To start, I have a comment on the idea that begins “Media Machines,” that computers can simulate other machines. I think there are some important further distinctions to make about the ways in which this can happen. If what I’m suggesting below is too detailed and nuanced to be explored at length in the main text, you may want to at least pursue it to some extent in notes:
I guess there are other ways, too: A machine can be a multi-tool or Swiss Army knife, for instance, or it can be a single device that is versatile and can be used in many ways, as with duct tape. I know you are working toward introducing process and the book overall here, but I think that being precise about this could be very useful in showing what thing, or set of things, is new about computing.
Thanks for this and the “Authoring Process” section, and good luck…
January 30th, 2007 at 6:44 pm
Nick, thanks for your thoughtful comment!
It won’t surprise you to hear that I agree with you entirely. I’m just trying to find an accessible, quick way to introduce people to, in particular, the first two ideas you mention: (1) a general purpose computer is general purpose and (2) this lets it be a platform for the operations of many existing and new types of media.
Obviously, lots of machines can be used for more than one thing, and some are even designed to do a wide variety of things. Do you think, while reading the above, I’ll have readers thinking, “This doesn’t make any sense. What about a Swiss Army knife?” If so, I should definitely get to some re-writing.
A footnote, or more than one, is probably a good idea. I know I’m doing a bit of a high-wire act here, trying to leap directly to the consequences (for media) of certain computer science concepts — without actually explaining the computer science involved. It’s an act I’m going to have to keep up for the whole book if the project is going to come off (unless I want to write a book with a bunch of computer science tutorial in it, which doesn’t sound as readable for the audience or as fun to write for me).
January 30th, 2007 at 8:23 pm
No, I don’t think most readers will progress on to Swiss Army knife and duct tape reveries. But computers are capable of simulating machines in a variety of ways, and I think that their abilities as universal computers and as metamedia are significantly distinct, and worth distinguishing. Someone might reasonably assume that a computer is an audio tape + motion picture player for reasons that have more to do with the Swiss Army knife than with universal computation.
That said, explaining what universal computation is doesn’t mean proving in detail that all modern computers have this capability; similarly, the metamedium quality and its connection to multimedia hardware can probably be related quickly. I imagine there may be some way to cover these issues quickly, at this point or elsewhere.
February 2nd, 2007 at 3:30 pm
Yes, of course you’re right. Universal computation isn’t the same as being a metamedium — to start with, it enables many things besides media. I’ll try to find some good way to note that, and also point people to some appropriate resources.
October 10th, 2007 at 10:22 pm
[…] s) Fossil Rivers (from BLDGBLOG) Redistricting Game Finally, Noah Wardrip-Fruin on “authoring new processes.”
Comments»
no comments yet – be the first?
[…]