February 26, 2005

Beyond Procedural Literacy

by Noah Wardrip-Fruin · , 12:08 am

I didn’t have a chance to comment on Michael’s Why Johnny Must Program post back in January. I started to write a comment earlier this evening, but then realized I should just make a new top-level post. In this post I’m going to agree with Michael about procedural literacy, disagree with him on the same point, argue for the unavoidable synthesis of my two opposing points of view, and then make the case that we need another layer on top.

To put that a bit more clearly, the short version of my argument is that procedural literacy is only one of three types of education around these issues that we should be offering students of digital media (students focused on scholarship and/or creation of computational media).

But before I get into all that, I want to begin a bit oddly, by offering a response to Scott’s first comment that is a slightly different version of Mark’s comment.

Michael, in the draft paper linked from his post, wrote:

Without an understanding of how code operates as an expressive medium, new media scholars are forced to treat the operation of the media artifacts they study as a black box, losing the crucial relationship between authorship, code, and audience reception. Code is a kind of writing; just as literary scholars wouldn’t dream of reading translated glosses of work instead of reading the full work in its original language, so new media scholars must read code, not just at the simple level of primitive operations and control flow, but at the level of the procedural rhetoric, aesthetics and poetics encoded in a work.

Scott wrote:

While a game developer who sets out to write a game without learning how to program is much like an author who sets out to write a novel without bothering to learn to read and write, I think that a games scholar who does not know how to program is more on the level of an English professor who claims that she is able to analyze a sonnet without having studied theoretical linguistics.

Mark responded:

Being fluent with programming languages and tools isn’t quite the same as having an extensive background in the underlying theory. I’m drawing a blank trying to come up with a good analogy in the Sonnet case (there may not be one?), but I’d compare it to someone studying theater (the good old-fashioned non-interactive drama). It’s possible to study theater merely by watching a lot of plays… However, it seems like it would be useful to be able understand what goes on in putting on a play—the whole mess of stagehands and props and lighting and scripts and casting that goes into the final product.

I’d like to offer a similar response, but one that shifts our attention to a more obviously technical, screen-based medium. Procedural literacy is not to software/game analysis as theoretical linguistics is to sonnet analysis. Rather, it’s more like knowledge of film production (lighting, editing, directing, continuity management) is to film analysis. If you have no clue about the authoring process for the medium’s primary component (sequenced shots for film, procedures for software) you’ll be limited to studying things like audience perception (which isn’t uninteresting, but shouldn’t mark the limit of inquiry) and anything you try to say about the work itself will be prone to embarrassing gaffes.

That said, Scott’s point is well taken that we can’t wait to analyze Halo 2 until it is no longer a commercial product and is perhaps eventually released open source. While there may be interesting ways to read code, we don’t often get our hands on the code, and what we’re really talking about at this point is reading procedures. That’s why I say it’s something like knowing about how film editing is done, as a process, rather than being able to actually see what did and didn’t end up on the cutting room floor. Michael points to the same thing when he mentions Kurt Squire’s procedurally-focused reading of Viewtiful Joe. (I assume Michael is referencing Educating the Fighter, which breaks down processes with no indication of access to the code.) Michael may use the term “code” – but when he expands the statement he refers to “the procedural rhetoric, aesthetics and poetics encoded in a work.”

(By the way, does anyone reading this know of a single example of interesting code-level analysis for code not written to be analyzed? I’ve heard the stories that people have read ideological assumptions in old AI systems from the fact that data structures were given names like “beliefs” when they could have been called “myTable” — but I’m not sure I ever got a reference for one of these readings. I think the only code I know of that’s been read critically is the code for software art and “codework.”)

Of course, there’s also a potential being spoken of in Michael’s paper. What happens if we graduate a generation of digital media scholars who (1) know digital media has a history and (2) are procedurally literate? Perhaps then we might have people who are able to do things like interpret the creation of new programming languages — from influential languages like LISP to (thus far) single-work languages like ABL. They might be able to interpret the histories from which these languages arise, the assumptions they embed, the problems they were designed to express elegantly, and the blind spots they encode. These scholars could also help us understand what constitutes an interesting or virtuosic performance with code. (Why is writing a single-room Inform piece notable? Perhaps because it runs against the fundamental design pattern the environment was created to support. Next question: why is implementing a chess program in Inform something anyone would do? And why would I ask that?)

As becomes clear, I want to have my cake and eat it too. I want to agree with Michael that “it is not the details of any particular programming language that matters, but rather the more general tropes and structures that cut across all languages.” I want to agree that our goal is to educate scholars and artists that can, by being generally procedurally literate, understand the “procedural rhetoric, aesthetics and poetics” in a work even without access to the code or details of the development environment. I also want to disagree strongly. I think there are important parts of digital culture, and particularly digital media, that we can only understand by knowing the details of particular languages and environments, such as those discussed in the paragraph above. For educating students, I think both are important goals — general procedural literacy and nitty-gritty familiarity with the tools and circumstances of writing and running code. And luckily one can’t actually achieve understanding of either the former or latter without the other. I’d say, rather than be embarrassed that students have to learn the details of particular languages in the process of becoming procedurally literate, we should celebrate the fact that they’re learning the first part of what will be another important body of knowledge for them — both as interpreters and creators of digital work. I know that Michael understands this (for one thing it’s apparent in the section discussing Java and Processing, what they require, and their communities of practice) but I wish it were more foregrounded in the discussion. I’d like to see it promoted to a top-level goal, on the level of procedural literacy.

What’s more, I want to add another layer to this increasingly thick sandwich. (Let’s hope the students can open their mouths this wide.) I think it’s important, when doing this kind of education for new media students, to also teach them something about how the field of computer science has viewed these topics. The digital media field is built on computer science research results, and often the research (e.g., in graphics, in AI, in hypertext/media, in NLP/G, in sound synthesis, in HCI) is ongoing. Computational media students need to know at least the basics of the vocabulary and conceptual structures used by computer scientists in areas related to their work — otherwise they won’t be able to do a meaningful review of their field of interest, they won’t be able to find and use tools related to their work that come out of computer science, and they won’t even be able to construct a web search that hits a large body of work related to their own.

I realize it’s a tall order. Michael’s approach is, I think, starting us down a road that will prove more successful (for people in our field) than earlier attempts such as Design by Numbers. I think it deserves to be a national model. But I also think we have to broaden our approach, and do even more than Michael’s course does, in order to give our students the background in this area that will help our field develop as we believe it should.