Grand Text Auto

How can people embedded in large, for-profit companies find a way to direct some of the present expertise and resources to make a positive difference? In this talk from the Media Systems gathering, Donald Brinkman describes how he has worked to do this within Microsoft. In particular, he describes Microsoft’s role in enabling the digital projects engaging the AIDS Memorial Quilt that were also discussed in Anne Balsamo’s talk.

In an emerging, interdisciplinary area, how can those getting started understand the possible paths forward? One way is through looking at the (often wending) paths taken by pioneers. In this week’s Media Systems talk from Pamela Jennings (recently-appointed director of the Center for Design Innovation) she provides insight into how she became a field leader, as well as into the work she has done to help the nascent field find its way. Jennings has been a key figure in much of the field development of recent decades, including recent successes such as the NSF’s CreativeIT program and the founding of the SEAD network. (Some related reflections can be found in the Media Systems talks of Brenda Laurel and Janet Murray.)

I am happy to announce that we are publishing the final four videos from the Media Systems gathering — and that the final report, “Envisioning the Future of Computational Media,” is now available through print-on-demand!

At the just-concluded 2014 Game Developers Conference I organized and spoke in a session titled, “U.S. National Investment in the Future of Games?” I was joined by William S. Bainbridge (Program Director for the National Science Foundation), Elaine Raybourn (Principal Member of the Technical Staff in Cognitive Systems at Sandia National Laboratories, on assignment from to the Advanced Distributed Learning Initiative, Office of the Deputy Secretary of Defense), and Jason Rhody (Senior Program Officer for the Office of Digital Humanities in the National Endowment for the Humanities). I’m posting here my slides and notes from the session introduction and my talk, the latter of which focused on three recommendation areas from the Media Systems final report that would benefit from joint effort by federal agencies and the game development community.

Today we are publishing the final report of the Media Systems project — including a set of 12 key recommendations for building the future of computational media.

This report is the result of bringing more than 40 field leaders together for a meeting made possible by an unprecedented set of organizations: the U.S. National Science Foundation, U.S. National Endowment for the Humanities, U.S. National Endowment for the Arts, Microsoft Studios, and Microsoft Research. We followed the meeting with more than a year of additional analysis, conversation, and writing.

What can transform a field? Janet Kolodner argues, in her talk for the Media Systems gathering, that individual projects are not enough. But programs do potentially offer a route to transformation — by being larger, integrative efforts.

While critics may now say a film’s action scenes “seem like a video game,” it doesn’t just seem like film and games are borrowing from each other. It is not only happening stylistically, but also at a deep technical level. At the Media Systems gathering, Chad Greene from Microsoft Studios discussed how the common basis of computation is leading to transformations in film and games, enabling borrowing between the two, as well as both borrowing from academic research.

Bill Gaver’s group — The Interaction Research Studio — does design as a means of research into people and technology. At the Media Systems gathering he used examples from their work to illustrate a number of approaches to one of our major topics: “… guidance and evaluation methods from arts, design and the humanities.”

In computer science, we often guide and evaluate work by metrics such as efficiency (of execution, of task performance, of maintenance, etc). But such metrics do not make sense for many types of computational media work. Fox Harrell’s talk at the Media Systems gathering, “Matching Methods: Guiding and Evaluating Interdisciplinary Projects,” suggests that, rather than there being one answer to evaluating computational media research, part of the work is in identifying values and goals, which can then point to the methods that might be appropriate.

What would it mean to have “big” projects — bigger than a single investigator, lab, or even institution could handle — that are not arranged by science and engineering concerns, but by cultural concerns? In this talk from the Media Systems gathering at UC Santa Cruz, Anne Balsamo gives the shape of two major, ongoing digital humanities projects of this sort: the AIDS Memorial Quilt Browser and FemTechNet.

We can build a computer system that could generate a surprising event, and we can build a computer system that would recognize it.

When Mary Lou Maher said these words at the Media Systems gathering at UC Santa Cruz, she wasn’t talking about hypothetical systems working in sterile domains like block stacking. She was talking about the already-demonstrated power of computational models in rich areas of human creativity, like music and humor… creative domains in which strong expectation is key to our experience.

As Ian Bogost explains in this video from the Media Systems gathering at UC Santa Cruz, his work in procedural rhetoric is not “operationalizing” particular rhetorical tropes (the way Nick Montfort’s work operationalizes elements of Genette’s Narrative Discourse) but rather:

It’s a theory or a design philosophy. It’s a way of making things. A way of thinking about the process of translating systems in the world into representations of those systems in the computer…. It gives you a framework through which to ask questions about what a particular situation might demand.

Among those doing computational media work, the concept of “operationalization” — as Nick Montfort discusses in this video from the Media Systems gathering at UC Santa Cruz — involves the formalization of theories from the humanities, arts, and social sciences and the implementation of these in a computational system, where they can be effective in new ways and “tested” in certain senses. This has proven a very powerful approach. For example, the entire field of 3D graphics could be seen as operationalizing arts knowledge about visual perspective and other knowledge from the visual arts. Or, more specifically, Facade (generally seen as the first interactive drama) is explicitly operationalizing concepts from arts and humanities theories of dramatic writing.

In this video from the Media Systems gathering at UC Santa Cruz, Alex McDowell — one of the most influential designers in the world today — talks about how computational media are transforming storytelling. We are moving from the linear, auteur-oriented storytelling model of the printing press and industrialized film production to a collaborative, non-linear approach he terms world building.

One goal sometimes pursued by interdisciplinary programs is to move beyond the arbitrary divides in knowledge represented by the schools and divisions of universities. One way of accomplishing this is to report to multiple deans, or to no dean at all (perhaps directly to the provost level). This sounds appropriate in theory, but at the Media Systems gathering we discussed the difficulties such models of interdisciplinary organization have presented for pioneering programs such as Animate Arts at Northwestern and Arts, Computation, and Engineering (ACE) at UC Irvine.

At the Media Systems gathering Janet Murray made a clarion call for deeper fundamental research in computational media, moving forward interdisciplinary understanding through the creation of new genres:

There has to be someplace where you say, “How do we reconfigure knowledge?” Because that is what happens when you have a new medium of representation, as with the printing press. And we’re not making fast enough progress there, because nobody’s getting rewarded for it, nobody’s being paid to do it.

When asking how the humanities, the arts, and computer science can come together to create new possibilities for media making and understanding, we might choose to be purely theoretical. But why would we do this, when we have decades of experience to draw upon?

The Media Systems gathering last summer brought together a remarkable group of participants from digital arts, digital humanities, and media-focused computer science. It was convened by a historic group of partners — the National Science Foundation, National Endowment for the Humanities, and National Endowment for the Arts (in their first-ever collaboration) together with both Microsoft Studios and Microsoft Research. I was amazed and pleased to spend three days with the group we assembled, discussing important topics for the future of computational media.