November 6, 2008

Creativity, Cognition & Computers in Mexico City

by Nick Montfort · , 1:01 pm

I’m here in México D.F. participating in the UAM Cuajimalpa’s 3rd International Colloquium in Creativity, Cognition and Computers, celebrating the 50th anniversary of the first computer installation in México (and Latin America). Today we had short presentations in a roundtable format.

To start us off, Rafael Peréz y Peréz (UAM Cuajimalpa) described a project, now underway, to have two MEXICA agents improvise a story together. MEXICA is a system that produces plots about the ancient inhabitants of México, using an engagement-reflection (E-R) cycle. The three core elements are emotional links between characters, tensions within situations, and operators (that is, story actions, which each have preconditions and postconditions.) THe knowledge-structure module first takes two text files, defining valid story actions and listing previous stories. It produces a knowledge base, which the next module, the ER module, accepts as input along with an initial story action. One agent in the improvisational system is set up to do only engagement (like daydreaming) while the other uses the full E-R cycle.

Brian Magerko (Georgia Tech) then offered some comments on what improvisation actually is: the real-time addressing of a problem, the creation of a performance (not an artifact) in real time. The creation itself must be a performance. The visible process, not the end product of, say, music, is important. It’s therefore useful to know what the definition of performance is, and how spontaneity is part of it. Comedy improv is interesting because it balances having many constraints with having few. Improv is neither preset nor 100% spontaneous. Improvisers practice and borrow from themselves. One of Magerko’s projects is to create a cognitive model of what improvisers do.

Atocha Aliseda (UNAM) spoke about logical abduction in automatic storytelling. Beginning with a quote from Poe’s “The Philosophy of Composition” explaining how “The Raven” was supposed to proceed “step by step, to its completion with the precision and rigid consequence of a mathematical problem.” The poem’s beginning is at its end, Poe says, in keeping with abductive reasoning: observe that the surprising fact C is true; if A were true, C would follow; there is a reason to suspect A is true. In MEXICA, abduction is used to “explain” high-tension situations that have been generated, adding to a plot by adding events before these situations.

Eduardo Peñalosa-Castro (UAM Cuajimalpa) presented a preliminary cognitive analysis of improvisational narrative, from an educational psychology perspective. Improvisation is essentially collaborative, it cannot be understood by considering the individual psychology of actors. No leader, the performance emerges collaborative as paths are closed off by lines. “Yes, and …”, minimal rules, no blocking. Knowledge must be constructed by agents and the shared mental space (common ground) must be co-constructed; this includes ideas no individual is likely to produce alone. Several complexity levels are involved in collaborative cognition, in comprehension, application of knowledge, and problem solving. He proposed to considering these elements for improvisation: emotional styles (for each agent), individual knowledge (for each), shared knowledge, and a writing model. The collaborative process described is a collaborative learning process.

Vicente Castellanos Cerda (UAM Cuajimalpa) then spoke on film studies and cognition. Borwell’s and Buckland’s comments describe the value of cognitive approaches to film. Classical theory is about film as art, modern theory about film as language, cognitive theory about film as mind. Castellanos screened part of Minority Report to show how the structure of a particular sequence is similar to the mind. Could a computer model be based on film structure – multiple points of view, non-sequential coordinates?

Luis Pineda (UNAM) spoke about how the emotional model of MEXICA might be enhanced, asking whether it could encompass more than characters. He discussed the ER model and some reservations about the use of abduction, which he sees as properly being used for explanations of real behavior rather than justifications; furthermore, do narrative stories have to be logical? Leaving some actions unexplained may be better. “Keeping the story inconsistent gives you a window into human emotional complexity.” Another challenge in improv: accepting speech acts is a prerequisite for exchanging propositional content.

Pablo Gervás (Universidad Complutense de Madrid) began with the story of his recent surprise, that he was supposed to prepare a talk for today. Given the topic, he decided not to prepare, and to improvise his remarks instead. Delivery and performance are different. What could he say that hadn’t been said before? (An important aspect of improvisation and performance; they occur in the dynamics of a group with previous speakers.) Perry and Lord’s investigation into who Homer was; finding Yugoslavian bards who look at the audience to identify whether they are bored (in which case they’ll skip ahead) or really into it (in which case they can elaborate). But he also has to decide, when telling a story of a battle, who is the good guy and who is the bad guy – based on the composition of the crowd. Could also be thought of as a driving process, with the computer engaging and person reflecting. Serial collaboration on a novel can become antagonistic, with a later writer discerning that X is supposed to be the main character and thus killing X off. How do authors’ plans conflict, and how are these conflicts resolved? Construction and delivery must be distinguished to create an improvisor, and improvisors must be able to accept feedback.

Mark Riedl (Georiga Tech) mentioned some examples of improvised narrative: made-up bedtime stories, the improv game called the herald, and a scripted play developed through improv techniques, “The 25th Annual Putnam County Spelling Bee.” Some questions: What is “real time”? How much thinking ahead is allowed? What is “good enough” as improv? What is the purpose of storytelling? Some narrative psychologists suggest that a basic sense-making process, used to understand temporal events, has become a form of entertainment as a side effect. Improv is pleasurable because we like seeing people under pressure, think that there might be a train wreck (or we might enjoy the avoidance of the same). Deliberation and improvisation are both cognition, though, with different granularities of time. Our ability to do revision and editing is different in different cases. Riedl’s work begin with AI planning. Then, a different idea: Predict the questions that the reader might have, head them off at the pass by answering them in the context of the story. This changed the planner into a system for selecting aesthetically pleasing pieces. Lastly, the question of expertise: some say it has nothing to do with intelligence, just experience and pre-compilation based on these. Experts generates more guesses and have better pruning algorithms for deciding between them. Can we build expert improvisors?

Nick Montfort (MIT), your faithful blogger, spoke last. My comments were entitled “Why Tronny Can’t Improvise” and offered a few unusual reasons why computer improvisation has proved difficult: they are too fast, they know too much, and they are too well-prepared. This makes our efforts to develop less pensive improvisers extremely difficult, because it’s so easy to have the machine act as a composer who is not acting in the moment. I showed ppg256 and suggested paring down systems as one way to jolt ourselves into creating something non-deliberative, something more improvisational.

I don’t plan for my coverage tomorrow to be nearly as extensive, if there is any. We’re going to have longer presentations that are likely to have more technical detail than today’s conversation did, and which will be less useful to relate on the blog. But this writeup hopefully gives some idea of what we’re discussing at the colloquium.

One Response to “Creativity, Cognition & Computers in Mexico City”


  1. Erik Says:

    It sounds like a fascinating conference. I am tempted to question, however, how improvisation is defined, (if seen as “the creation of a performance (not an artifact) in real time”), because the on-air and in-studio real-time painting by Rolf Harris, for example, appears to be both improvised and creating a product. So perhaps rather than improvisation is creating a performance (but not creating a product), improvisation must always (here) involve creating a performance. I’d also question whether improvisation (especially by computer) must be in real-time as conventionally understood, but that is a more minor point.

Powered by WordPress