June 22, 2003
Modes of AI-based art
Harold’s post inspired me to post on what I see as the modes or genres of AI-based art. The modes described here are not mutually exclusive; a single piece may simultaneously explore multiple modes or genres. I would love to hear any comments describing a new mode (with an example) or an alternative categorization scheme.
Procedural Portraits of Human Meaning-Making
A procedural portrait is a dynamic representation of some psychological, social or cultural process. For example, Terminal Time (Domike, Mateas, Vanouse) is a procedural portrait of the ideologically-biased construction of mainstream historical documentaries. While the constructed documentary, and the relationship between this documentary and the audience input, are the directly sensed portion of the audience experience, the mechanisms by which the system constructs documentaries are also part of the piece. Terminal Time’s architecture is a caricature of totalizing thought, of the mono-maniacal pursuit of an ideological position. How Terminal Time constructs biased histories is as important a part of the conceptual exploration of the piece as are the histories that are actually constructed.
Harold Cohen’s Aaron is another example of procedural portraiture. But, as Harold described in his post, he was not motivated by the end result, by the idea of a painting produced by a machine, but rather by the process of painting. Writing Aaron was a prolonged meditation on his own mental processes when painting, an attempt to understand his own image-making behavior.
“What the computer provided was a way of externalizing, stabilizing my speculations about image-making behavior: not only my own behavior, but what I thought I could see operating in drawings generally, and especially in children’s drawings and so-called primitive art.”[McCorduck 1991:78]
The notion of procedural portraiture is similar to longstanding research approaches in AI that build models of psychological processes, such as models of human problem solving behavior when faced with laboratory tasks. But in traditional AI research these models are taken as simulations, possessing a one-to-one correspondence with the system being simulated and thus, in a functionalist framework, identical to the system being simulated. But simulation is just one of many possible relationships that can exist between a representation and the world. Representations can simplify, embellish, highlight, subvert, satirize – procedural portraits do not make claims of identity with that which they represent, but are rather thought experiments, dynamic representations of some aspect of human activity.
Characters
AI-based characters are dynamic portrayals of humans or animals, with the focus not on realism but believability. Two of the earliest (and best known) AI-based characters are the Doctor script of Eliza [Weizenbaum 1966], and Parry [Colby 1975]. Both Doctor and Parry are chatterbots, programs that process textual natural language input and produce text output. Doctor used simple pattern matching techniques to respond to user input in the guise of a non-directive Rogerian psychotherapist. Parry portrayed a paranoid schizophrenic. Technically, Parry was more advanced than Doctor in that it maintained internal state beyond individual player utterances, pursuing its own delusions as well as maintaining a simple emotional state. Both programs demonstrated that relatively simple techniques can support the suspension of disbelief, allowing users to participate in the illusion.
Doctor and Parry are in some sense the first believable agents, autonomous characters that convey rich personality, emotion and social behavior. Contemporary work on believable agents tends to focus on real-time, animated representations. For example, Naoko Tosa has built a series of screen-based characters that respond to the participant’s emotional state. GTxA’s own Andrew Stern worked for a number of years on the Petz and Babyz projects.
Alien Presence
Characters leverage viewer expectations of human and animal forms and behaviors in order to achieve believability, making use of known character types from other media representations, such as movies, animated films or comics. Alien presences, like characters, are dynamic depictions of living creatures. But unlike characters, alien presences have novel forms and/or behavioral repertoires. While perhaps borrowing bits of form and behavior from the animal or human world, they primarily have their own perceptions, reactions, concerns and thoughts, divorced from the everyday human world. For the participant, the pleasure of interacting with an alien presence is to enter its alien world, to discover how it perceives and acts. Office Plant #1 (Böhlen, Mateas), a desktop robot that responds to the social and emotional tone of the email received by its owner, is an example of an alien presence; it offers an ambient commentary the email stream.
One of the earliest examples is cyberneticist Gordon Pask’s Colloquy of Mobiles, presented at the Cybernetic Serendipity show in 1968 [Pask 1971]. The mobiles form a simple social system. Communication takes place by modulating colored lights and audible tones. The mobiles attempt to maintain a “territory” – this results in a combination of competition and cooperation between the mobiles. Left to their own devices, the mobiles pursue their own goals, rotating, flashing lights and tones at each other, and exhibiting various forms of social organization. Audience members entering the environment were provided with devices to generate the lights and tones of the mobile’s language, and were thus able to participate in the alien society.
Simon Penny’s Petit Mal [Penny 1997] is an autonomous robotic being that interacts in physical space with audience members. Penny explicitly sought to avoid anthropomorphism, biomorphism, or zoomorphism, wanting instead “… to present the viewer with a phenomenon which was clearly sentient, while also being itself, a machine, not masquerading as a dog or president” [Penny 2000]. Petit Mal’s body consists of a double pendulum system that dynamically stabilizes its two-wheeled stance. Audience members project rich and complex readings of Petit Mal’s internal life – much of the complexity of its “personality” arises from the physical dynamics of its alien form.
David Rokeby’s Giver of Names [Huhtamo 1998] is a computer-vision based work that offers poetic commentary on the objects placed in front of its camera. The audience is provided with a collection of objects that can be placed on a podium in front of the camera. The colors, forms and juxtapositions of the objects stimulate nodes in a semantic network; the poetic commentary results from the activated concepts in the semantic net. After listening to the commentary for awhile, audience members begin to discern the private, idiosyncratic structure of Giver of Names’ conceptual space.
Narrative
Story-telling and narrative are fundamental to human experience [Mateas & Sengers 2003]. As children, we are immersed in stories and learn to approach the world via narrative frameworks. As adults, we order events and find meaning by assimilating them to more-or-less familiar narratives. AI-based narrative art explores narrative meanings by generating, modeling and understanding narrative structures and meanings.
Façade (Mateas, Stern), with its incorporation of autonomous character and player activity into a story framework, and Terminal Time, with its generation of ideologically biased documentary histories, are examples of AI-based narrative art. There is a rich tradition of story generation work in the AI research literature; this literature offers a useful collection of ideas and technical approaches for AI-based narrative art.
Robotic Art
Robotic art is concerned with building physical, often sculptural, systems whose behavioral responses are a function of the system’s perception of the environment. Robotic art is concerned with the aesthetics of physical behavior. Jack Burnham described the possibility of an aesthetics based on cybernetics and artificial intelligence in 1968 [Burnham 1968].
Edward Inahtowicz’s The Senster is an early example of robotic art. Inahtowicz was interested in how humans ascribe intention to physical motion and behavior:
“This is not the place to present the general argument that all perception is dependent in some way on an interpretation of physical movement, but all the pieces I have made so far and all that I am planning to make aim ultimately at making the spectator aware of just how refined our appreciation of motion is and how precisely we are capable of interpreting the intention behind even the simplest motion. For an artificial system to display a similar sense of purpose, it is necessary for it to have a means of observing and interpreting the state of its environment.” [Inahtowicz: 4-5]
Robotic art often explores the responsive behavior of abstract sculptural forms that have little or no relationship to human or animal forms. For this reason, robotic art is often an alien presence, an abstract responsive “life form” with its own idiosyncratic behaviors and drives appropriate to its physical form.
Meta-Art
Meta-art systems produce art as output; that is, they autonomously create writing, visual imagery, music, etc., that might be considered art in its own right. The goal of meta-art is often conceptual, an exploration of the dynamics of creativity within a specific genre; the generated output itself is often not the primary goal.
One body of work has explored the generation of writing, such as poetry or stories. Masterson’s haiku generator [Masterson 1971] is an early example of poetry generation. More recent examples include Loss Glazier’s sound poetry generator (sound poetry is interested purely in the sound of often nonsense words, rather than in semantic meaning), Kurzweil’s Cybernetic Poet, which learns language models from example poems and generates poetry in that style, and Manurung’s work, which generates poetry using a natural language generation framework that makes use of both syntactic and semantic knowledge [Manurung, Ritchie & Thompson 2000]. Story generators include the first generator, Tale-Spin [Meehan 1976 (Warren Sack’s rewrite of original) (Michael Cox’s drug-addicted Elvis version)], a character-based generator that generates stories by pursuing character goals and plans, Universe [Lebowitz 1984; Lebowitz 1985], a generator that coordinates multiple characters via authorial plans, and Minstrel [Turner 1994], a case-based generator that simultaneously pursues character and author goals within a single model of creative problem solving.
There is a vast body of work in music generation. Notable examples include David Cope’s EMI (Experiments in Musical Intelligence), a composition system that analyzes musical examples in terms of their stylistic components and recombines these components to generate new music in the same style [Cope 1996], and George Lewis’ Voyager, an improvisation system that performs with human musicians [Roads 1985].
There is a similarly vast body of work on systems that generate drawings or paintings, with Aaron being a prominent example. Visual grammars have been developed to describe the painting style of painters such as Wassily Kandinsky, Joan Miro, Juan Gris and Richard Diebenkorn; programs implementing these visual grammars are then able to generate endless variations on these styles. Similarly, shape grammars have been implemented that can generate endless architectural forms in the style of various well-known architects.
Note that not all meta-art is procedural portraiture. Meta-art may employ methods that are not dynamic representations of human activity. For example, shape grammars or n-gram poetry generation models offer little interesting commentary on human processes of architectural design or poetry writing, yet can serve as a generative framework for producing an interesting output space.
Burnham, J. 1968. Beyond Modern Sculpture: The Effects of Science and Technology on the Sculpture of This Century. New York: G. Braziller.
Colby, K. M. 1975. Artificial Paranoia: A Computer Simulation of Paranoid Processes. New York: Pergamon Press.
Cope, D. 1996. Experiments in Musical Intelligence. Madison, WI: A-R Editions.
Huhtamo, E. 1998. Silicon Remembers Ideology, or David Rokeby’s Meta-Interactive Art. Catalog essay for The Giver of Names exhibit at McDonald-Stewart Art Center.
Lebowitz, M. 1985. Story Telling as Planning and Learning. Poetics 14, pp. 483-502.
Lebowitz, M. 1984. Creating Characters in a Story-Telling Universe. Poetics 13, pp. 171-194.
Manurung, H. M., Ritchie, G., Thompson, H. 2000. Towards a Computational Model of Poetry Generation. In Proceedings of AISB Symposium on Creative and Cultural Aspects and Applications of AI and Cognitive Science, 79-86, Birmingham, April 2000.
Masterson, M. 1971. Computerized haiku. In J. Reichardt (Ed.), Cybernetics, Art and Ideas. Greenwich, CT: New York Graphic Society Ltd. 175-183
Mateas, M and Sengers, P (Eds.). 2003. Narrative Intelligence. Amsterdam: John Benjamins
McCorduck, P. 1991. Aaron’s Code: Meta-art, Artificial Intelligence, and the Work of Harold Cohen. New York, NY: W. H. Freeman and Co.
Meehan, J. 1976. The Metanovel: Writing Stories by Computer. Ph.D. Dissertation. Yale University.
Pask. G. 1971. A comment, a case history, and a plan. In J. Reichardt (Ed.), Cybernetics, Art and Ideas. Greenwich, CT: New York Graphic Society Ltd. 76-99.
Penny, S. 2000. Agents as Artworks and Agent Design as Artistic Practice. In K. Dautenhahn (Ed.), Human Cognition and Social Agent Technology. Amsterdam: John Benjamins
Penny, S. 1997. Embodied Cultural Agents at the Intersection of Robotics, Cognitive Science and Interactive Art. In K. Dautenhahn (Ed)., Working notes of the Socially Intelligent Agents Symposium. AAAI Fall Symposium Series. Menlo Park: Calif.: AAAI Press.
Roads, C. (1985). Improvisation with George Lewis. In C. Roads (Ed.), Composers and the Computer, 75-88. William Kaufmann, Inc.
Turner, S. R. 1994. The Creative Process: A Computer Model of Storytelling and Creativity. Lawrence Erlbaum Associates.
Weizenbaum, J. 1966. Eliza – A computer program for the study of natural language communication between man and machine. Communications of the ACM 9(1):36-45.
June 23rd, 2003 at 12:15 pm
I find these categories pretty useful. They overlap with each other a bit here and there, but generally seem to work. (Also, congratulations on the first GTxA post that requires a reference list! :-)
The term “robotic art” is the most problematic for me. I feel it isn’t descriptive enough about the concept involved – the aesthetics of physical behavior – whereas the terms procedural portraits, characters, alien presence, narrative, and meta-art are more descriptive. Perhaps “Physical aesthetics” would be more descriptive?
What about genetic art, a-life art? Do you consider those separate from “AI”?
What about AI-based art that does interesting things with huge databases of information? (data mining, subversive uses of Google, etc.)
Finally, a question, which I’m sure comes up in such discussions on a regular basis: What is considered AI? At what point does an interactive computer program become considered AI? For example, Eliza has very little actual intelligence. Petit Mal, from what I understand, has little or no “AI” per se. Does a work simply have to have the intention of intelligence, or the illusion of intelligence, to be consider AI-based art? Or put another way, if a work simply addresses the concept of AI, but doesn’t necessarily have actual AI in it, is it AI-based art?
Also, that phenomenon of “once we figure it out and build it, it doesn’t seem like AI anymore” seems to come into play here.
June 24th, 2003 at 8:28 pm
They overlap with each other a bit here and there, but generally seem to work.
The categories aren’t meant to be mutually exclusive. A single piece might participate in multiple categories. For example, Office Plant #1 is an alien presence and robotic art. Terminal Time is a procedural portrait, a narrative system, and since it constructs documentary histories, meta-art. I think of these categories as ways in which AI can be used in art, with any single piece potentially in multiple categories.
The term “robotic art” is the most problematic for me. Perhaps “Physical aesthetics” would be more descriptive?
I used “robotic art” because it is an already accepted category, growing as it did out of machine sculpture. I agree that it is not as descriptive a label as the others. “Physical aesthetics” is probably too broad in that the making of any physical object might be concerned with physical aesthetics. The real key is the aesthetics of physical motion, the relationship between the generated motions and the sensed environment, and the sense of intentionality behind the motions. If a piece consists only of physical motion, with no sensing or intentionality behind the motion, then we’re talking about machine art, but not robotic art (and certainly not AI-based art).
What about genetic art, a-life art? Do you consider those separate from “AI”?
I consider A-Life to be under the same general umbrella as AI. Saying that something is A-Life art tends to indicate the technique rather than the conceptual category. For example, someone might use a genetic algorithm for image generation – this would be meta-art. Someone might employ a simulated endocrine system coupled to a neural net to control the action of a creature – this is an alien presence (possibly a character).
What about AI-based art that does interesting things with huge databases of information? (data mining, subversive uses of Google, etc.)
Do you have a specific example in mind? It’s hard for me to think about without a specific example.
Finally, a question, which I’m sure comes up in such discussions on a regular basis: What is considered AI?
AI is a set of technical and rhetorical strategies for relating to computation. To say that a system is (or uses) AI is to say that interesting relationships have been established between system details and ordinary language used to describe the phenomenal world of humans (e.g. “goals”, “sensing”, “behavior”, “learning”, etc.). To say this more simply, I sometimes define AI as “The attempt to make machines do, what people and animals do, for all values of ‘do’.” The machine in question may be a digital computer, but doesn’t have to be. Some examples of things that people and animals do include: falling in love, feeling pain, having an argument, enjoying a sunset, packing for a trip, dancing, reading a book and talking to a friend about it, inventing scientific theories, writing a novel, and, occasionally, playing a game of chess or solving a logic problem.
Peti Mal makes use of behavioral robotics techniques and thus has as much AI as, for example, behavioral robots built in Rod Brook’s lab at MIT. And yes, these techniques are AI because they concern themselves with what it means to be an agent.
The details of whether any specific work counts as AI-based art depends on whether the piece makes use of computation in a way that relates to the phenomenal world of human experience (“goals”, “behavior”, “sensing”, etc.). The easiest way to do this is to use techniques that are already established as AI techniques (planning, neural nets, etc.) – the hard work of constructing the rhetorical machine that relates the technology to the human world has already been done. If a piece makes use of an entirely novel technique, then whether it is AI or not requires the artist to do the hard rhetorical work required to establish the technique as AI, to connect it to the human world – this is the same work that any AI researcher has to do when establishing a new technique. Which is to say that every AI system is doubled, consisting of a technical machine and a rhetorical machine. Both are necessary in order for the system to function.
July 7th, 2003 at 11:27 am
just saw this festival that seems pertinent to this thread
machinista a.i. as art
this year with a focus on the machine as co-artist
February 10th, 2004 at 6:40 am
FYI:
More on my Art& AI efforts @
http://neon.1colony.com
best,
.pekka.