February 12, 2008
EP 4.2: Eliza and the Turing Test
While Eliza is the first well-known digital character, its roots trace back to a highly influential proposal for computer-driven conversation (less than two decades earlier) from the father of general-purpose computing: Alan Turing, mentioned earlier in this book’s introduction. Writing for the philosophy journal Mind, Turing initially proposed to consider the question, “Can machines think?” (1950). However, finding this question hopelessly ambiguous, Turing instead replaced it with a set of questions involving an “imitation game.”
The human version of this game has three participants: “a man (A), a woman (B), and an interrogator (C) who may be of either sex.” During the course of the game the interrogator asks questions of A and B, trying to determine which of them is a woman. A and B, of course, do their best to convince C to see it their way — the woman by telling the truth, the man by “imitation” of a woman. The proposed game is played over a teletype, so that nothing physical (tone of voice, shape of handwriting) can enter into C’s attempt to discern the gender of the other players based on their performances.
Turing then asks, “What will happen when the machine takes the part of A in this game?” How will interrogator’s results compare to when the game is played based on gender? These questions are proposed as a closely-related replacement for the question “Can machines think?”
The ideas in Turing’s essay have been widely discussed — the imitation game is now commonly called the “Turing test” — and vigorously debated.1 For my purposes, however, the key element of Turing’s game is that it is based purely on surface behavior. In part this is no doubt due to his audience — many readers of Mind would have understood little of any discussion of computational processes. But, given the vast influence of Turing’s work, it should also be considered in terms of larger attitudes about the relationship between surface appearance and internal processes that shaped the AI community.
The Turing test is the most famous example of the idea that we need not consider the internal operations of systems when evaluating them. From this point of view, whatever model drives a hypothetical system that can be said to have passed the Turing test, we should consider it to embody something close to “thinking.” Given that — in the limited time available for interaction — some of those who interacted with Eliza/Doctor in the time around which Weizenbaum constructed it appear to have thought it was genuinely thoughtful, Weizenbaum’s famous paper on the system was specifically at pains to dispel this sort of idea. In fact, the paper could be read as a long, detailed counterexample to the argument Turing put forth, which failed to take the workings of the Eliza effect’s initial stages into account.
But Weizenbaum’s paper was not wholly an attempt to help people see through Eliza’s illusion. Among other things, it also speculated on possible future directions for the Eliza project. His projected future Eliza would “slowly build a model of the subject conversing with it” (1966, 43). This would, in turn, enable an Eliza that didn’t simply transform the previous audience utterance, but would say things guided by aspects of this model — aspects that might indicate audience rationalizations, contradictions, or other objects of interest to a more advanced Doctor script. Looking toward how such a system could be built, Weizenbaum cited a then-recent paper by Robert Abelson and J. Douglass Carroll.
Note
1Some have argued that Turing was providing a behaviorist definition of intelligence, while others have argued that, at most, Turing was presenting one possible criterion for thinking (and that it would be possible for things that ought to be described as thinking to not pass the test). Similarly, some have argued that Turing’s test is deeply gendered (the machine attempting to pass as a woman) while others have argued that this is a red herring (at other points the machine is described as imitating a man) and yet others have argued that the gender-driven test plays a key role: as a scoring mechanism for the human/machine test. See The Essential Turing (Turing, 2004) for versions of Turing’s most influential writings and summaries of debates surrounding them.
Moving beyond these debates, Mark Marino (2006b) has positioned the Turing test as an “Ontological Turing Machine.” The name is chosen in relation to the “Universal Turing Machine” — Turing’s famous outlining of the concept, and a possible implementation, of the notion of universal computation. For Marino, Turing’s Mind essay does similar foundational work, both conceptually and in a possible implementation, for the doubt produced by network communication (which, from MMO games to instant message chats, continually raises questions as to the actual “age / sex / location” and human/software status of others).
February 12th, 2008 at 8:30 am
I’ve just built an online experiment based on the Eliza project:
http://davemiller.org/experiments/paxton/
This is an interactive conversation, and visual story combined. It deals with a controversial, topical and emotive local issue, concerning the development of a historic park in London, UK after long years of neglect. Plans are put forward, and local people consulted. It occurred to me that one voice is sadly missing from the consultation, that of the master architect and plannner Joseph Paxton, who conceived the original design for the park. I thought it would be interesting to imagine his point of view.
I’ve tried here to make this online interactive story unfold as a conversation, which is often considered the highest form of (or ultimate) interactivity. I’m trying to tell the story through interaction.
More info here: http://davemiller.org/art_blog/?p=6
February 12th, 2008 at 1:14 pm
Turing proposed his test as a *replacement* for the question of whether machines think, not as a *definition* of thinking.
He wanted to avoid the question of whether machines think, because he thought (under the influence of Wittgenstein) that it was a pseudo-question, which would produce nothing but interminable, unproductive debate.
February 12th, 2008 at 8:55 pm
Did Turing say to change only the player A, and nothing else, or did he say to change player A, and also inform the Interrogator that the goal of the game was to discern human from machine? If I didn’t already know what the Turing test was, I would read it as the former, which will still work as a Turing Test if it is assumed that men and women are equally difficult to imitate.
February 12th, 2008 at 9:10 pm
Whoops. The answer to my question is in note 1. I kind of prefer the test in its original form, because it leaves the Interrogator blind to the task he is actually performing.
February 13th, 2008 at 7:11 am
Richard, that’s certainly how I read the text of Turing’s 1950 paper. But I’m no Turing scholar. I have only read a few of his other writings and a little of the biographical information that’s out there.
Among that biographical material, Andrew Hodges’s book is quite readable, and seems authoritative — but as Jack Copeland points out in The Essential Turing (in the preface to Turing’s 1950 article that begins on page 433) Hodges is one of those who positions the game as a definition (“an operational definition of ‘thinking’ or ‘intelligence'”). Copeland argues, on the other hand: “There is no textual evidence to support this interpretation… Success in the game is arguably a sufficient condition for thinking; but success in the imitation game is not also a necessary condition for thinking.”
Luckily, this is somewhat to the side of my main argument here — so I decided to refer people to Copeland’s book for a summary of some of the controversies, rather than stake out a position as my own. The important thing, for my purposes, is that the idea of the “Turing test” as a measure of computer intelligence has had a major influence. It weighs in on the side of “How it works doesn’t matter.” Yet Weizenbaum, whose project is generally the first actual software mentioned in discussions of this area, was very strongly of the opposite opinion.
February 13th, 2008 at 7:25 am
Dave, thanks for the pointer. You might also be interested in Mark Marino’s work as an artist (as opposed to the critical work I cite in the note below). Both the time travel and dating game pieces use conversation to tell stories — though, of course, rather different ones from yours.
February 13th, 2008 at 10:29 am
Hey, Noah,
As I prepare the manuscript version of the text, I’m building on this work to reread the Turing Test and ELIZA in a new context that I call: Conversational Actor Networks or Conversational Actor Network Theory (CAN/T). In this context, I read the Test and conversational agents as systems of humans and computers that form into actor-networks through conversational exchanges.
I will be presenting some of my newer direction at the Visionary Landscapes conference this summer.
February 14th, 2008 at 9:50 am
Mark, sounds like it’ll be interesting (and funny) — as your work often manages to be. I’m looking forward to the presentation.