January 31, 2008
EP 2.2: The Eliza Effect
I don’t remember the specifics of my first conversation with Eliza, but here is Eliza’s most famous exchange (with the computer’s responses in all capital letters):
Men are all alike.
IN WHAT WAY
They’re always bugging us about something or other.
CAN YOU THINK OF A SPECIFIC EXAMPLE
Well, my boyfriend made me come here.
YOUR BOYFRIEND MADE YOU COME HERE
He says I’m depressed much of the time.
I AM SORRY TO HEAR YOU ARE DEPRESSED
It’s true. I am unhappy.
DO YOU THINK COMING HERE WILL HELP YOU NOT TO BE UNHAPPY
I need some help, that much seems certain.
WHAT WOULD IT MEAN TO YOU IF YOU GOT SOME HELP
Perhaps I could learn to get along with my mother.
TELL ME MORE ABOUT YOUR FAMILY
My mother takes care of me.
WHO ELSE IN YOUR FAMILY TAKES CARE OF YOU
My father.
YOUR FATHER
You are like my father in some ways.
WHAT RESEMBLANCE DO YOU SEE
You are not very aggressive but I think you don’t want me to notice that.
WHAT MAKES YOU THINK I AM NOT VERY AGGRESSIVE
You don’t argue with me.
WHY DO YOU THINK I DON’T ARGUE WITH YOU
You are afraid of me.
DOES IT PLEASE YOU TO BELIEVE I AM AFRAID OF YOU
My father is afraid of everybody.
WHAT ELSE COMES TO MIND WHEN YOU THINK OF YOUR FATHER
Bullies.
DOES THAT HAVE ANYTHING TO DO WITH THE FACT THAT YOUR BOYFRIEND MADE YOU COME HERE
(Weizenbaum, 1966, 37)
As you might imagine, I was impressed. And I certainly wasn’t the first. Weizenbaum’s employers at the Massachusetts Institute of Technology had seized on Eliza as an accessible demonstration of the potential of computing, exciting to those without the specialized knowledge needed to appreciate much of the ongoing research. The computer world agreed. In fact, when the first demonstrations of the ARPANet (the predecessor of the Internet) were carried out, some of the terminals ran Eliza from a remote location (Leonard, 1997, 52). Given the excitement produced by Eliza, astronomer and science popularizer Carl Sagan suggested that the future might hold “a network of computer psychotheraputic terminals, something like arrays of large telephone booths” (Weizenbaum, 1976, 5).
Originally Eliza ran on a computer less powerful than the one in my mobile phone, though at the time it was one of the most advanced at MIT. Rather than being hooked up to a monitor, keyboard, and mouse — or HAL’s microphone, speaker, and camera — Eliza was experienced through something like a typewriter, allowing the computer to type to the user and the user to type back. People could type anything they wanted to Eliza, and the software would respond in different ways depending on the currently-active script.
This last word, “script,” is important. There is nothing magical about Eliza — it is simply a bundle of data and processes, and pretty simple processes at that. Each time Eliza is run it uses a particular script to guide its behavior. The example conversation above was created using Eliza’s most famous script, Doctor, which causes the software to parody the conversational patterns of a non-directive therapist during an initial visit. All of this is described in Weizenbaum’s 1966 article in Communications of the ACM.1
How Eliza works
A session with Eliza can begin with a greeting. Weizenbaum’s Doctor script begins, “How do you do. Please tell me your problem.”2 After this point Eliza will not take the initiative again — only respond.
Each time an audience member types something Eliza examines it, looking for words that have entries in the currently-active script. Some of the words are “keywords,” on which more below. Some words are marked for simple substitution. For example, when Eliza runs the Doctor script, these substitutions switch all the first person pronouns for second person pronouns (“I” becomes “you”) and vice versa (“yourself” becomes “myself”). For example, “Well, you are very helpful” would become “Well, I are very helpful.” A word can be both substituted and used as a keyword.
Periods and commas are treated as delimiters. If a period or comma is encountered, Eliza checks to see if a keyword has already been found. If one has, then everything the audience member typed after the delimiter is discarded. If no keyword has yet been found, everything before the delimiter is discarded. For example, “Well, I are very helpful” would become “I are very helpful.”
Each keyword has a priority level, or rank. When the first keyword is found in a text it is added to a “keystack.” Each time another keyword is found, the rank of the new keyword is compared with that of the highest-rank keyword yet found. If the new word has a higher rank it is added to the top of the stack, otherwise it is added to the bottom of the stack. The result, at the end of scanning a text for keywords, is that the highest rank keyword is at the top of the stack.
After keyword scanning, the next step is to find a “decomposition rule” that matches the post-substitution version of what the audience member typed (minus any parts discarded because of commas or periods). Decomposition rules are associated with keywords, so this search begins by popping the top keyword off the stack. (I’ll soon discuss what happens if there is nothing on the stack.) Eliza tries out the decomposition rules associated with that keyword, looking for one with an appropriate pattern of words in combination with the wildcard “0” (which is equivalent to “any number of words, including zero”). For example, if the keyword is “you” the first decomposition rule is:
(0 I remind you of 0)
This wouldn’t match the sample phrase from above, “I are very helpful.” The first rule would only match a post-substitution phrase like, “With that last statement I remind you that I are a machine” or (because 0 can represent zero words), “I remind you of the happier days of your childhood.” However, the next decomposition rule for the “you” keyword would match the example phrase used earlier:
(0 I are 0)
Once a match is found, the decomposition rule does what it is named for: it decomposes the phrase into parts. This decomposition is based on the number of words and the number of 0s in the rule. For example, “I are very helpful” would become:
(1) empty (2) I (3) are (4) very helpful.
All that remains, at this point, is to decide what to say back to the audience. Each decomposition rule can have a set of reassembly rules associated with it. An example reassembly rule (for the decomposition “0 I are 0”) is:
(What makes you think I am 4)
This would result in the response, “What makes you think I am very helpful.”3 The reassembly rules were, once used during any run of Eliza, marked so that all the reassemblies associated with a particular decomposition would be cycled through before any were repeated.
This is the bulk of Eliza’s approach. There are a few refinements, such as techniques that allow scripts to share rules between keywords (e.g., mother and father). But from this the dominant operational logic at work in Eliza should be clear: transformation. Each statement by an Eliza script is the result of a multi-step transformation of the most recent audience statement.
Many of these transformations include a verbatim portion of the statement being transformed, as outlined above. Others are more indirect transformations, as when “Perhaps I could learn to get along with my mother” is translated into “Tell me more about your family.” The real transformation difficulty comes, however, when no keywords — a central aspect of Eliza’s transformation logic — are found in the audience’s most recent text. This is the situation, for example, when Eliza/Doctor asks, “What else comes to mind when you think of your father” and receives the response, “Bullies.”
One set of possible transformations, in these cases, can be defined in the script based on a special keyword called “none.” In Doctor these are content-free phrases such as “Please go on” and “I see.” However, some of the most surprising moments with Eliza come from another special structure called “memory.” This structure works in combination with a particular keyword. In the case of Doctor it works with the keyword “my” and the decomposition “0 your 0.” When this keyword is selected a response is generated in the usual way. In addition, another reassembly may be created and stored in a queue for later use when no keyword is found. In Doctor these reassemblies are “Lets discuss further why your 3,” “Earlier you said your 3,” and “Does this have anything to do with the fact that your 3” — the last of these we saw above, with 3 being “boyfriend made you come here.” In other words, the surprising punchline in the above conversation was nothing but a fallback, a default move Eliza can make when the most recent response has no identifiable keywords.
Here we can see what moved Weizenbaum to write, in the same article, that “once a particular program is unmasked, once its inner workings are explained in language sufficiently plain to induce understanding, its magic crumbles away; it stands revealed as a mere collection of procedures, each quite comprehensible” (36). But when we interact with a piece of software, we don’t necessarily achieve such understanding, we don’t get a clear picture of how it actually operates internally. And Dr. Sagan and I were not alone regarding Eliza/Doctor — its audiences often developed very mistaken ideas about its internals. They assumed that, since the surface appearance of an interaction with the program could resemble something like a coherent dialogue, internally the software must be very complex. Some thought it must be something close to the fictional HAL: a computer program intelligent enough to understand and produce arbitrary human language. This happened so often, and was so striking, that in computer science circles people began to use a specific term for this kind of misunderstanding, one mentioned earlier in this chapter: “the Eliza effect.”
Why Eliza worked
The stories of Eliza/Doctor’s effectiveness are many. Weizenbaum himself tells us that “my secretary, who had watched me work on the program for many months and therefore surely knew it to be merely a computer program, started conversing with it. After only a few interchanges with it, she asked me to leave the room” (1976, 6). There is also the oft-repeated tale, that exists in many versions, of the boss thinking he is communicating with a subordinate while actually conversing with Eliza/Doctor:
Vice president: Do you think I can use the computer this morning?
Teletype: WHY DO YOU ASK?
Vice president: I might be able to make some additional telcomp (computer system) sales.
Teletype: WHY AREN’T YOU SURE?
Vice president: My prospective customers have never seen a demonstration of the system.
Teletype: DOES THIS MEAN VERY MUCH TO YOU?
Vice president: Of course it does.
Teletype: YOU SEEM VERY POSITIVE ABOUT THAT.
Vice president: This is ridiculous.
Teletype: WHAT DO YOU MEAN THIS IS RIDICULOUS?
(Murray, 1997, 7)4
I think, for me as a high school student, Eliza/Doctor created an effective initial illusion for the same reasons it worked for the iconic “emotional” secretary and “clueless” boss evoked by the above stories. First, all of us were accustomed to text-only computing and to having conversations with other people within that environment. Second, Eliza/Doctor makes a remarkably good match between process and data. The situation of the initial visit to the therapist, the clever writing in the reassemblies and non-responses, as well as the well-chosen keywords do the most possible to leverage the simple linguistic tricks available via Eliza’s transformation processes. (In fact, there were other Eliza scripts created besides Doctor, but none of them became nearly as well known or widespread.) Third, for myself, the secretary, and the boss, this was one of our first experiences with computer characters. But all three of these reasons are only Eliza/Doctor’s specific nuances on a much more general phenomenon: When a system is presented as intelligent, and appears to exhibit intelligent behavior, people have a disturbingly strong tendency to regard it as such.
This phenomenon derailed Weizenbaum’s career. He came to focus his work on the conceptual mismatch that gives the Eliza effect its name, and specifically on how it could “induce powerful delusional thinking in quite normal people” (1976, 7). He wrote a book dedicated to demonstrating that the internals of computers aren’t magical and that we do ourselves a disservice when we assume that human beings are so mechanical that we could, or should, have our intelligence matched by computational machines. In a sense, he moved from being a computer scientist to being one of the first knowledgeable critics to interrogate the cultures of computing and artificial intelligence.
Following Weizenbaum, a number of other authors saw the Eliza effect as important to address in understanding our relationship with computers, and our culture more generally. A decade after Weizenbaum’s book, Lucy Suchman published Plans and Situated Actions (1987), in which she sees Eliza/Doctor as an iconic example in human-computer interaction of the broad phenomenon of treating systems as intelligent based on very limited evidence. Specifically, she discusses what ethnomethodologist Harold Garfinkel (citing Karl Mannheim) has called the documentary method of interpretation.
Suchman presents one of Garfinkel’s experiments as a demonstration of the idea that people tend to “take appearances as evidence for, or the document of, an ascribed underlying reality, while taking the reality so ascribed as a resource for the interpretation of the appearance” (23). In this experiment student subjects were introduced to a new kind of therapy in which they asked yes/no questions about their personal problems. These were answered by counsellors who were not visible to the subjects. Unbeknownst to the subjects, the “counsellors” answered each question randomly.
After the experiment, the students were found to have constructed stories that made sense of each string of answers as a coherent exchange and set of advice. This happened even when, as would almost inevitably happen in such a circumstance, the answers given were self-contradictory. The apparent contradictions were explained away by the experimental subjects as revised views based on further information, evidence of a deeper agenda on the part of the counsellor, or something else that fit with the frame of the therapeutic situation.
Yet another decade later, Janet Murray revisited Eliza/Doctor in Hamlet on the Holodeck (1997). Here she makes a crucial turn for my purposes, seeing the project in terms of media. She views Eliza/Doctor as a dramatic character, Weizenbaum as a kind of playwright, and the source of the Eliza effect in such cases as “the human propensity to suspend disbelief in the presence of a persuasive dramatic presence” (224). Here Murray makes visible what wasn’t clear in previous accounts: that people enjoy Eliza. Initial interactions are often quite fun and engaging, and create the sense of open-ended possibilities with an interesting character.
But that first impression doesn’t necessarily last.
Notes
1This article is not only a good, clear source of explanation for Eliza’s processes (and the most-cited publication about Eliza in the computer science literature). It also served as the basis for many home-grown versions of Eliza created at computing centers across the country in the years that followed. This was still decades away from when researchers and hobbyists could easily transport software to one another over computer networks — so paper publications such as Communications of the ACM (for researchers) and Byte (for hobbyists) often included all the information needed (including, in the case of Weizenbaum’s article, the complete Doctor script) for reimplementing the software locally.
2While the original script text is in all capitals, I am regularizing it here. Also, this description focuses on the core processes at work in Eliza — to describe them all would make this section as long as Weizenbaum’s paper.
3Question marks were not used because they were interpreted as “line delete” on the computer system Weizenbaum was using.
4This is the version of the story reported in Janet Murray’s Hamlet on the Holodeck. In addition to this version, many apocryphal versions circulate, including one in which Weizenbaum is a participant in the events.
January 31st, 2008 at 6:37 am
I must admit that looking at it now in comparison to what we have it was truly amazing, a wonder of technology. Then again, my ‘session’ with Eliza was a lot less impressive. I likened it to a psychic reading in that it in that ‘she’ asked leading questions and fed back what was given to her.
January 31st, 2008 at 7:50 am
Yes, Eliza/Doctor was a pretty amazing project at the time. But it is also an experience with a real tendency toward breakdown, which isn’t that often discussed. I’m going to get into that in tomorrow’s section.
February 8th, 2008 at 12:09 am
I don’t think the argument I’m about to summarize is novel, but I can’t remember offhand who I’m summarizing. Basically, machines aren’t unusual or particularly “disturbing” in this respect: When people interpret the behavior of anything presented as intelligent, whether another person, an animal, a machine, etc., they tend to explain its behavior by projecting thought processes similar to their own on it. Hence the common anthropomorphization of, say, cats.
And it’s not only with machines where the projection can be wrong: the real therapist at such a visit might well be following a pretty mechanical learned script too, despite being incorrectly perceived as generating his responses via some complex cognitive process (that’s one explanation for why Eliza’s Doctor script worked as well as it did—that it’s actually a reasonably accurate simulation of this particular interaction). And of course all sorts of instinctual hardwire animal (and human) behaviors are misperceived as the result of complex cognition—even ant behavior tends to get perceived as intelligent.
February 10th, 2008 at 3:26 pm
Mark, I’ve been trying to figure out if I can remember where I first heard an argument like that — but I think I’m as stumped as you are in that regard. The closest I came was remembering a book first suggested to me by Julianne Chatelaine: The Media Equation by Reeves and Nass. (A short essay of Chatelaine’s includes some good summary of the arguments.) They don’t get into cats and ants, but they do talk about forms of media as far afield from software as radio.
I’m sure there are also other people who make similar points in other domains, and that’s probably why you and I are having a hard time remembering our first encounters with these ideas. The important thing here, however, is that when I say “system” in the last sentence of the paragraph, that might be read as “computer system.” I mean it much more broadly — which I’d hoped to indicate with the non-computer example of Garfinkle’s experiment, which runs along the lines of what you suggest about some therapists. It’s probably a good idea for me to include a note, or some other explanatory text in the chapter, making it clear that I’m not just talking about software.
February 11th, 2008 at 2:23 am
Oh ok, in that case I don’t really have an objection. I had interpreted your comment as singling out AI systems as being particularly bad in this fake-intelligence respect, while I tend to see them as not much worse than a lot of other things that people ascribe more intelligence to than they probably should.
On the other hand, I think it’s an open question what “real” intelligence actually is, and when even people exhibit it. AI systems have a bit of a rhetorical disadvantage here because you can open them up and say, “oh, it’s just doing [foo], that’s not actually intelligent”, while the homo sapiens source code is much more obfuscated.