The Chinese Room Thought Experiment

Work in Artificial Intelligence (AI) has produced computer programs that can beat the world chess champion, control autonomous vehicles, complete our email sentences, and defeat the best human players on the television quiz show Jeopardy. AI has also produced programs with which one can converse in natural language, including customer service “virtual agents”, and Amazon’s Alexa and Apple’s Siri. Our experience shows that playing chess or Jeopardy, and carrying on a conversation, are activities that require understanding and intelligence. Does computer prowess at conversation and challenging games then show that computers can understand language and be intelligent? Will further development result in digital computers that fully match or even exceed human intelligence? Alan Turing (1950), one of the pioneer theoreticians of computing, believed the answer to these questions was “yes”. Turing proposed what is now known as ‘The Turing Test’: if a computer can pass for human in online chat, we should grant that it is intelligent. By the late 1970s some AI researchers claimed that computers already understood at least some natural language. In 1980 U.C. Berkeley philosopher John Searle introduced a short and widely-discussed argument intended to show conclusively that it is impossible for digital computers to understand language or think.

Searle argues that a good way to test a theory of mind, say a theory that holds that understanding can be created by doing such and such, is to imagine what it would be like to actually do what the theory says will create understanding. Searle (1999) summarized his Chinese Room Argument (herinafter, CRA) concisely:

Imagine a native English speaker who knows no Chinese locked in a room full of boxes of Chinese symbols (a data base) together with a book of instructions for manipulating the symbols (the program). Imagine that people outside the room send in other Chinese symbols which, unknown to the person in the room, are questions in Chinese (the input). And imagine that by following the instructions in the program the man in the room is able to pass out Chinese symbols which are correct answers to the questions (the output). The program enables the person in the room to pass the Turing Test for understanding Chinese but he does not understand a word of Chinese.

Searle goes on to say, “The point of the argument is this: if the man in the room does not understand Chinese on the basis of implementing the appropriate program for understanding Chinese then neither does any other digital computer solely on that basis because no computer, qua computer, has anything the man does not have.”

Thirty years after introducing the CRA Searle 2010 describes the conclusion in terms of consciousness and intentionality:
I demonstrated years ago with the so-called Chinese Room Argument that the implementation of the computer program is not by itself sufficient for consciousness or intentionality (Searle 1980). Computation is defined purely formally or syntactically, whereas minds have actual mental or semantic contents, and we cannot get from syntactical to the semantic just by having the syntactical operations and nothing else. To put this point slightly more technically, the notion “same implemented program” defines an equivalence class that is specified independently of any specific physical realization. But such a specification necessarily leaves out the biologically specific powers of the brain to cause cognitive processes. A system, me, for example, would not acquire an understanding of Chinese just by going through the steps of a computer program that simulated the behavior of a Chinese speaker (p.17).

“Intentionality” is a technical term for a feature of mental and certain other things, namely being about something. Thus a desire for a piece of chocolate and thoughts about real Manhattan or fictional Harry Potter all display intentionality, as will be discussed in more detail in section 5.2 below.

Searle’s shift from machine understanding to consciousness and intentionality is not directly supported by the original 1980 argument. However the re-description of the conclusion indicates the close connection between understanding and consciousness in Searle’s later accounts of meaning and intentionality. Those who don’t accept Searle’s linking account might hold that running a program can create understanding without necessarily creating consciousness, and conversely a fancy robot might have dog level consciousness, desires, and beliefs, without necessarily understanding natural language.

In moving to discussion of intentionality Searle seeks to develop the broader implications of his argument. It aims to refute the functionalist approach to understanding minds, that is, the approach that holds that mental states are defined by their causal roles, not by the stuff (neurons, transistors) that plays those roles. The argument counts especially against that form of functionalism known as the Computational Theory of Mind that treats minds as information processing systems. As a result of its scope, as well as Searle’s clear and forceful writing style, the Chinese Room argument has probably been the most widely discussed philosophical argument in cognitive science to appear since the Turing Test. By 1991 computer scientist Pat Hayes had defined Cognitive Science as the ongoing research project of refuting Searle’s argument. Cognitive psychologist Steven Pinker (1997) pointed out that by the mid-1990s well over 100 articles had been published on Searle’s thought experiment – and that discussion of it was so pervasive on the Internet that Pinker found it a compelling reason to remove his name from all Internet discussion lists.

This interest has not subsided, and the range of connections with the argument has broadened. A search on Google Scholar for “Searle Chinese Room” limited to the period from 2010 through 2019 produced over 2000 results, including papers making connections between the argument and topics ranging from embodied cognition to theater to talk psychotherapy to postmodern views of truth and “our post-human future” – as well as discussions of group or collective minds and discussions of the role of intuitions in philosophy. In 2007 a game company took the name “The Chinese Room” in joking honor of “...Searle’s critique of AI – that you could create a system that gave the impression of intelligence without any actual internal smarts.” This wide-range of discussion and implications is a tribute to the argument’s simple clarity and centrality.