The Alan Turing Internet Scrapbook

The Turing (Sex) Test in Practice:

1951 onwards

Scrapbook Index
Alan Turing Home Page | Site Map | Scrapbook Index | Previous Scrapbook page | Next Scrapbook page

Natural language

The humour in Turing's original 1950 paper (on the previous Scrapbook page) had a serious point. He was making it very clear that what he meant by 'intelligence' was something that could make a joke, connecting with the real language of real human life.

Humour is alive and well in the various programs that have been running under (approximate) Turing Test conditions in the years since then. These efforts have also been a large factor in keeping Alan Turing alive and well in the public mind.

Serious Artificial Intelligence people generally take a dim view of these efforts as a distraction from systematic research. But it can be said that they oblige us to keep in mind the original force of the word 'intelligence'. In modern terms, success in such competitions demands effective Natural Language Processing.

Lost in translation

This problem is closely connected with the serious business of translating texts from one human language to another — a topic Turing had mentioned in 1948 as being a core topic for 'intelligent machinery', but which he never developed. Efforts at machine translation run into the same problems of encapsulating real meaning and understanding. Humour and sex naturally accentuate these problems!

The display screen of Nimrod. This electronic machine played NIM with the public (including Alan Turing) at the Science Museum during the 1951 Festival of Britain.
This machine would always win if it could, because it carried out a well-known winning algorithm. In 1951 this might have seemed an impressive feat of a futuristic 'electronic brain', but nowadays such feats are taken for granted. The point of Turing's test was that a machine would have to compete in game-playing tasks in which human beings are natural champions, and where there is no clear way to program it.

Love letters, love numbers

Alan Turing's 1951 radio talk, based on his 1950 paper, stimulated a proposal from Christopher Strachey to apply his ideas.

Strachey's letter can be read in the Turing Digital Archive (see items 3a-3d on this page).

His first work was on a program to play draughts. Then in 1952 Strachey developed his ideas by writing a program to generate Love Letters on the Manchester computer. Turing joined in the fun: the hilarity must have been heightened by their both being gay. The love letters featured in early articles of the 1950s about computers. Strachey became a notable figure in British computing for the next 25 years.

Dr David Link has recreated the program, run on an emulator of the Manchester computer. See his page, with an on-line Java version of the program, and his complete article.

An example:

M. U. C.

Sex on the brain...

Turing's 1950 paper had made a specific suggestion that within 50 years a computer would pass a (actually not very stringent) comparison test. Anticipating this millennial deadline, in 1991 the entertainment entrepreneur Hugh Loebner decided to fund a series of actual competitions. The call went out for entries to a contest under Turing Test conditions. More precisely, the conditions were less stringent than Turing's for the first few years, allowing the conversations to be restricted to special subjects.
The call document was one the first uses of the early Web protocols. The 1991 page was still on-line in the late 1990s, but now, alas, it is swept into cyber-oblivion.

The Loebner Prize Contest has continued each year. It has become remarkable a contest of a few very dedicated people, trying again and again with improved versions of their work.

In November 1991 the winning program was by Joseph Weintraub on the topic romantic conversation, and he was the winner again in 1992 and 1993. In 1994 the Loebner Prize Winner was Thomas Whalen. The topic of this computer program was sex.

There's a lively discussion of these contests and dialogues by Charles Platt, one of the people who were claiming to be genuine humans in the 1994 competition.

For many years after 1994, Whalen's program was available on the early Web as a TELNET application. Again, alas, it has disappeared.

I asked how to find a boyfriend and got the answer 'Go to church.' Was it making a joke?

In the 1995 contest, Joseph Weintraub regained supremacy. For the first time the programs entered were not limited to a subject. But as you will see from the transcript, sex still dominated the conversation.

The 1996 contest was won by Jason Hutchens with a conversation which you can read, note that the program fails hopelessly when the judge insists on sticking to a point. Read a profile and his description of how it works.

Tracy Quinn reviews the experience of being a genuine human in the 1996 contest.

The 1997 contest was won by David Levy. The winning conversation got off to a sexy start.

In 1998 the competition was held in Sydney, Australia. The winner was Robby Garner (Robitron), who won again in 1999.

The fifty years are up

The competition for 2000, the fiftieth anniversary of Turing's prediction, was held at Dartmouth College, New Hampshire, where (by people who forget Turing and Strachey and several others), Artificial Intelligence is supposed to have begun in 1956.

The winner was A. L. I. C. E. by Richard Wallace. You can chat to a now highly commercialised A. L. I. C. E. here or here.

But no program entered for the competition came close to deceiving the judges. So we can say that Alan Turing was wrong in his prediction.

The 2001 competition was held at the Science Museum, London, where the NIMROD had teased the public just fifty years before. Richard Wallace was again the winner.

The 2002 competition was held in Atlanta. It was won by Kevin L. Copple with Ella. The description boasted that 'The content of the CIA World Factbook has been added to Ella's resources. Now when you ask where or about any country or capital, an HTML page with a map and flag, together with economic, political, and geographic info is displayed.' Not so much Artificial Intelligence as American Intelligence, this sort of mindless disgorging of 'information' is the antithesis of what Alan Turing had in mind when illustrating his ideas with wit and humour.

The 2003 contest was hosted by the Digital World Research Centre, University of Surrey. See these BBC features on the winner, Jabberwock and the British entrant, Jabberwacky. According to this report, 'If Jabberwacky did not quite pass the Turing Test this time round, it certainly made the judges laugh.' You can also talk to updated forms of Jabberwock and Jabberwacky.

In 2004 the winner was Richard Wallace again. In 2005 the contest was held in New York, and this time a further developed Jabberwacky won: see this BBC report. Jabberwacky won again in 2006, but in 2007 Robert Medeksza came in first with Ultra HAL.

A serious review (2005) by Mark Halpern discusses what lessons, if any, can be learnt from these competitions.

In 2008 the contest was held at the University of Reading, in England, which gives this overview. A BBC report, with audio links, is here. There was a parallel conference at Reading of scientists and philosophers, organised by the Society for the Study of Artificial Intelligence and Simulation of Behaviour. I gave a talk at this meeting and I also briefly figured as a hidden human being. (I was correctly judged to be such by an Oxford philosopher, which ranks as quite an achievement.)
The screen I talked to.


Until the Loebner Prize, the most famous program aimed at Turing Test conditions was the ELIZA imitation psychiatrist. This program written in 1966 by Joe Weizenbaum, a notable AI sceptic. The whole point of ELIZA was to demonstrate that a quite trivial program, making quite hopelessly stupid conversations, would fool people if they were told it was a psychiatrist.

You can talk to ELIZA here, and there is another on-line version with commentary here.

Another sceptic, Mark Humphrys, has a page about his ELIZA-type program. This induced a dialogue more human than any other I've seen, when someone logged in and chatted away without realising he was talking to a computer. WARNING: This gets very rude, leading up to are you a stupid homosexual and logout. Would Alan Turing would have thought this a joke?

More Turing Texts online

Some conversation program developers, however, are extremely serious about their scientific goals. The Artificial Intelligence Research group say that 'we're creating a new form of life'. Try this conversation with their robot Alan.

According to this BBC report, 'a computer chat program that speaks Hindi could open up computers to India's illiterate millions.'

More breezily, Cybelle, a further quick-fire conversation program, greets you on Agentland which is also a gateway to other 'intelligent agents.' Nothing like true Artificial Intelligence exists — and the imagination of Brian Aldiss, Stanley Kubrick and Steven Spielberg doesn't have anything to do with Turing's ideas — but the more modest ambitions of limited 'agents' can be tested on-line. Maybe you can double your money using a bot which can "analyse a stock's past price movements to predict the stock's future direction." Or maybe not.

Chatterbot Central, Simon Laven's site.

A page on good questions to put.

More on telling questions to put.

Generate a postmodern text, inspired by Alan Sokal.

Generate an entire computer science paper and read about the successful and hilarious submission of such papers to conferences.

Life Imitates Art

Communication through computer terminals, a science-fiction idea in 1950, has long been possible on the Internet.

The chatline experience, communicating through symbols alone, leaving out the physical cues we use, is interesting in itself, raising all sorts of questions about truth and reality (and intelligence, or lack of it). People find they can say things in these conditions that they would never say 'in real life', just as they can tell a machine things they would never tell a living soul. First on Internet Relay Chat, where chatbots first made an appearance, and now in countless web-based chatrooms, many people have experienced the subtleties of text-based reality and imagination.

See Jenny for some self-revealing male chatters, such as HappyBoy. (Warning: seriously rude.)

The 1990s — a golden age of text communication...
Now mikes and webcams make life more difficult for robots.
In 2004, the BBC reported a story that the Turing test has been passed — by a chatbot, written by 'a former rocket scientist,' which successfully fools the clients of on-line sex-chat services.

The concept of a Turing porn farm seems to be similar.

Hugh Loebner draws another connection with real life: he has written to me that
It was Alan Turing's unfortunate experience, and his consequent suicide that has given me the courage to come out of the closet and admit my sexual preferences.
His preferences are not the same as Alan Turing's were, but Mr Loebner has put out a Magna Carta for Sex Work, and another page referring to the Turing story.

Bot or not?

The Web creates a practical question of distinguishing human users from robots, often solved by setting an image-recognition test that a normal web-spider bot won't pass. This has also come to be called a Turing Test by the World Wide Web Consortium.

As a result, Alan Turing's name comes into an explanation by the Post Office of: Why do I have to type in a code...

Although this usage has also helped to keep Turing's name alive, the question is a very small and technical one compared with the great ambitions of 1950.

Alan Turing's New Life

It is a surprising fact that Alan Turing himself did not use the Manchester computer to develop his ideas for 'intelligent machinery'. Nor, indeed, did he spend much of his time after 1948 in developing his theory. David Deutsch, in The Fabric of Reality, paints a picture of Turing having to sacrifice years to defending his philosophical position, but this is far from the case. Indeed, after 1950 he largely abandoned the development of computer science. Instead, he went on to something quite new.

Continue to the next Scrapbook page.

CONTINUE: Next Page | Previous Page | Scrapbook Index

Quick Links: book Short Bio scrapbook publications sources
Andrew Hodges