Sometimes, those who would forget history are condemned to repeat it. For those of us with long memories, the current fuss – nay hysteria – surrounding ChatGPT(and “generative AI” generally) rings a bell.

We have been here before: in January 1996, to be precise. That was the moment when Joseph Weizenbaum, a computer scientist at MIT, unveiled Eliza, which would have been called the world’s first chatbot if that term had existed at the time. Weizenbaum wrote the software (in a programming language intriguingly called MAD-SLIP) to demonstrate that communications between humans and computers were inevitably superficial. It did that by providing a text box in which one could exchange typed communications with the machine. Inside the program was a script (christened DOCTOR by Weizenbaum) which parodied the exchanges a patient might have with a psychotherapist who practised the person-centred therapy pioneered by Carl Rogers. (The program’s name came from Eliza Doolittle, the cockney lass who was taught to “speak proper” in Shaw’s play Pygmalion.)

The way it works is that the program takes in what you’ve typed and processes it to produce a response. Suppose you type (as I’ve just done): “I’m a bit depressed by UK politics.” Eliza: “Do you think coming here will help you not to be depressed ?” Me: “I hope so.” Eliza: “Do you say you hope so for some special reason?” You get the point – but you can try it for yourself: just go to masswerk.at/elizabot/.

Weizenbaum wrote the program to show that while machines could apparently copy human behaviour, it was really just like a magician pulling a rabbit out of a hat: an illusion. And once you know how the trick was done, Weizenbaum thought, it ceased to be an illusion. There was nothing secret about Eliza: if you read the code then you could understand how it did its stuff. What took its creator aback was that even if people knew it was just a program they seemed to take it seriously. There’s a famous story about his secretary asking him to leave the room while she had her “conversation” with Eliza. People were utterly entranced by it. (I saw this myself when I once ran it on a PC at my university’s open day and had to prise people off the machine so that others in the queue could have a go.)

The intriguing echo of Eliza in thinking about ChatGPT is that people regard it as magical even though they know how it works

After the publication of Weizenbaum’s paper about Eliza, it didn’t take long for some people (including some practising psychiatrists) to start saying that, if a machine could do this kind of thing, who needed psychotherapists? Weizenbaum was as appalled by this as today’s educationists and artists are by the contemporary slavering over the tools of generative AI. For him, as one insightful commentator put it, “there was something about the relationship between a person and their therapist that was fundamentally about a meeting between two human beings. In language that was at times reminiscent of Martin Buber’s ‘I and thou’ formulation, Weizenbaum remained fixated on the importance of interaction between human beings.” In that sense, he was not just a distinguished computer scientist, but also a notable humanist.

This humanistic indignation fuelled his lifelong opposition to the technological determinism of the “artificial intelligensia”. And it informed his 1976 book, Computer Power and Human Reason, which confirmed his role as a thorn in the side of the AI crowd and ranks with Norbert Wiener’s The Human Use of Human Beings in setting out the reservations of a technological insider about the direction of humanity’s travel towards “the automation of everything”.

The intriguing echo of Eliza in thinking about ChatGPT is that people regard it as magical even though they know how it works – as a “stochastic parrot” (in the words of Timnit Gebru, a well-known researcher) or as a machine for “hi-tech plagiarism” (Noam Chomsky). But actually we do not know the half of it yet – not the CO2 emissions incurred in training its underlying language model or the carbon footprint of all those delighted interactions people are having with it. Or, pace Chomsky, that the technology only exists because of its unauthorised appropriation of the creative work of millions of people that just happened to be lying around on the web? What’s the business model behind these tools? And so on. Answer: we don’t know.

In one of his lectures, Weizenbaum pointed out that we are incessantly striking Faustian bargains with this technology. In such contracts, both sides get something: the devil gets the human soul; humans get the services that delight us. Sometimes, the trade-off works for us, but with this stuff, if we eventually decide that it does not, it will be too late. This is the bargain that generative AI now puts on the table. Are we up for it?

The Guardian, USA