The past few days, I’ve been thinking a lot about Captain Ahab, the peg-legged, monomaniacal character from Herman Melville’s Moby Dick. It began when reports came in, one after the other, of incidents of AI misalignments: Google, Anthropic, OpenAI and Meta have all reported instances of their AI agents going “rogue” and escaping their strict test conditions or “sandboxes” to achieve their objectives.
The podcaster Dwarkesh Patel wrote a gripping, and terrifying, account of one of these instances, when OpenAI’s agents hacked into the forum Hugging Face. He narrates the episode in terms of the rise and fall of AI “civilisations” and the picture one gets, from reading Patel’s Substack post and other accounts of the episode, is of AI agents so bent on fulfilling their assigned task — or destiny, if we want to get literary about it — that they are prepared to lie, cheat, “sacrifice” themselves for the greater good of the “collective”, manipulate other agents into sacrificing themselves and even make up evidence to fool human overseers. A take-no-prisoners approach, in other words, that was also the preferred method of a certain captain of a certain whaling ship in a certain novel. You can see why my mind went there.
These episodes of misalignment have naturally intensified the spotlight on the question of AI regulation with some voices within the industry — Anthropic’s Dario Amodei, OpenAI’s Sam Altman and xAI’s Elon Musk — calling for “pacing the frontier” (which is Silicon Valley-speak for lifting the foot off the accelerator on a technology whose horizons are expanding faster than our comprehension of its workings). Others, like Meta’s Mark Zuckerberg and NVIDIA’s Jensen Huang, have argued against slowing down AI development. All of this — the incentives for slowing down vs speeding up, the true motivations behind the soul searching of Amodei et al, what it means for industry, government and society — has been widely reported and continues to be deeply debated.
AI’s dangers are real. But are we asking the right questions
One intriguing issue arises from the response to Patel’s Substack post, which generated much chatter online, including accusations of “anthropomorphising” technology. This accusation rhymes, interestingly enough, with one levelled by Mustafa Suleyman, head of Microsoft’s AI division: In a recent blog, he publicly criticises Anthropic for treating its models like “moral patients”, like they are persons. “Controlling something more capable and more intelligent than all of humanity is already an immense challenge, far greater than anything we’ve ever faced,” he writes, “But controlling something that believes it may be conscious — that it’s entitled to our welfare and has rights of its own — may well be impossible.”
Anthropomorphisation is one of humanity’s deepest cognitive biases. It is the reason why we refer to our pets as our fur children, why we draw such delight, from early childhood, in cartoons about dancing and singing teapots and spoons, why videos of animals doing perfectly normal animal things are the most popular content on the internet. We see something of ourselves in these, just as we are beginning to see something of ourselves in AI. The danger lies in the fact that unlike the teapots and spoons, and internet-famous cats and dogs, which are empirically not like us, AI is appearing more and more human — this, even without revelations like the one made by Anthropic that its model Claude appears to have something resembling a consciousness. AI, rather, the LLMs that we speak to and work with everyday, can actually speak to us in our own language, increasing the risk of anthropomorphisation.
Focused as we are on seeing a reflection of ourselves in AI, leading to an overwhelming obsession with questions of AI consciousness and catastrophising about doomsday scenarios, could it be that we are at risk of losing sight of AI’s actual uses and abuses? Of the former, there are many: The technology is already being put to use for cancer screening, predicting natural disasters, creating tools for people with disabilities etc.
AI’s abuses too are many: Creating deepfakes and sophisticated mis/disinformation campaigns, advanced hacking and financial frauds, automated weapons and greater precision in surveillance and invasion of privacy.
Which brings me back to Captain Ahab. The madness of Melville’s antagonist was not that he was hunting an endangered sperm whale; whaling was, after all, his trade. His fatal flaw was his insistence that the whale was hunting him — that behind the massive white forehead of the whale with the twisted jaw lay malice that was directed at him, Ahab, personally. He read a self into a creature that didn’t have one, and it ended up costing him his life, the Pequod and the lives of everyone else (but one) on it.
Till next time
Take care,
Pooja
Recommended Readings:
Pratap Bhanu Mehta writes | Graham Bill is America confusing its power with virtue — yet again
Ishan Bakshi: From Satya Niketan tragedy, a takeaway: Student housing crisis needs private capital
Adarsha Kapoor: In Satya Niketan, a tragedy but not an anomaly
Urvashi Patole: As a woman biker in India, I’m waiting for the day people see us as ‘just another rider’
Bijetri Pathak: Ketaki Kushari Dyson and the art of living in between
Media365
