UK needs to form Cobra committee for AI, says Anthropic co-founder

Photograph by Tom Pilston for the Observer

Jack Clark thinks we are in denial. The technology he is building is about to change everything – the economy, the way we think, the way we work, the way we relate to each other – and the rest of us haven’t caught up. Not to the benefits, and not to the risks.

Clark, born in Brighton, is the co-founder of AI lab Anthropic, one of the most powerful startups in history – its valuation is expected to exceed $900bn in the coming months. A former journalist who once covered data centres for Bloomberg, he left OpenAI in 2020 to co-found a rival startup that promised to build AI more responsibly. Today, he tells me, we are on the cusp of a threshold the industry calls artificial general intelligence. By the end of 2028, he predicts, AI systems will be able to design their own successors, machines building smarter machines without human input.

We meet in a nondescript room in the Schwarzman Centre for the Humanities in Oxford, a glass-and-stone new-build on the edge of the old university, funded by the American billionaire Stephen Schwarzman, the CEO of Blackstone. Flanked by two bodyguards and a sizeable communications team – a far cry from his days filing copy on server farms – Clark is here to deliver the Cosmos Lecture, an annual address to students and academics, in which he will lay out his case.

He will tell the room that the progress of the last three years has been “extremely abnormal”, predicting that within a year a team of humans working with an AI system will make a discovery worthy of a Nobel prize. He will describe how, inside Anthropic, many engineers have stopped writing code entirely, instead directing Claude, the company’s AI model, to do the work for them. He will say the company already feels less like 4,000 employees and more like 40,000. He will also, in a later Q&A, state plainly that this technology carries “a non-zero chance of killing everyone on the planet”.

It seems a strange pitch for a co-founder worth an estimated $7bn, thanks entirely to Anthropic’s ascent. The world, you might think, has heard enough about the potential of AI. Clark disagrees. “The problem with AI is that individual achievements come and go,” he says. “But if you just look at the trends over time, you come away with this view of, oh my goodness, there is this tremendous, constantly rising level of capabilities.” Much of the progressive world, he tells the audience later, “seems to be in denial about the capabilities of AI systems today as well as where they might be in six months. And to me, this feels like an unhealthy or dangerous pressure building up in the system.”

It is “the toughest part of the job” to strike the right balance between optimism and alarm, he tells me. That is true now more than ever. In April, a 20-year-old man threw a Molotov cocktail at the San Francisco home of Sam Altman, the CEO of OpenAI. And in the past fortnight, three senior executives have been booed by crowds of students while speaking about AI at university commencement ceremonies in the US – most notably Eric Schmidt, the former Google chairman, who was jeered throughout his address at the University of Arizona when he told graduates that AI would “shape the world”. (“It did not go well for him,” Clark says, smiling, when I raise it with him.)

Clark has described the public mood as a “generalised anxiety” towards AI. Others are calling it a brewing anti-AI populist movement.

It’s fair to say he and his company have fed the public fear. Clark has written that he is “deeply terrified” of this technology, and his co-founder, Dario Amodei, predicted that AI could wipe out 50% of entry-level white-collar jobs within a couple of years. “In the early years, it felt like there was an under-discussion of the potential risks,” he tells me. “We talked about worries about bioweapons. We talked about the potential ways AI systems could get better at conducting hacking or cyber war… Now, I think, is the time for us to try and talk more about the positives,” he says.

So rather than warning about what AI could do to the world, on stage Clark talks about what it has done for him. With self-deprecating warmth, he describes using Claude to navigate a challenging relationship with his father (“Claude said: ‘Go see your dad.’ That’s the correct answer.”) He says the model encouraged him to return to therapy. And he describes how, that morning, he asked Claude to read several hundred biology papers and map progress in the field – work that would have taken him weeks, done in 20 minutes.

There is no booing; the lecture is delivered to a room of technologists and students who appear to have bought in to the promises of AI. But two protestors stand outside the building, human hints of the generalised anxiety. Part of Clark’s difficulty is that he cannot say exactly how the technology will change us, only that it will. “I can’t reconcile this technology with the economy staying normal,” he says.

For Britain, whose economy is roughly 80% services, he sees both vulnerability and opportunity. “In some sense, the UK, just as it experienced a financial crisis, is highly vulnerable to swings in service jobs and industries like finance or other white-collar work,” he says. But he argues that automation tends to create new high-value work above whatever it has replaced. “The UK has this tremendously large financial services and insurance industry,” he says. “There are parts of the UK which are positioned really well to build some of the new jobs that the AI economy implies.”

He compares the coming challenge to the Covid pandemic: some governments prepared early and paid lower costs, while others were caught flat-footed. The UK, he thinks, needs something like a Cobra for AI – a standing capacity within government to plan for disruption before it arrives. “If you just had 10 to 20 people tasked with this within government, the UK would be better positioned than any other nation on the planet,” he says. “It doesn’t take much to get ahead of this.” He has, he tells me, been saying exactly this to ministers all week.

Does he agree with Amodei’s prediction about the coming bloodbath for white-collar workers? “I don’t think this is borne out in current data,” he says. “My view is that it’s more likely that something far harder to predict happens.” He expects difficulty in certain types of early hiring, particularly in the knowledge worker and software engineering sectors; an explosion of new jobs no one can yet name; and, eventually, vast changes when AI combines with robotics. But he cannot draw a clean line from here to there.

Is there an argument, I ask, for not building the technology at all? For pausing while we figure it out? “If there was a way to do a coordinated global slowdown on development, to give us more time as a society to deal with the changes it implies, that would be good,” he says. “Part of why I’m saying it is that maybe, if I say it, other executives will say it as well.”

In the meantime, Anthropic is blazing ahead. In February, Time magazine reported that the company had changed the central commitment of its responsible scaling policy, the founding promise that it would never train a model unless safety was assured in advance. Anthropic’s chief science officer, Jared Kaplan, said “it wouldn’t actually help anyone” to stop training models while competitors were advancing. When I put this to Clark, he says internal policies change all the time. “We didn’t drop our responsible scaling policy. We changed it.”

He adds: “The real question is: why isn’t there hard regulation here?” The answer is, of course, Washington. On Thursday, the day after I spoke to Clark, the White House abruptly postponed a much-awaited AI executive order that would have introduced voluntary federal review of advanced AI models. Trump’s former AI czar, David Sacks, who has accused Anthropic of “fear-mongering”, lobbied the president, arguing that the order would slow down American AI in a race against China.

“If you’re at the point where individual companies changing their self-regulation is such a big deal, that is a giant flashing sign that it should be a regulated thing,” says Clark. He is calling for mandatory pre-deployment testing for AI models, something closer to the way we regulate cars or aeroplanes.

A few hours later, Clark stands at a lectern and tells the audience a version of what he told me. The situation, he says, “isn’t ideal”. A powerful technology is being built by a small number of actors in competition with one another. Commercial and geopolitical rivalries are drowning out the larger questions about what this technology will do to the species building it.

He says he is trying to find a way to live with this. Then he carries on.