It’s over for Keir Starmer – how has it come to this?

“It’s over,” one senior government source messaged me on Monday night. As I write, Keir Starmer is still officially prime minister. This morning he told his cabinet that he intends to fight on but it is clear that his leadership has run out of road. “He’ll never work it out for himself, his instinct will be to fight, he’s obstinate bordering on selfish,” according to a former cabinet minister. The decision has, however, been taken out of his hands. More than 80 Labour MPs have called for their leader to go and Starmer has lost the trust of his cabinet. Three influential junior ministers have quit. Others are expected to follow. Nobody in the government thinks the prime minister will lead them into the next general election. The only question is over the timing of his departure. “This has got to be done quickly,” a Labour grandee says. “We can’t have chaos for months on end.”

There will be a Labour leadership contest, possibly kicking off as soon as this week, so it is worth asking what went wrong for Starmer and how his successor can avoid making the same mistakes. Less than two years after winning a landslide general election victory the prime minister is among the most unpopular leaders in polling history. Labour MPs out campaigning ahead of last week’s local elections heard voters expressing “hatred” of Starmer rather than just disappointment or disillusionment with their party.

It’s strange because the prime minister is not a hateful figure. He is inoffensive, in many ways a decent man trying to do the right thing. His supporters suggest that he has fallen foul of an anti-incumbency mood that is sweeping the world. There is something in that. The driving force in modern politics is this hostility to politicians and the anti-establishment sentiment inevitably hits hardest those who are in charge. But that is not enough to explain the speed and scale of the implosion of Starmer’s premiership.

I think the real reason for the catastrophic collapse in support is that Starmer has never been able to explain – and perhaps has never really known – why he wants to be prime minister. When I saw him on Saturday, he declared that he wanted to be in Downing Street for a decade but even as he was fighting for his political life he seemed oddly devoid of ideas about what that “ten year programme of national renewal” would involve. He was determined to put Britain at the “heart of Europe” but refused to look again at his manifesto red lines on the single market and the customs union. The same vacuum was apparent in the speech he delivered on Monday, which is why so many of his MPs turned on him.

Starmer has been able to analyse the mistakes of the past but he has never articulated a clear view of the future he wants to shape. He has therefore been unable to put in place a programme for government that could bring about the transformation in the public services and the economy that the voters are demanding. It means he has not lived up to the one word slogan he campaigned on at the last election – change.

There are lots of reasons for that. One is that Starmer was nervous of going all guns blazing for the two things that do drive him on in politics – the desire to ensure every child fulfils their potential and a support for human rights and international law. His political strategists told him these issues were unpopular or irrelevant so he always hedged and havered over them. A few months ago, when he joined me for The Observer Walk, I asked him directly why he wanted to be prime minister. “Young people have been collateral damage for the failure of past governments and I intend to turn that around,” he replied. But within days Downing Street was declaring that the cost of living crisis was his only priority. The walk took place in a corridor. His aides would not let the prime minister go outside, citing rain and security, which somehow seemed symbolic of a leader trapped by his office.

Another possible explanation for the Labour leader’s failure to cut through is psychological. Starmer’s own personal story is one of triumphing against the odds – he was the first in his family to go to university and as a child had to deal with the trauma of his mother’s debilitating illness – but those who know him well think he has been unable to tap into the emotional power of that narrative because he feels guilty that he has “succeeded” in the eyes of society while his siblings struggled. At his mother’s funeral, a few days after Starmer became an MP, his father’s eulogy merely stated that he had four children “now all grown up and leading useful lives”. This helps to explain why Starmer so often seems buttoned up and unable to form an emotional connection with the voters. He has suppressed what could have been his political superpower.

In the end though it is deeds not words that count in politics as in life. Starmer seems to think that making a speech or changing the faces around him (by getting rid of Morgan McSweeney or bringing in Gordon Brown and Harriet Harman) will be enough to save him. It won’t be. Those are human resources and communications issues, when what Labour needs is a governing project. As Tony Blair likes to say, good policy is good politics. It’s no good talking about there being no return to the status quo without having a plan to deliver something different.

There is an almost bizarre lack of interest in policy in Downing Street. Starmer is on at least his fifth head of policy. It may be more. At one point there were three people who thought they were in charge of the policy unit but no-one who was coming up with any interesting ideas. Labour has more people with strategy in their job title than you can shake a stick at but it has downgraded the importance of substance. The prime minister declared this week that incremental change “won’t cut it” but his government has been defined by incrementalism because it has not dared to press ahead with radical change in so many areas, from welfare reform to social care, the curriculum and assessment system or digital ID. As one senior figure in Whitehall puts it: “the prime minister won’t make any fucking decisions.”

This is why Shabana Mahmood was among the first cabinet ministers to tell Starmer he needed to consider his position. The home secretary is a radical who wants to get on with reforming the prisons, the courts, the police or the immigration system. Her ideas are sometimes controversial but she knows what she wants to do and has the courage to make the political case for them. Her approach to policy is “go big or go home”, as she told Blair recently, whereas Starmer’s attitude as prime minister has been “go small or U-turn”. As Jess Phillips put it with devastating clarity in her resignation letter: “The desire not to have an argument means we rarely make an argument, leaving opportunities for progress stalled and delayed.”

The next few days and weeks will be defined yet again by a Westminster game of who’s up and who’s down. Is Wes Streeting going to pull the trigger? Will Andy Burnham get back into parliament? Is Angela Rayner or Ed Miliband going to represent the soft left if he does not? It would be a mistake, though, for Labour to think that just changing the personality at the top is enough or to see the leadership contest as a battle between left and right. Charisma is important but what matters more is having a policy platform that will make the country work better for the voters in tangible ways rather than soothing the ideological yearnings of the Labour Party.

Photograph by Lauren Hurley / No 10 Downing Street