UK considers ban on X as anger over sexualised AI images grows

The UK is hoping to build an ­informal “coalition of decency” in its response to Grok and X after the chatbot began enabling users to create non-­consensual sexualised images of women and children.

Last week Keir Starmer said an update of Grok – an AI tool which allows users to alter clothed images of women and children by making them appear in bikinis and sexually suggestive poses – was “disgraceful… disgusting, and it’s not to be tolerated”.

The Internet Watch Foundation (IWF), which monitors child sexual abuse material (CSAM) online, claimed it had found illegal imagery of girls aged 11 to 13 on the dark web that appear to have been generated using the Grok app.

In response to the allegations, Grok’s parent company, xAI, sent The Observer an automated email: “Legacy Media Lies”. Previously, Grok said it had “identified lapses in safeguards and [is] urgently fixing them”.

Ofcom, the media watchdog with responsibility for online safety, is undertaking a quickfire ­investigation, expected to conclude as early as this week. The prime minister said he had told the regulator that “all options (should) be on the table”.

Technology secretary Liz Kendall said Ofcom would have her “full support” if it decided to block the platform owned by Elon Musk.

“No business model should be built on the exploitation and abuse of women and children,” Kendall told The Observer.

“The Online Safety Act includes the power to apply to the courts to block services from being accessed in the United Kingdom if they refuse to comply with UK law,” she said. “If we reach a point where Ofcom decides to use them, they will have the full support of this government.”

Musk accused the government of being “fascist”.

Government insiders told The Observer that while they believed an outright ban was unlikely, a range of other options was on the table.

Under the Online Safety Act, Ofcom can fine companies up to £18m or 10% of qualifying worldwide revenue, whichever is greater. Kendall said that she expects Ofcom to update on next steps – and the future in the UK of Musk’s businesses X and xAI – “in days, not weeks”.

Insiders noted that conversations were taking place with Australia, Canada and key member states in the EU, and that even Maga-friendly Republicans increasingly believed that “the will to uphold decency in public life is taking precedence over free speech”. JD Vance, the US vice-president, called the situation “entirely unacceptable”, David Lammy told the Guardian.

Not everyone agrees, however. Anna Paulina Luna, a US Republican congresswoman and ally of Donald Trump, tweeted that she would bring forward legislation to “sanction not only Starmer, but Britain as a whole” if a ban was implemented.

Conversations among allies are understood to be taking place about showing a united front rather than taking any coordinated action, given each country’s unique regulatory environment.

Australia’s prime minister Anthony Albanese has backed Starmer for raising the issue already, saying: “The use of generative artificial intelligence to exploit or sexualise people without their consent is abhorrent.”

Canada’s minister for AI, Evan Solomon, posted – on X – that his government was legislating “to include deepfakes as intimate images for the offence of publication of an intimate image without consent”.

At home, Starmer is under pressure, including from his backbenchers, over the government’s use of X as one of its primary vehicles for communication. Last week dozens of MPs, as well as the women and equalities select committee, said they would no longer be using the platform. Sarah Owen, chair of the ­committee, said: “It’s completely incompatible to stay on a site that platforms and creates... NCII (non consensual intimate images) with our work.”

Insiders said that while there was an “open conversation” about remaining on X, change was unlikely while the platform was still accessible.

There has been no government spend on X since 2023, which marks part of a wider trend: the social media site recorded a 58.3% fall in UK revenues from £69.1m in 2023 to £28.9m last year, according to the latest financial figures published at Companies House covering the year to 31 December 2024.

Even if action is taken against Grok, the row exposes the whack-a-mole AI environment that regulators – and citizens – now find themselves in.

Defending his platform, Musk reposted images of Starmer in a bikini that had been created by Google’s Gemini AI tool and OpenAI’s ChatGPT.

A growing market of chatbots and AI companion services explicitly permit sexual content, typically through niche apps and standalone websites offering erotic roleplay. Ofcom has already used its new powers against such services, fining the operator of an AI nudification site £50,000 for failing to introduce effective age checks to stop children accessing ­pornographic content.

For now, most mainstream assistants, with the exception of Grok, heavily restrict pornographic content. But OpenAI, the marker of ChatGPT, which had 15.1 million monthly UK users in July 2025, has said it plans to debut “erotica” in the coming months. It is not clear exactly what shape these changes will take and whether they will involve the generation of sexualised images, audio or video in addition to sexual roleplay with the chatbot.

“I think it is extremely difficult to launch sexual content generators safely and responsibly,” said IWF chief technology officer Dan Sexton. “When the specific purpose is to create sexual images, videos and text, the risk of misuse is incredibly high. Furthermore, this kind of technology can often serve to reinforce misogyny and gender inequality – root causes of violence against women and girls (VAWG).”

OpenAI says its usage policies prohibit using any OpenAI service for exploiting, endangering or sexualising anyone under 18, and that the company strictly prohibits the use of its services for any non-consensual intimate content or sexual violence, or to use another person’s likeness without their consent.

“We also care very much about the principle of treating adult users like adults,” CEO Sam Altman posted on X in October. “As AI becomes more important in people's lives, allowing a lot of freedom for people to use AI in the ways that they want is an important part of our mission.”

The company is liaising with Ofcom before the release of the new feature.

Photograph by Jonathan Raa/NurPhoto/Getty Images