NHS England pauses AI project fed health data of 57m patients

One of the biggest AI projects in the NHS has been paused after concerns were raised that it may have used the health records of 57 million people without the correct permissions.

Foresight, a project using Meta’s open-source AI model, Llama 2, was designed to “predict what happens next based on previous medical events” and fed with data from millions of patients’ records, stripped of identifying information and addresses. Experts have warned that even anonymised health records can retain enough information to make individuals identifiable. Meta gained no access to patient data.

The project was signed off using a process created during the pandemic to fast-track Covid-related research. However, it is not clear that it primarily relates to the pandemic, and one researcher expressed concern that it was possible to make a tenuous link to Covid to gain approval. “How is it informing our understanding of Covid?”, they asked.

Both the Royal College of GPs (RCGP) and the British Medical Association have raised concerns with NHS England that proper procedures were not followed.

They accuse the research consortium, led by Health Data Research UK, of neither informing nor consulting an advisory body of doctors before feeding patient data to Foresight, and suggested that this could undermine public confidence and professional support for the use of AI systems in the NHS. Both bodies have asked NHS England to refer itself to the Information Commissioner’s Office, which oversees data protection.

“GPs take the management of their patients’ medical data very seriously, and we want to be sure data isn’t being used beyond its scope, in this case to train an AI program,” said Professor Kamila Hawthorne, chair of the RCGP council. “Patients need to be able to trust their personal medical data is not being used beyond what they’ve given permission for, and that GPs and the NHS will protect their right to data privacy. If we can’t foster this patient trust, then any advancements made in AI – which has potential to benefit patient care and alleviate GP workload – will be undermined.”

An NHS spokesperson confirmed that research using the Foresight model had been paused and said doctors could review the general data-sharing agreement used for the model even if they had not been consulted on that particular project.

Photograph by Lynsey Addario/Getty Images