In a digital world, the use of outdated stats simply doesn’t add up

Economic statistics are artefacts of the mid-20th century, invented and honed to assess industrial economies made up of workers and citizens who dutifully responded to government requests for regular information. Now, as the first quarter of the 21st century draws to a close, that system is in disarray.

On both sides of the Atlantic, the most troubling statistics have been monthly employment reports that, along with inflation and GDP, form the basis for how policymakers assess the health of economies.

The Labour Force Survey in the UK was repeatedly delayed over concerns about accuracy, which culminated in the sudden resignation of the head of the Office of National Statistics in May. In August, after a particularly large revision in the unemployment report, the head of the Bureau of Labor Statistics (BLS) in the US was fired by President Trump, who claimed political bias. While that accusation is almost certainly unfounded, the problem of increasing inaccuracy in the monthly report is real and has been brewing for some time.

The main culprit in both cases is that reports rely heavily on surveys; in the American case, the BLS canvasses 60,000 people each month to gauge the employment situation. For much of the 20th century, that worked smoothly. People had one landline phone, without caller ID, that they tended to answer, and when told they were talking to a government official who wanted to ask some questions, they were even more likely to respond.

Today that framework is broken. Few people answer an unrecognised number on their smartphone, and they are as likely as not to refuse to respond to questions from the government, suspecting that it may do more harm than good. The result is plummeting response rates to surveys (in the UK, it went from about 39% pre-pandemic to 20% now; US rates are better but also declining precipitously), which makes it increasingly hard to come up with accurate data and hence accurate statistics.

This problem isn’t confined to the US and UK, although these recent examples are the most acute (last week it was revealed the ONS had undercounted by half heroin and opioid deaths in England and Wales).

It isn’t just a problem of declining participation in government surveys. The entire statistical framework is designed to measure a mid-20th century world when the core of most economies was industrial production of physical goods, when workers were physically located in one place doing one job and when trade consisted almost entirely of stuff made in country X and shipped to country Y.

That isn’t our world today, particularly in the more advanced economies. It’s a much more complicated mix of goods and services, of workers who are remote, or in the gig economy, or self-employed, or move from job to job with increasing frequency.

It’s one where capital flows instantly and freely from country to country, with trillions in digital currencies, where services are exchanged that never touch a physical national border. And many digital services are free to the user, be they Google maps or basic AI chats, requiring us instead to pay with our data.

Measuring output, inflation and employment for economies that are digital and service-oriented is not what our suite of statistics was designed to do. We are using an antiquated, outdated statistical roadmap, and it’s no wonder that we are increasingly mismeasuring our world and getting lost.

How should we update our statistical framework? Efforts are being made to find new ways of gauging our world.

The Billion Prices Project at MIT sought to use data-scraping of prices from the internet to create inflation measures that were more current and didn’t rely on surveys. Others, including the Stanford Digital Economy Lab, are honing digital and AI-enhanced tools to crowdsource data on employment that could eventually replace the antedated survey method.

For now, we are left with a wonky compass that often doesn’t know where true north is. That is deeply disconcerting for governments and policymakers who use this data to inform potentially momentous decisions about interest rates, tax and employment policy and sovereign debt.

Until we renovate our data collection and statistical methods, we would do best to rely less on these official but flawed numbers and expand our sources of information. Thankfully, in our AI and internet age, those sources are more bountiful than ever and more digestible. We are blessed with digital tools even as we rely on analogue methods to measure economies. That is a silver lining in an otherwise muddled time.

Zachary Karabell is founder of the Progress Network and author of The Leading Indicators

Photograph by Fox Photos/Getty Images