How data took over the world

“Data is the new oil,” the mathematician Clive Humby said 20 years ago. It’s a statement that seems to have been borne out in the intervening decades, as data became a major element in the business model of big tech. With the rise of AI, its value is now of critical importance. Two new books take different approaches to the long history of data, and how we should consider its use now.

In Data Empire, Roopika Risam adopts a global view of data’s backstory, starting with prehistoric cave paintings in southern France, and exploring the idea that humans are a “data species”. Throughout, the rise of empires, from the ancient world to the present, is linked to the use of data as a means of control. While not everything is data, Risam writes, it is everywhere. Her chronologically arranged chapters begin with a real-life “use case”. Thus we are introduced, in fictionalised scenes, to the royal court in ancient Babylonia, to Musa, the King of Mali, in 14th-century Cairo, and to the ruler of Tahiti in the 18th century, who encounters the botanist Joseph Banks. These are imaginatively brought to life, but aren’t really necessary, as the core story of how humans have sought out and used data over millennia is interesting enough.

Risam, though, doesn’t engage enough with the surviving evidence, of which there is a surprising amount. Tens of thousands of clay tablets from the ancient archives of Mesopotamia exist in museums and libraries around the world, and later sources of data are in even greater abundance: medieval building ledgers, taxation records in parchment rolls, paper account books. We are given tantalising glimpses: the tallies of grain in ancient Egypt, or the printed bills of mortality in 17th-century London, for example, where the numbers of people dying in each parish in the city, and more importantly the causes of death, were collected and published. Samuel Pepys used this data to avoid visiting parts of the capital with high levels of contagion.

The book suffers from a key problem: lack of definition. “Data” is used variously to mean information and knowledge, and the varying forms of each are not clearly defined. Thus we are told about the House of Wisdom in medieval Baghdad, with its famous library, and about the importance of translation and the way that scribal copying helped disseminate ideas. But these books contained texts, which are (at least until recently) different from data.

Where her account is strongest is in the final chapters, on our present world, in which the economy is now dominated by data-driven companies, with huge implications for society on how the info they trade in is derived. Amazon (especially Amazon Web Services), Google and Palantir are among the most recognised of these companies, but Risam highlights others such as Cross Match Technologies and Morpho, which are selling mass-population data surveillance “solutions” to governments across Asia, Africa and Latin America. These firms, she argues, remain hidden from democratic oversight.

The new book from Carissa Véliz, a philosopher whose previous work has focused on online privacy, switches the balance from the historical development of data to its place in the present. Prophecy describes in detail the story of how humans have sought to predict the future. She deftly explores the historical development of prediction, looking at the birth of the algorithm (a term deriving from the Arabic scholar al-Khwarizmi, and which came to refer to mathematical rules in calculations) and the growth of insurance markets in early modern Europe, pointing out the similarities of these advancements with practices such as divination. “Part of what people are buying when they pay for a divination or insurance is peace of mind,” she writes.

Véliz also charts the development of the use of data in government planning, highlighting the recent and rapid growth of government agencies using algorithms that “drink in surveillance data”. This is creating a huge financial boom for companies such as Palantir, which sell their services as well as data to governments, and can control the kind of predictions that this combination of data and algorithm is able to generate. In this way, Véliz argues, the balance of power is shifting. Governments started off employing big tech companies as outsourced agents for undertaking large-scale bureaucratic tasks, but we are now entering a phase in which the tables have turned: the state is becoming an agent of the ambitions of big tech.

The ancient Oracle of Delphi is a recurring reference point for Véliz, and she posits AI as its modern successor, with the tech bros as the new prophets (Larry Ellison’s software company Oracle is a screamingly obvious case here). Her book is as much a warning as it is an analysis of our modern data predicament. The more we rely on personal data to feed AI, the closer we come to creating a police state through mass surveillance. More than this, as Véliz highlights, AI’s dependence on data is “replacing creativity, critical thinking and quality in too many endeavours”. For Risam, this same combination “dehumanises at scale”.

What these titles agree on is that we are now in an era in which the power of data to influence our everyday lives is enormous – and most people are unaware of it. Véliz, a specialist on AI ethics, points out that we must not confuse prediction with truth. For Risam, “algorithmic intuition converts a correlation based on statistics into certainty … probabilities treated as fact”.

Both books leave the reader asking questions about what we can do to deal with the new data regimes, whether controlled by both the state or by big tech. Véliz argues that regulation can only be part of the answer. Our personal behaviour is ultimately the best response. She leaves us with 10 lessons in prediction at the end of Prophecy, urging us, for example, to prepare and not predict, and to increase our exposure to serendipity. The most important events in our lives, she argues, are likely to be the least predictable. The last of her lessons is the most appealing: “live well”.

Data Empire: How Information Shaped Human History by Roopika Risam is published by Torva (£25). Order a copy from observershop.co.uk for £21.25 (15% off RRP)

Prophecy: Prediction, Power and the Fight for the Future, from Ancient Oracles to AI by Carissa Véliz is published by Swift Press (£25). Order a copy from observershop.co.uk for £22 (10% off RRP)

Photographs by Konstantin Kulikov/Alamy, Michaela Lam