To shed some light on the subject, we sought the expertise of renowned Austrian data scientist Viktor Mayer-Schönberger, a professor of Internet Governance and regulation at Oxford University. Mayer-Schönberger’s latest book, Big Data was shortlisted for the Financial Times/Goldman Sachs Business Book of the Year Award in 2013 and we have spoken to him about how Big Data will be shaping our future, and the potential danger that it poses.

Do we have a moral obligation to use data and information in a way that our society have always needed – meaning to help?

Data is, as a Taoist would say. What really matters is our human ability to collect, analyze and store that data (the latter so that it can be reused many times). For all of human history we have tried to make sense of the world around us, in part by observing it.

So data has always been with us, but dealing with it was so costly and time-consuming that we devised our entire methods, structures, and institutions for making sense of the world through data with the assumption that we could only gather a small sliver of it, but never capture it comprehensively. That is changing right now, and it should prompt us to rethink how we use data. Using data at scale – that is comprehensively, and not just small portions of it – greatly improves our ability to understand the world we live in, not just in general and abstract terms, but in all its beautiful complexity and detail.

That to me is our human duty: to realize that we have to be humble, that we do not know as much as we think we do, and that we need to utilize data to know more and to know better. It is the same ethical thought that drives us to use land sustainably, and to use our production capabilities to improve everyone’s lives.

In your work with government & university faculties, think tanks and liaising with world leaders what can you gauge about their moods based on Big Data?

My sense is that Big Data is gaining traction. A year ago, this was the topic of a small group of interested folks, now it is becoming more mainstream. And it is obvious to me why: at its core Big Data promises to improve human decision making. That must be of particular interest to everyone in leadership positions. But with all the hype surrounding Big Data, and with the many buzzwords it is at times difficult for these decision-makers to differentiate between the signal and the noise, so to speak.

Patrick Wolfe at the University of London mentions that the Big Data is “the wild west right now.” He goes on to say, “People who are clever and driven will twist and turn and use every tool to get sense out of these data sets, and that’s cool. But we’re flying a little bit blind at the moment.” What do you think he means by flying blind at the moment? And would you agree

Yes, it is early days, and thus unruly, and wild, and hectic, and rough. It feels like the web did around 1999 or 2000. Many of the tools we need for robust and effective Big Data analysis have not been built, or are at least not available in easily packaged forms. Big Data analysis is still being home-grown, much like web pages used to be hand-coded in HTML before the days of powerful CMS. But this also means that there is a wonderful open space for innovation, for great new ideas, new tools, new ways – and such opportunities attract a wide variety of people, some with great ideas and others perhaps not so.

Turning to the subject of the right to be forgotten, The New Yorker in a recent article called you, “One of the intellectual godfathers of the right to be forgotten.” You mention in the article that “The roots of European data protection come from the bloody history of the twentieth century,” An interesting perspective. Can the Government’s policy and regulations potentially stop data abuse from happening?

There is only one way by which we can make sure that data is not abused in the future and that is to not collect it and not to use it, and to not store it. But I believe that is not a realistic option. Data offers so many potential near-term benefits that small long-term risks may on balance not be worth protecting against. That does not mean, however, that we should not be vigilant about privacy – on the contrary it implies that we need to put in place more effective laws regulating the use of Big Data to protect against its dark sides, because we can no longer hope to protect against them by not collecting the data in the first place.

As a large corporation Google is almost an invisible partner to our society. It has a silent reach into almost every part of our lives. It apparently does not represent a force for evil, but nevertheless it is at a point where even the EU is trying to break it up. How do you view Google’s intentions and the work it has done over the last decade?

There is no one “Google” – like every large organization it embodies many views, many preferences, and many goals. Some are commercial, others may be more idealistic. I don’t think we should break Google up just because it is big and powerful. But Google needs to realize that with scale and power also comes heightened responsibility (and public scrutiny), and they need to demonstrate that they accept that responsibility.

You’ve said Big Data is on par with the Internet (or perhaps even the printing press) with regard to how it will change the way we think about business, health, politics, education, and innovation. Are there industries that we have not prophesized that it will affect as of yet?

Of course. I am learning every day of new and intriguing ways to use Big Data. Recently I was involved in thinking about how manufacturing could benefit from Big Data, and we realized that large manufacturers could turn themselves into Big Data analysts, and turn the tables on their supply chain relationships. That was a distinct “a-ha” moment for me.

Can you give us three clear examples of how you envisage Big Data being used in the near future?

The first is that I see Big Data fundamentally changing health care. Currently every one of us is always compared to the average human being to determine whether we are sick or not. So sickness is essentially defined as ‘deviating from the average’. But we are all different, and the fact that we deviate – in one dimension – from the average may not necessarily signal that we are sick, or that we have a problem. Rather than defining sickness in that strange way, we should define sickness as a sudden change in our body over time. We haven’t done such evidence-based medicine at scale yet because we did not have individual data. But soon we will. And that will let us not only live longer, but better.

Similarly, Big Data will change how we learn. So far, the little data we have collected in our schools captures learning outcomes, but it does not tell us much about the process of learning. Wouldn’t it be much better to focus on improving how we learn, rather than comparing exam results? Would it not be better to help every student reach their personal potential rather than comparing them relentlessly to a national (or international) standard? If Big Data is employed properly, this is precisely what we can do, as we now have the tools to capture and analyze the relevant data.

Thirdly, of course, is transportation. With Big Data cars will not only drive themselves, but as a consequence, our entire notion of what transport is will change. It is a paradigmatic shift that will drastically lower emissions, traffic jams, and accident rates to the point where doctors are already predicting an impending “organ donor crisis” because of the steep drop in traffic fatalities.

Some suggest that in the near future Big Data analysis may bring about a situation in which judgments of culpability are based on individualized predictions of future behaviour. How do you know this isn’t happening already?

This IS happening already. In 30 states in the US, the decision about whether somebody is freed on parole is, in part, made by a Big Data algorithm predicting the likelihood of that person being involved in a future homicide. In a growing number of cities around the world, Big Data is already employed to predict where and when the next crime is likely to happen, and policing resources are allocated accordingly (note the troubling, self-fulfilling nature of such predictions!) In the Big Data age we need to act swiftly to protect human volition and free will, much like we guaranteed access to justice and the rule of law in many democratic nations the 20th Century. We are quite likely to decide as a society that there are certain forecasts we just don’t like to be made, and for a certain limited number of things, we prefer ignorance over insight. Now is the time to have this discussion.

It seems certain that Big Data will change the way we process and think about problems – from a governmental level to an individual level. Perhaps you could enlighten us on some of the ways you can imagine this will happen?

On the public policy level, we could use Big Data to better understand demand for public transportation, especially as demographics and work practices change. We could then build public transportation infrastructures that are motivated by actual demand rather than politics. Companies could use Big Data not just to sell more of their products, but to use it to gain insights into what other products and services might be useful to offer. Individuals could use Big Data to better model their mobile phone usage preferences, so that they could choose the plan that is right for them (rather than the one with the strongest sales pitch).

Do you think there needs to be some kind of overall charter – much like the Declaration of Independence charter – outlining how Big Data is to be mined, stored and used by future generations?

Yes, I do think we need a set of ethical principles outlining how to use Big Data, and what are its limitations.

I recently read about Kenneth Cassman, an agronomist at the University of Nebraska, Lincoln, who has unveiled a new interactive mapping tool for agriculture, committing to the fact that there will be 9 billion people in the world by 2050 and that the world needs more harvestable food. Are people like Cassman the Marco Polo’s of our time, charting unexplored territory, creating new ways to harvest the bad news and turn it into good news?

Yes, they are. And there are many more all over the world, who help us improve our use of natural (especially energy) resources to achieve a higher level of sustainability. It’s really interesting to see how alternative electricity generation (solar, wind) has taken off, but as these sources produce electricity at different times during the day (no solar energy during darkness for instance) and have limited ability to transport electricity over long distances, we need to find the most useful mix of electricity generation over the time of the day to avoid having to over-generate for peaks. But that requires us to better and more precisely understand demand. Similarly, electric cars like the Tesla will not only require a new infrastructure of refueling spots, but Big Data to tell us where and how many are needed.

I like how you use the metaphor in your book of  “Icarus and the over reliance on Big Data”, you mention that “We must guard against over reliance on data, rather than repeat the error of Icarus, who adored his technical power of flight but used it improperly and tumbled into the sea.” Do you think there could be a lot of hubris attached to people relying on big data to solve the issues of our time?

There always is the danger of relying too much on data, on imbuing it with more meaning than it actually has. Doing so is abusing Big Data, and essentially succumbing to a dictatorship of data that seems to offer insights, but only continues to keep us in the dark.

What is it that you are working on at the moment? Perhaps you can give us some insight into how you work as well. Your day-to-day work.

I look at the role of risk in society, and how Big Data will affect that. It keeps me awake at night, and I take walks to come to grips with it, and hot showers. My hope is that I can help ensure a future for our children (including mine) that is enlightened by our improved understanding of reality, rather than endangered by Big Data’s dark sides.

What would you want your legacy to be?

That data matters. And that I am forgotten.