Cognoscenti Cognoscenti

Support the news

Learning To Love Big Data

This article is more than 7 years old.

In the wake of the NSA surveillance leak, it’s easy to see why “big data” has a big image problem. Explaining why he risked prison to reveal secret documents, Edward Snowden said that he didn’t want, “to live in a world where everything I do and say is recorded."

Tremendous harm can come from total surveillance (just think of East Germany’s Stasi), and few Americans would want that sort of world.

Yet, this dystopian vision obscures a more Utopian perspective on the use of big data that goes well beyond issues related to national security and surveillance. Big data can be a force for good, and it is increasingly being deployed to strengthen our health, wealth, and democracy. For this to happen, and for us to avoid learning the wrong lesson from the NSA leak, we must implement safeguards that make big data not solely the province of commercial enterprises and intrusive governments: Big data belongs to all of us.

Big data can be a force for good, and it is increasingly being deployed to strengthen our health, wealth, and democracy.

Virtually all Americans leave digital breadcrumbs as they move through their lives. Scientists and businesses — not just government agencies — now have the capacity to inconspicuously track the behaviors, purchases, movements, interactions and thoughts of whole nations of people, in real time. Because of credit cards, cell phones, online social networks, health information systems, and so on, such “massive/passive” data are revolutionizing our ability to understand — and change — human behavior in ways that save lives and money.

For instance, by taking advantage of big data and new insights into human social network structure, we recently invented a whole new way to forecast epidemics. In 2010, by passively monitoring students and their social networks at Harvard University, we were able forecast the onset of the H1N1 epidemic up to six weeks in advance; similar techniques can be used on a global scale by taking advantage of online data available for millions of people. Many lives can be saved with such early warnings.

Or, in the 2010 election, in an experiment involving 61,000,000 Americans, scientists working with Facebook introduced very small changes in messages in people’s newsfeed (specifically, they let them know if their friends had voted), and they showed that it’s possible to motivate hundreds of thousands of people who would not otherwise have voted to do so, thereby improving the strength of our democracy.

There are many other concrete applications of data science across an amazing range of challenges. We can use big data to develop new ways to target behavioral interventions in settings as diverse as American schools (to reduce bullying or drug use) or Indian villages (to improve neo-natal care or agricultural practices). Big data can be used to isolate the social roots of dementia (by evaluating how social isolation is a risk); to reduce devastating market swings (by evaluating how information goes viral); to fight Chinese censors; to synchronize traffic flows; to monitor and respond to hurricanes; and to foster optimal prescription drug use by millions of people (by mapping the diffusion of innovation among physicians).

There is no need to fear big data any more than any other innovation.

Like any technology, big data can be used for good as well as evil. This is an age-old dilemma when it comes to scientific and technological advances. Do we use nuclear physics to build bombs or to free us from fossil fuels? Do we exploit behavioral insights to help people improve their lives or to exploit their weaknesses?

Whether big data will be a positive or negative force depends on two key factors: democracy and transparency. If data technology continues to be held primarily in the hands of governments and corporations, we put society at  risk. But if it falls under democratic control, and is used with transparency, we can help assure a Utopian rather than dystopian future. We regulate all sorts of technology — from financial instruments to medicines — and big data should be no different. There is no need to fear big data any more than any other innovation.

We can also make it easier for people to share their data. Just like people can donate their bodies for research, why not provide them the ability to donate their data to science? To more rapidly usher in an era when big data is used for good, we should encourage the private sector to make it easy for people to voluntarily share their data openly — with each other, with scientists, with other companies, and even with the government. Diverse data wikis are indeed emerging just for this purpose.

For the past century, people have looked to the physical and biological sciences to solve important problems, and they have reaped great rewards from discoveries as diverse as computers, plastics, and antibiotics. But, in the 21st century, data science is enabling us to develop startling new insights into individual and collective human behavior. And this, in turn, offers powerful new tools for improving human well-being. We should not stop this progress.


This program aired on June 18, 2013. The audio for this program is not available.

Support the news