The new age of algorithms: How it affects the way we live
'Big Data' impacts how we work, elect our presidents, and play tennis. It also affects the way we're watched.
(Page 2 of 7)
Want to get elected to public office? Put a bunch of computer geeks in a room and have them comb through databases to glean who might vote for you – then target them with micro-tailored messages, as President Obama famously did in 2012.Skip to next paragraph
Subscribe Today to the Monitor
Want to solve poverty in Africa? Analyze text messages and social media networks to detect early signs of joblessness, epidemics, and other problems, as the United Nations is trying to do.
Eager to find the right mate? Use algorithms to analyze an infinite number of personality traits to determine who's the best match for you, as many online dating sites now do.
What exactly is Big Data? What makes it new? Different? What's the downside?
Such questions have evoked intense interest, especially since June 5. On that day, former National Security Agency analyst Edward Snowden revealed that, like Ms. Mandelbaum or Rothman, the NSA had also asked a question:
Can we find terrorists using Big Data – like the phone records of hundreds of millions of ordinary Americans? Could we get those records from, say, Verizon?
The dark side of Big Data involves much more than Snowden's disclosure, or what the US does. And what made Big Data possible did not happen overnight. The term has been around for at least 15 years, though it's only recently become popular.
"It will be quite transformational," says Thomas Davenport, an information technology expert at Babson College in Wellesley, Mass., who co-wrote the widely used book "Competing on Analytics: The New Science of Winning."
What exactly will it transform? To find out, let's go back to the beginning.
* * *
Big Data starts with ... a lot of data. Google executive chairman Eric Schmidt has said that we now uncover as much data in 48 hours – 1.8 zettabytes (that's 1,800,000,000,000,000,000,000 bytes) – as humans gathered from "the dawn of civilization to the year 2003."
You read that right. The head of a company receiving 50 billion search requests a day believes people now gather in a few days more data than humans have done throughout almost all of history.
Mr. Schmidt's claim has doubters. But similar assertions crop up from people not prone to exaggeration, such as Massachusetts Institute of Technology researcher Andrew McAfee and MIT professor Erik Brynjolfsson, authors of the new book "Race Against the Machine."
"More data crosses the Internet every second," they write, "than were stored in the entire Internet 20 years ago."
A key driver of the growth of data is the way we've digitized many of our everyday activities, such as shopping (increasingly done online) or downloading music. Another factor: our dependence on electronic devices, all of which leave digital footprints every time we send an e-mail, search online, post a message, text, or tweet.
Virtually every institution in society, from government to the local utility, is churning out its own torrent of electronic digits – about our billing records, our employment, our electricity use. Add in the huge array of sensors that now exist, measuring everything from traffic flow to the spoilage of fruit during shipment, and the world is awash in information that we had no way to uncover before – all aggregated and analyzed by increasingly powerful computers.