Organizing and synthesizing the world's energy data

Energy is a sector potentially well-suited to be mined with big data—the accumulation, integration, synthesis and interpretation of enormous amounts of data from disparate sources—Stuebi writes. 

Ann Hermes/The Christian Science Monitor
A solar energy meter takes readings at the home of Austin resident and resource-efficient home architect, Eric Rauser in this July 2008 file photo. Big data is being touted not only as a driver of increased efficiencies for companies, but also increased revenue, Stuebi writes.

One of the most hyped trends in high-tech is “big data”:  the accumulation, integration, synthesis and interpretation of enormous amounts of data from disparate sources.

Big data is being touted not only as a driver of increased efficiencies for companies, but also increased revenues:  as customers indicate or reveal their preferences through their behavior and choices, companies can then develop new products or services to offer in a very targeted and efficient manner to those most likely to purchase. 

With technology enabling faster mass-computation at lower cost, and a growing set of data available from all sorts of places and easily collectable from a thoroughly-connected world, big data could be a boon — especially for those industries for which cost reduction and efficiency opportunities are limited, and for companies desperately searching for new revenue streams.

Energy is a sector potentially well-suited to be mined with big data.  The industry is largely quantitative already, with scads of measured (and if not measured now, measurable) parameters across vast geographies with innumerable supply sources, processing facilities, transportation nodes, and consumption points.  And, for the most part, energy companies are largely stuck in a mode of selling commodity products, and need new twists and differentiators for which a premium can be charged.

Proactively, I went searching the Internet for the best insight, wisdom and perspective on the incursion of big data into energy.  Frankly, I didn’t find terribly much — which tells me that the space at the intersection of big data and energy is ripe for innovation.

One of the better pieces I found was this posting from early 2012 by Katie Fehrenbacher, “10 Ways Big Data is Remaking Energy”, which identifies the 10 types of data that will increasingly be mined in the energy sector:

  1. Weather
  2. Cell phone usage
  3. Thermostats
  4. Hadoop
  5. Solar/wind sources
  6. Electric cars
  7. Power lines
  8. Real estate
  9. Variable energy prices
  10. Behavioral analytics

Although this is a good list of the innovative places from which data will be gathered, it still leaves open for entrepreneurs to identify specific customer needs to be met and value propositions that can be developed from big data.

It seems that a big challenge will be in making data from disparate sources mesh accurately.  As this article from Intelligent Utility indicates, integrating “unstructured data” — especially from hand-written and other manually filled out forms — will be particularly difficult.

Big data in the energy sector will likely be a large opportunity for the major players in the IT sector — IBM, Intel, SAP, Oracle, Google, Microsoft, and so on.  My advice to them is that they will need to be patient:  the big players in big energy move glacially, and big data will take a long time to penetrate in a major way.  But, the opportunity is vast, so it’s probably worth the effort and the waiting…and waiting…and waiting.

You've read  of  free articles. Subscribe to continue.

Dear Reader,

About a year ago, I happened upon this statement about the Monitor in the Harvard Business Review – under the charming heading of “do things that don’t interest you”:

“Many things that end up” being meaningful, writes social scientist Joseph Grenny, “have come from conference workshops, articles, or online videos that began as a chore and ended with an insight. My work in Kenya, for example, was heavily influenced by a Christian Science Monitor article I had forced myself to read 10 years earlier. Sometimes, we call things ‘boring’ simply because they lie outside the box we are currently in.”

If you were to come up with a punchline to a joke about the Monitor, that would probably be it. We’re seen as being global, fair, insightful, and perhaps a bit too earnest. We’re the bran muffin of journalism.

But you know what? We change lives. And I’m going to argue that we change lives precisely because we force open that too-small box that most human beings think they live in.

The Monitor is a peculiar little publication that’s hard for the world to figure out. We’re run by a church, but we’re not only for church members and we’re not about converting people. We’re known as being fair even as the world becomes as polarized as at any time since the newspaper’s founding in 1908.

We have a mission beyond circulation, we want to bridge divides. We’re about kicking down the door of thought everywhere and saying, “You are bigger and more capable than you realize. And we can prove it.”

If you’re looking for bran muffin journalism, you can subscribe to the Monitor for $15. You’ll get the Monitor Weekly magazine, the Monitor Daily email, and unlimited access to CSMonitor.com.