Data Should be the Intel "Outside": The Power of Data Network Effects

The folks at Puhpin had a great comment they posted to our last blog entry on “free public data“. I thought there was enough interesting content to expand on the comment thread with another blog post. The Pushpin team did a great job providing far more nuanced thoughts on the issues of “for fee” data. At the end of the day my issue is truly with the government/s for not providing the data in easy to use formats or even open standard non-proprietary formats. In an open market anyone is free to take that government supplied data, make it easy to use, and charge a price the market is willing to pay. In addition to making the data easy to use many vendors also add an additional layer of quality assurance and many times value added data derivatives like forecasts.

There are many instances where vendor supplied data is truly value added and worth the money an end user pays, but there are also situations where it is not and there is a better alternative. Take for instance the 2000 Census data ESRI provides to Pushpin to resell – the added work there is taking the boundary files provided by Census and joining them to the data tables provided by the Census. I’ll be the first to admit it is tedious to do all the database joins, and it requires having pricey GIS software, but in my opinion the ratio of value add to price is way out of wack.

That is the philosophical difference with GeoCommons. If you have a community of people willing to put in that little bit of work to extract the data from places like Census and share it with the community you get a network effect. Since the data goes in under Creative Commons, anyone can take that data and combine it with their data or anyone else’s contributed data. Allowing any user to make something new and innovative with the collective data. Anytime you work to create a dataset/database there is value created and work done. Every member of OpenStreetMaps GPS-tracing roads has put in solid sweat equity, but they choose to contribute that to the community because the collective value of that data is far greater than its value alone.

In the end I believe this helps the data vendors because there is more data the market can mashup with the vendor data (vendors benefit from the network effect also). There is also a larger market of people that realize the value of the data because the barrier to entry to experience it has been removed. That said, I believe it also means the data providers are really going to have to add true value and not just do a few database joins. The real value comes in the technology and not the raw data itself. The data is what enables the technology to be more valuable.

Tim O’Reilly states that one of the key value drivers for Web 2.0 is “Data is the Intel Inside“. Specifically O’Reilly cites NAVTEQ’s proprietary database of streets as a big value drivers for many GeoWeb applications. I agree that databases (i.e. SQL is the new HTML) are creating new value propositions, but now the value is having data on the “outside” not the “inside”. The walled proprietary gardens of “inside” data are being trumped by open source “outside” data that allows a network effect to be created. With data on the “outside” not only can new combinations (data mashups) be created, but the data itself can adapt (like OpenSteetMaps and TomTom). In response to Brady’s post on the Nokeia acquisition of NAVTEQ O’Reilly comments, “the real question is going to be whether there’s a web 2.0 answer (i.e. a user-generated content) answer to the expensive data development and curation currently employed by Navteq.” I think the answer is a resounding yes and as standards like KML 3.0 progress and technologies evolve around them, the power given to the user so they can contribute meaningful data and context is only going to increase. The real value is in the technology that allows the data to be delivered, mashed up, and interconnected.