Sharing and Archiving Public Data

Over the recent several years governments have begun publishing more of their quality local data online. The reasons vary but typically range from inventive and progressive thinking administrations to decreed government mandate.

Publishing data online is not as straight-forward as it may first appear. It is easy for an organization to simply create a webpage that links to data files such as spreadsheets or shape files. They’ll include the title, some description, probably a publication date and some licensing information. This fulfills the simplest idea of putting data online but is overly simple, problematic to maintain, and difficult to find and use. Government organizations start building out more and more pages to link to and list their data – probably through a small team or even a single person. Over time this person gets other duties or moves on and the result is a hodgepodge of data online that is old, disappears and is unused by citizens and other organizations.

The end result is a surging publish and fade of public data. An initial release and excitement but then subsequent atrophy. From an archival perspective gathering and maintaining this data is untenable.

There are several problems that create this end result. The data is published in ad-hoc and unique silos. Institutionally there is likely little effort to put in a process or a plan for updating and maintaining the data in a way the encourages more members rather than fewer to participate in publishing their data. In addition the data are separate from tools to use the data – the equivalent of giving someone flour and water without the utensils to cook and prepare the ingredients for something more generally consumable.

Fortunately, in conjunction with the availability of public, open data there have been online tools that make it easy for any organization to publish, update, and track their data for users.

Providing easy and free to use tools

A little over 3 years ago we launched “Finder”, the first component of GeoCommons that focused on providing a simple way for anyone to upload, link, and share data online. Since that launch the community has contributed more than 80,000 public datasets. Demographics, weather, imagery, social media data, zoning and historic districts, and more. Community members spent time and effort to find and contribute that data so that themselves and others could easily access the data in a variety of open and common data formats.

Through this process we’ve learned a lot about how to make data sharing easy and valuable. For government organizations there is an excellent opportunity to quickly and effectively publish data, update and track it’s usage. Tools built into GeoCommons make it easy for someone to quickly visualize and analyze the data creating real value for the effort that went into gathering and sharing the data. And users often have their own tools they’re comfortable using, so we developed toolbars for Excel and ArcGIS that allow a user to publish and access data from interfaces they’re already comfortable with.

Through multiple organizations each contributing their data the community is even more empowered. Local governments, transportation agencies, academic and scientific institutes all contribute their data – making it an interesting melting pot of multi-domain data to mix and match. The value of contributing data isn’t limited to the single dataset but the many permutations of combining this data with other data.

GeoCommons offers support for many open standards and formats that government organizations are encouraged, or sometimes required, to publish – removing the burden of having to build their own infrastructure. By creating a simple web page for each dataset, search engines are also able to easily index the data for search and discovery through portals such as Google or Bing.

Publishing data online for the public is the right thing to do for constituents and departments. By providing it through simple and free to use online portals like GeoCommons it doesn’t require a large IT budget or workflow burden, making it an even smarter decision.