Federating with GeoCommons: Trying to Keep it Simple and Standards Based

At Where 2.0 last year we gave a talk on data federation and at the time we were working with Andrew Turner on implementing the concept with GeoCommons and Mapufacture. We got a little carried and ended up just merging Mapufacture and GeoCommons, but we continued to develop the federation ideas.

Recently, the team implemented federated GeoCommons search for our appliances, so that customers can search across multiple appliances on the network. Our hope was to develop a simple approach based on commonly used standards that could be replicated with data stores other than GeoCommons. So, I thought this would be a good opportunity to share the basic approach and get some feedback on what was developed. For the GeoCommons implementation it was a pretty straight forward proposition. There are multiple appliances in an enterprise and users would like to be able to search across them for useful data.

The team’s approach for solving this was simple and I think fairly elegant. The various appliances on the network advertise their index of data via Atom. The user performs a search with OpenSearch and we use a Solr-Lucene combination to manage it. The various appliances return a GeoRSS feed of the results with links to the data. The user then requests the data and it is transfered to their local GeoCommons instance. From there they can download the data in a format they like (KML, .shp. csv. etc.) or create a map with it.

While we built this approach around GeoCommons, a goal was that the same approach could be used to federate any collection of data stores. So with that in mind I took a stab at diagramming what such an architecture might look like:

In the GeoCommons architecture the data in steps 4 and 5 go back into the appliance (federation server) for styling and/or analysis, but in a more generic deployment downloading from the source seemed to make more sense. Options like GeoServer and ArcGIS server (I think) could also play the styling/analysis role in the architecture if you wanted to move beyond raw data/service download. We’ve found the approach has worked very well for us sharing data across multiple repositories and thought there might be value in sharing it.

I’m sure that some out there will say that this is a spatial data infrastructure (SDI), but I would disagree. I consider this simply an approach for sharing data across disparate repositories. There is no national body behind it and there are no request for funds to build it. In fact there is no cost involved only a straight forward approach to exposing existing data to make it consumable by folks following the same standards and approach. Similar to the development of GeoRSS and other bottoms up approaches to solving problems in the community with deployed technology.

The approach itself builds on but simplifies best practices from GeoNetwork and open archive initiatives, like OAIPMH. Andrew discussed the architecture at the OGC Geospatial Search Summit last fall. The concept received a positive reception then, and would be great to see how folks feel about it after some more development and deployment.