Don’t Hide Your Data in Description Fields – My Plea to KML and GeoRSS Creators

By Sean Gorman On August 29, 2011 · 5 Comments

I was pulling together some data for a customer following Hurricane Irene today and kept running into the same problem. Folks creating KML and GeoRSS feeds with awesome statistical data, but leaving it all mixed up with text in a description field. The folks at Google put up a really nice Hurricane Irene response map and had links to many of the layers for download. Downloadable data is awesome, but less awesome when the best data is trapped. Take for instance the Pepco power outage layer from the map; when you load up the data on how many people are affected by the power outage it is trapped in the description field with a bunch of other text. That means you can’t use that data for any kind of analysis unless you want to go parse it out as a new field. This is not Google’s fault. Pepco generated their GeoRSS with the numeric data in the description field. USGS does the same thing with their stream gauge data as you can see here. The majority of KML/GeoRSS feeds we see in the wild fall into this bad habit.

Bill Dollins tweeted out an old school power outage map for SMECO in Southern Maryland. While it lacks a sweet zoomy map it did have a table with zip codes and number of people effected. So, I could copy and paste it into a CSV and load it into GeoCommons no problem. The result was a nice thematic map of people effected by outages by zipcode. Once the numeric data is broken free you can have all sorts of statistical fun with it. So, as we get past red dot fever and just making pretty pictures online, let’s start making data portable for analysis. Get your numeric values in their own field not all junked up with your text. My plea to the KML/GeoRSS data creators of the world.

5 Responses to Don’t Hide Your Data in Description Fields – My Plea to KML and GeoRSS Creators

Timothy says:

August 29, 2011 at 4:28 pm

I’ve run into this quite a bit as well–but I’ve come to the conclusion that this is sort of baked into the cake. KML is not a data format so much as it is a data presentation format. As long as people are using KML as a basic data standard, I think this is going to be a problem. (GeoRSS I can’t speak to as much.)

Reply
Sean Gillies says:

August 29, 2011 at 5:40 pm

Right on. The need to scrape data wasn’t what people meant when they said KML was “the HTML of the GeoWeb.”

Reply
andrew says:

August 29, 2011 at 7:15 pm

Timothy – actually KML has this capability as ExtendedData. That allows you to put in structured data in KML. All KML out of GeoCommons uses ExtendedData so you can ensure you still have the underlying data attributes in a structured way.

KML is often referred to as a “presentation format” but it’s more structured than an HTML page and when used properly is an excellent data exchange format.

GeoRSS by comparison is very simple and while you can always extend it, these customizations make it more difficult to generally share around without custom code.

Reply
James Campbell says:

August 29, 2011 at 7:29 pm

I’ve used six different technology options to parse and convert KML and none of them can support every variety of KML that I come across.

I hate the format for reasons including the one mentioned in this post. It seems that there
is plenty of room in the ‘standard’ for different interpretations about what is valid and even
more permutations from users interpreting how to structure their data.

At least the shapefile has been around for 10 years and every major technology can work
with it.

Reply
Diş beyazlatma says:

April 14, 2012 at 8:06 am

Günümüzde en çok yapılan estetik diş işlemlerinin başında gelen diş beyazlatma yöntemleri bugün birçok kişi tarafından tercih edilmekte.

Reply