Vector One’s Jeff Thurston discusses the representation part to his GIS series. He says that representation part is an integral feature and one of the primary functional capabilities of GIS. Thurston discusses the many ways GIS is represented, including tabulated spreadsheets, numerically instead of graphically, through maps, charts, etc. He also talks about visualization tools that ‘take GIS data output and use it to develop other forms of visualization.’

The Washington Post released a flashed based Google Map mashup called TimeSpace: World. The map is a compilation of world news from the newspaper, its online site – washingtonpost.com, PostGlobal, Foreign Policy magazine and other partner sites including The Associated Press. The coverage is represented by clusters around hot-spots on the map. Each cluster lets you view articles, blog posts, photos, videos and even reporter twitter feeds.

Microsoft Research India created a system called the Robust Location Search, which enables location addresses in structured formats from any country. Microsoft plans to add it into Window Live Local.

The unemployment is getting worse. “Initial jobless claims surged by 58,000 to 573,000 in the week ending Dec. 6, the highest level since 1982.” MSNBC created an interactive map that displays the unemployment rate by month for each state starting in September 2007.

Blogger added geotagging! Now the Blogger community can geotag blog entries and not just photo. Now feed readers, map applications and search engines can associate posts with their locations.

Before the GeoWeb came into mainstream popularity both Microsoft and Yahoo! had mapping applications. Microsoft offered their browser based Terraserver which hooked up USGS imagery for the map tiles. Microsoft launched Terraserver in June of 1998 – practically prehistoric.

Microsoft had also been active in the mapping space with products like MapPoint (both desktop application and web services). Yahoo! also was an early adopter of mapping applications in conjunction with their local search destination (although I completely failed at finding a date for when they first added maps). Despite the early adoption of web based mapping applications by Yahoo! and Microsoft it was arguably the launch of Google Maps in 2005 that jump started both the GeoWeb and the mash up craze.

Shortly after Google Maps launched, Paul Radamacher hacked the application to allow it to display Craig’s List rental listings on the Google slippy map. Shortly there after Adrien Holovaty followed suit mashing up Chicago crime statistics with Google Maps. Google quickly released an API to allow developers to do the same thing seamlessly and we were off to the races. Microsoft quickly created Virtual Earth and Yahoo! pushed out Yahoo! Maps. Microsoft created compelling innovations with birds eye imagery and Yahoo! launched several popular GeoWeb services like free geocoding and Flash based mapping APIs.

Microsoft Collections

Through all these innovations there was a constant one way flow of content creation – developers could create unique maps and users could view them. Microsoft changed this when they launched Collections May 23, 2006:

Collections. Social networking functionality allows customers to create lists of favorite landmarks and locations, attach personal photos and save them to a Scratchpad. Collections can be saved, recalled later, “permalinked,” and shared with friends and community in e-mail or through their MSN® Spaces blog.

While not well publicized the “Collections” concept fundamentally changed the work flow for creating maps. No longer did you need to be a developer or GIS pro to create a basic map and share it with other people. The Virtual Earth folks even gave users a decent amount of cartographic power and options:

Customized pushpins. A pushpin is essentially a marker indicating points of interest on a map view. A customized pushpin can easily be added with a simple right click, anywhere on a map, which will display a small red dot and a pop-up menu. A pushpin title or note of up to 200 characters can be added that will appear with the pushpin whenever a mouse hovers over it. Pushpins can easily be edited or deleted. When a pushpin is removed, whether customized or standard, the remaining pushpins will be automatically renumbered.

2-D drawings in Collections. Users can add lines and drawings in a variety of colors, shapes and styles to personalize their Collection. They also can draw lines and shade areas that they want to mark on the map, such as for marking a running or bike trail, or neighborhood boundaries).

MyMaps

Despite the potential of the innovation the new functionality did not get much coverage in the press or massive levels of adoption. The TechCrunch article on it was lumped in with other new features from Yahoo! Maps.

Just short of a year later Google launched Google MyMaps on April 4th 2007 to big headlines across the blogs, including MyMaps being the death knell of popular map mashups like Platial, Frappr and Tagzania.

Fundamentally the functionality and features of MyMaps was not remarkably different than Collections, but the buzz around it was at least ten fold. So why was the attention so skewed towards Google for fundamentally the same innovation Microsoft had launched a year earlier? A few guesses:

MapMixer

Yahoo! was not too far behind launching their own map creation application, Yahoo! Mapmixer on September 13th 2007. Mapmixer took a different angle on map creation by allowing users to put static maps on top of the Yahoo! Maps applications. For instance after the Buscan oil spill in the San Francisco Bay last year I made a lot of calls trying to get the raw data on the location of the spills, for GeoCommons, but had no luck.

I did find a PDF with a map of the oil spills so I saved it as a PNG then uploaded it to Yahoo Mapmixer and they took me through three easy steps to georeference the map on Yahoo! Maps. The user experience I thought was the best of the three and there were lots of great social features for me to give a short description of the map and for other users to comment on the map. Although much like Microsoft the application did not generate lots of buzz as with Google MyMaps, and the gallery only features 38 user submitted maps today. Interestingly, in concept, it is quite similar to Microsoft’s MapCruncher, although it is a download and supports a wider variety of raster based formats that must already be georeferenced.

Since the launch of map creation applications by the three big players there have been two noticeable waves of enhancement 1) support for external data and 2) collaboration features. Microsoft put themselves out as being the first to support loading KML, “The October 07 release of Live Maps was the first to support KML viewing and import to Collections”. November 27th 2007 Google added KML, KMZ and GeoRSS support to MyMaps. Google followed this up with social features, like commenting, rating and open collaboration invitations for MyMaps.

Performance Trials

That covers features and functionality from a historic evolution stand point, but how do they perform? We did a very informal, one user, stress test. Create push pins as quickly as possible and see when the map application maxes out or gets sluggish. For Yahoo! Mapmixer this was pretty easy. You can overlay one picture or map onto the application, so you max out at one.

In the process of loading and georeferencing the image you get speedy performance and predictable response times. For MyMaps and and Collections we had a bit more to stress. We’ll start with Collections where we created 200 push pins with good response time then got the following message “You cannot add more than 200 items to a collection. To add more items, create another collection.”

When we went with the same test on MyMaps,we did high rate push pin creation and after about 30 the system got a bit sluggish, and sometimes it would create a listing for a pushpin on left hand pane but not create the push pin on the map. The caveat here is we were doing this high speed, and when we slowed down to a more deliberate pace the system handled it fine.

MyMaps also maxes out at 200 push pins on the map, but instead of providing a warning it generates a pagination for a continuing set of push pins. So when you click on the first page you get a map with the first 200 push pins and when you click on the second page you get the next 200 push pins on a new map in the same browser and tab. Oddly it stops at 820 push pins and starts back over at the number one but you can keeping adding push pins to the map.

What’s Next?

That pretty much wraps it up for a comparison of the big three, how they evolved in a competitive environment, and a very ad hoc test of their limits.

I believe the most interesting part will be where they evolve to next. What is the next set of functionality that will distinguish one from the other? Can Microsoft or Yahoo! introduce the next killer functionality that will catch up to 7 million maps that have been created with MyMaps?

We have teamed with Lockheed Martin to leverage our intelligent mapping services to address these problems in government markets. Lockheed has a long history in the geospatial space and has been very progressive in embracing advanced Web 2.0 technologies such as Intelligent Mapping and Wikis.

I thought it might be helpful to provide an example of the kinds of problems we are addressing with Lockheed. Let’s take a fictitious scenario of a military operation dealing with terrorist attacks in Iraq. Suppose I’m Sergeant Gorman and I’ve uploaded data on a spree of attacks that my patrol collected over the past week.

A GIS analyst at headquarters, in reviewing my data along with historical data from the last three years, notices a pattern of increasing Shia activity around Samarra and sends an alert to field units. The alert prompts me to scan for data on attacks tagged Shia and Samarra, where I find a photo from a previous attack that shows one of the locals we had suspected of being a Shia ring leader.

I post a geo-blog noting that this individual has been suspected of coordinating attacks in my sector. A flurry of responses from other patrol leaders indicates that the same individual has been seen in proximity of other attacks. A GIS analyst at headquarters validates the findings and generates a command report, which results in the order to apprehend the suspect. On our next patrol into Samarra, we locate him and discover a complex cell of terrorist Shia activity in the area.

While the account above is completely fictional, hopefully it conveys the power of democratizing geospatial information throughout an entire organization. The same principles apply to a variety of other environments, such as disaster response, homeland security, and intelligence, where enabling the entire organization to explore, create, and share geospatial information can enhance mission effectiveness. We are excited about the partnership with Lockheed Martin to bring these capabilities to market.

Once I had some time to really read through the post and the various comments for and against GeoCommons it seemed the core of the grumpiness was over metadata, more accurately the lack of it. Metadata is an important part of the GIS community with the goal of making data interoperable for projects like the EIS’s James Fee mentions. Interestingly it can be argued that metadata standards arose out of criticism of GIS technology by Geographers back in the day, “GISs are only tools, perhaps bad tools, because they are ‘simply big black boxes, slick, simple and utterly incapable of dealing with the sort of matters that are truly important’ (Curry 1994: 441-442).” Did metadata assuage Curry’s criticism, not really but it does sound awfully familiar to the criticism of web 2.0 mapping technologies we are hearing now, ironically from GIS people. So what exactly is GIS metadata ? I’d paste an example into this blog post, but it is 11 pages long, so I’ll just post up the link to the metadata for airport locations in the USA from the USGS National Atlas. From a technical stand point I understand why each bit of the information in the 11 pages is needed, or more accurately would be dictated by a standards body. While this is all well and good for some users, let’s face it how many people are going to read it or worse fill it out before they share content. So, the question in our mind is what is the minimal amount of metadata needed to make data useful and also opens up the process to as many people as possible? Our current answer for the same data set can be seen here.

We believe in the “less is more” mantra, and thus far that has held to be true at least in data standards. The simple and short specification of KML has resulted in over 8 million pieces of data on the web, the thorough but very very long specification for GML has resulted in roughly 800 pieces of data on the web (these are second hand statistics from a conversation, but you get the idea even if the numbers are not likely exact). We originally wanted to use GML as the standard on our system, but there just was not a market for it on the open web. So, now we are working on letting KML support featured attributes through the schema tag.

In the Web 2.0 world metadata has largely taken the form of “tags” – simple keywords describing the referenced object. There is an increasing amount of data that is being generated through web 2.0 technologies, and with free geocoding, scraping technologies, and geo conversion applications like G-Swivel, to name a few, there is only going to be more of this data. There is a solid argument this geographic data needs more than just tags, but do we need 11 pages of metadata? Let’s look at the two extremes 1) KML search in Google 2) GIS data search in the Geospatial One Stop. To keep it simple and some what fair we’ll stick with “airport” example. So if we do a search in both for airport we get:

Google – KML search “airport”

GOS – search “airport”

Google pros – it is incredibly easy to get the data and immediately view it in Google Maps or Google Earth.

Google Cons – the data is not particularly useful (three pushpins of Chicago Midway Airport) and you really do not know what the data is until you download it and view it in Google Earth.

GOS pros – produces useful search results (airports from the National Atlas, airports in Illinois etc.) and good descriptions, varying in quality, of what the data is before you download it. Although the value mostly comes from the abstract and reading the attribute definitions in the metadata is rather torturous.

GOS cons –The majority of query results only provide a summary and a list of metadata. The data itself is most often missing and the few that have data sets that can be added to their web app usually crash it or time out.

So, the question I pose to both the GIS community and the neogeographers is how much metadata is enough? Ideally geospatial data should not be any harder to find and consume than a local search. I would argue that the current KML search is not enough, and what is required in traditional GIS is far too much for the vast majority of people. Our answer to date has been to leave it as open as possible within GeoCommons. If you want to paste in all your traditional metadata into the description field you are free to do so. This could be the wrong approach. Maybe it is better to make the application more welcoming to the GIS community by providing an option to link directly to the source metadata of what you are uploading? In the bigger picture outside of GeoCommons we’ve been advocating the ability to add structured features to KML, so that more details and attributes about a data set can be specified. Between working on the next release to make the upload process easier and the KML work we would love to get feedback on how much metadata is enough. If the proble is – not enough metadata – I think the community can solve it. If the problem is – the rest of the world should not be mucking around in geospatial data because you need to be a trained GIS professional to do so – we have a different conversation on our hands. I really believe there is a great opportunity to create a bridge between the GIS and web 2.0 worlds to everyone’s benefit, and hopefully we can start a dialog that moves towards that. If not bring on the flames.

Data is a problem that is not exclusive to us, but something that is critical to growth of web mapping in total. O’Reily, through Where 2.0, has given this topic considerable attention and we are hoping we can add our small bit to the solution. My frustration with geospatial data goes back almost a decade when we first started looking for open source data. Back then it was the beginning of the .com boom and we were looking for spatial data on fiber optic infrastructure to see if we could sort out if the Internet was going to be the end of geography. At the time the pundits were saying that the Internet would mean location was no longer important because you would be able to connect to the Internet and work from anywhere. So, we were mapping where the nuts and bolts of the Internet to see what locations could support the new business of the Internet and if the geography was actually changing.

This seemed like a cool and straight forward idea at the time, but as we started going through all the different forms people were collecting maps of their fiber routes it started getting uglier and uglier. The frustrating array of different formats, projections, and data attributes was just the tip of the iceberg. We eventually sorted it all out and got a reasonable database going, but it was a long slog. With the advent of Google and other search engines it became easier to find different locations data was stored, but it was always a pain in the arse to turn it into something we could use for actual research and analysis.

When FortiusOne started we were doing homeland security work and got tasked with emergency response work after the London bombings and Hurricane Katrina. Both times we received a call on Friday night and needed to deliver analysis for Monday morning briefings. We burned the midnight oil and got the job done, but realized there was something inherently wrong with the effort. The analysis needed to be delivered in seconds not days if it was going to truly make a difference in a crisis and anyone should be able to perform it – no PhD. required. To do that we were determined we needed to be able to deliver our analysis through a web browser and not using the desktop applications that took hours to run analyses. This was the advent of GeoIQ.

GeoIQ was only half a solution though because we spent half of our 48 hour response trying to find the right data, download it, clean it, attribute it, verify it etc etc. We started off building an internal system to help ourselves out. The system allowed all our data to be easily found with a simple tagging and rating schema and we created wiki’s for all our datasets so that as we used them we could capture what we learned each time, then we’d collectively be smarter as we moved along. It was from this internal system that the idea for the data repository sprang. I loved looking at all the different mapping mashups that came along, but inevitably I wanted to be able to mashup the different mashups. Pull the data from all sorts of different mashups onto to one map to run analysis and analyze all new location problems. Then the – ah hah – what if we hooked up a social network to our data repository, so that anyone could share data – mashup their data with our data or anyone else’s data. So there could be a place where you could connect with the people behind that data and learn from their expertise – to pull the intelligence from the edges of networks to open new possibilities of what problems could be analyzed and how.

Some of my best sources of data were my friends that would send me different datasets they had found or had access to. If we could multiply that effect and add the benefits of the data search and wiki system we’d developed it could be a powerful combination. While we are going to be contributing all our data to the repository the real value is going to be from other people joining and contributing their expertise and data. The benefit is being able to expose your data and expertise to a whole new audience. The non-GIS public – the 100 million people that downloaded Google Earth and are enthralled by the thousands of map mashups on GoogleMapsMania, ViaVirtualEarth, and the ProgrammableWeb to name a few. The goal is to not only expose to our/your data to more people, but also have ability to combine that with other contributor’s data to examine all sorts of new possibilities. What we get is an audience for our geospatial software services to analyze the data, quickly search through it, and intuitively decipher it. This does not mean that only our web services can play. The value only increase with other technologies thrown at the data and that is the beauty of mashups in general. The key to all of this is participation. So far the email responses have been great and if you’d like participate please get in touch with us at info@geoiq.com.