Unfortunately when the wagons circle everything starts picking up the label – in this case semantic. This is especially dangerous when you have a word like “semantics” that can be defined, so many different ways. Just look at the definition tree created by Wikipedia:

*Semantics is the study of meaning in communication.
*In computer science semantics reflects the meaning of programs or functions.
*The Semantic Web refers to the extension of the World Wide Web through the embedding of additional semantic metadata

More often I see folks labeling things semantic that are really syntax. “Syntax” being the rules to construct and define something like a sentence or line of code and “semantics” the meaning of those rules or definitions. Syntax is fairly easy and semantics are fairly hard, as most folks in artificial intelligence would argue. Even going so far as saying all programming languages other than LISP are syntax and not semantic.

This is a bit more clear with an example. Lets take the Textwise announcement – a technology that will parse plain text on a website or elsewhere and categorizes it to predefined topics. One example in the Techcrunch comments was the following:

input text:
Call us crazy, but we think there are some brilliant minds out there that can find some really amazing uses for this incredibly powerful and scalable technology. Think you’re up to the Challenge? We think you are!

categories (ranked from 0 (worst) to 100 (best)):
Shopping/Health/Alternative/Hypnotherapy/Audio_and_Video 43 Business/Telecommunications/Services/Wireless/Software 33 Arts/Music/Bands_and_Artists/311/Tablature 28
Computers/Internet/Consultants/Research 26 Shopping/Health/Alternative/Meditation/Audio_and_Video 25

The output is really not telling me anything about the meaning of the text just setting up rules to provide categorization. So I would definitely put this in the syntax and not semantic category. I would also say what Yahoo! is doing is really more syntax than semantics although there is the possibility of building truly semantic technologies on top of what they are enabling. They’ve created a set of rules based on rich standards to allow applications to be built. Remains to be seen what will come of it, but in rush of market buzz I think it is easy to miss that building truly semantic technologies is quite hard. Some folks in AI (the Chinese room) would argue machines are not even capable of semantic meaning or understanding.

From this perspective I think we’ll see a lot of people building applications based on syntax that reorganize and categorize content by giving the “page web” a bit of structure. Oddly its like we’ve gone full circle back to DMOZ. While these technologies may be clever and useful I do not think they will fundamentally change the Web. In the other category I think we’ll see a few companies pushing towards something more sophisticated (call it a semantic, implicit, computational web) where new data and services are mixed with existing web content to provide answers to users questions.

Once I had some time to really read through the post and the various comments for and against GeoCommons it seemed the core of the grumpiness was over metadata, more accurately the lack of it. Metadata is an important part of the GIS community with the goal of making data interoperable for projects like the EIS’s James Fee mentions. Interestingly it can be argued that metadata standards arose out of criticism of GIS technology by Geographers back in the day, “GISs are only tools, perhaps bad tools, because they are ‘simply big black boxes, slick, simple and utterly incapable of dealing with the sort of matters that are truly important’ (Curry 1994: 441-442).” Did metadata assuage Curry’s criticism, not really but it does sound awfully familiar to the criticism of web 2.0 mapping technologies we are hearing now, ironically from GIS people. So what exactly is GIS metadata ? I’d paste an example into this blog post, but it is 11 pages long, so I’ll just post up the link to the metadata for airport locations in the USA from the USGS National Atlas. From a technical stand point I understand why each bit of the information in the 11 pages is needed, or more accurately would be dictated by a standards body. While this is all well and good for some users, let’s face it how many people are going to read it or worse fill it out before they share content. So, the question in our mind is what is the minimal amount of metadata needed to make data useful and also opens up the process to as many people as possible? Our current answer for the same data set can be seen here.

We believe in the “less is more” mantra, and thus far that has held to be true at least in data standards. The simple and short specification of KML has resulted in over 8 million pieces of data on the web, the thorough but very very long specification for GML has resulted in roughly 800 pieces of data on the web (these are second hand statistics from a conversation, but you get the idea even if the numbers are not likely exact). We originally wanted to use GML as the standard on our system, but there just was not a market for it on the open web. So, now we are working on letting KML support featured attributes through the schema tag.

In the Web 2.0 world metadata has largely taken the form of “tags” – simple keywords describing the referenced object. There is an increasing amount of data that is being generated through web 2.0 technologies, and with free geocoding, scraping technologies, and geo conversion applications like G-Swivel, to name a few, there is only going to be more of this data. There is a solid argument this geographic data needs more than just tags, but do we need 11 pages of metadata? Let’s look at the two extremes 1) KML search in Google 2) GIS data search in the Geospatial One Stop. To keep it simple and some what fair we’ll stick with “airport” example. So if we do a search in both for airport we get:

Google – KML search “airport”

GOS – search “airport”

Google pros – it is incredibly easy to get the data and immediately view it in Google Maps or Google Earth.

Google Cons – the data is not particularly useful (three pushpins of Chicago Midway Airport) and you really do not know what the data is until you download it and view it in Google Earth.

GOS pros – produces useful search results (airports from the National Atlas, airports in Illinois etc.) and good descriptions, varying in quality, of what the data is before you download it. Although the value mostly comes from the abstract and reading the attribute definitions in the metadata is rather torturous.

GOS cons –The majority of query results only provide a summary and a list of metadata. The data itself is most often missing and the few that have data sets that can be added to their web app usually crash it or time out.

So, the question I pose to both the GIS community and the neogeographers is how much metadata is enough? Ideally geospatial data should not be any harder to find and consume than a local search. I would argue that the current KML search is not enough, and what is required in traditional GIS is far too much for the vast majority of people. Our answer to date has been to leave it as open as possible within GeoCommons. If you want to paste in all your traditional metadata into the description field you are free to do so. This could be the wrong approach. Maybe it is better to make the application more welcoming to the GIS community by providing an option to link directly to the source metadata of what you are uploading? In the bigger picture outside of GeoCommons we’ve been advocating the ability to add structured features to KML, so that more details and attributes about a data set can be specified. Between working on the next release to make the upload process easier and the KML work we would love to get feedback on how much metadata is enough. If the proble is – not enough metadata – I think the community can solve it. If the problem is – the rest of the world should not be mucking around in geospatial data because you need to be a trained GIS professional to do so – we have a different conversation on our hands. I really believe there is a great opportunity to create a bridge between the GIS and web 2.0 worlds to everyone’s benefit, and hopefully we can start a dialog that moves towards that. If not bring on the flames.

Data is a problem that is not exclusive to us, but something that is critical to growth of web mapping in total. O’Reily, through Where 2.0, has given this topic considerable attention and we are hoping we can add our small bit to the solution. My frustration with geospatial data goes back almost a decade when we first started looking for open source data. Back then it was the beginning of the .com boom and we were looking for spatial data on fiber optic infrastructure to see if we could sort out if the Internet was going to be the end of geography. At the time the pundits were saying that the Internet would mean location was no longer important because you would be able to connect to the Internet and work from anywhere. So, we were mapping where the nuts and bolts of the Internet to see what locations could support the new business of the Internet and if the geography was actually changing.

This seemed like a cool and straight forward idea at the time, but as we started going through all the different forms people were collecting maps of their fiber routes it started getting uglier and uglier. The frustrating array of different formats, projections, and data attributes was just the tip of the iceberg. We eventually sorted it all out and got a reasonable database going, but it was a long slog. With the advent of Google and other search engines it became easier to find different locations data was stored, but it was always a pain in the arse to turn it into something we could use for actual research and analysis.

When FortiusOne started we were doing homeland security work and got tasked with emergency response work after the London bombings and Hurricane Katrina. Both times we received a call on Friday night and needed to deliver analysis for Monday morning briefings. We burned the midnight oil and got the job done, but realized there was something inherently wrong with the effort. The analysis needed to be delivered in seconds not days if it was going to truly make a difference in a crisis and anyone should be able to perform it – no PhD. required. To do that we were determined we needed to be able to deliver our analysis through a web browser and not using the desktop applications that took hours to run analyses. This was the advent of GeoIQ.

GeoIQ was only half a solution though because we spent half of our 48 hour response trying to find the right data, download it, clean it, attribute it, verify it etc etc. We started off building an internal system to help ourselves out. The system allowed all our data to be easily found with a simple tagging and rating schema and we created wiki’s for all our datasets so that as we used them we could capture what we learned each time, then we’d collectively be smarter as we moved along. It was from this internal system that the idea for the data repository sprang. I loved looking at all the different mapping mashups that came along, but inevitably I wanted to be able to mashup the different mashups. Pull the data from all sorts of different mashups onto to one map to run analysis and analyze all new location problems. Then the – ah hah – what if we hooked up a social network to our data repository, so that anyone could share data – mashup their data with our data or anyone else’s data. So there could be a place where you could connect with the people behind that data and learn from their expertise – to pull the intelligence from the edges of networks to open new possibilities of what problems could be analyzed and how.

Some of my best sources of data were my friends that would send me different datasets they had found or had access to. If we could multiply that effect and add the benefits of the data search and wiki system we’d developed it could be a powerful combination. While we are going to be contributing all our data to the repository the real value is going to be from other people joining and contributing their expertise and data. The benefit is being able to expose your data and expertise to a whole new audience. The non-GIS public – the 100 million people that downloaded Google Earth and are enthralled by the thousands of map mashups on GoogleMapsMania, ViaVirtualEarth, and the ProgrammableWeb to name a few. The goal is to not only expose to our/your data to more people, but also have ability to combine that with other contributor’s data to examine all sorts of new possibilities. What we get is an audience for our geospatial software services to analyze the data, quickly search through it, and intuitively decipher it. This does not mean that only our web services can play. The value only increase with other technologies thrown at the data and that is the beauty of mashups in general. The key to all of this is participation. So far the email responses have been great and if you’d like participate please get in touch with us at info@geoiq.com.