GeoCommons 2.0: It’s Not Just a Pretty Face

When we launched GeoCommons three years ago we had high hopes that it would turn into the vibrant community that it is today.  However, I must admit that from a technological standpoint we did not fully predict what the underlying GeoIQ platform would become – a complex, scalable set of distributed services that enable users to ingest and geo-enable data from an ever-increasing number of sources, visualize data in a various forms (maps, charts, images), and analyze data to uncover interesting information.  The GeoCommons of today is very different from what was launched in 2008.

Leading up to the launch of GeoCommons 2.0 our signup, data upload, and map-making rates soared to all-time highs, and as a consequence it became alarmingly clear that the infrastructure that has always been the backbone of GeoCommons would not scale well enough to support GeoCommons 2.0 – nor our ever-evolving plans for it.  It needed a serious overhaul from the bottom up, not due to poor planning or architecture three years ago – in fact the old GeoCommons infrastructure ran largely unmodified from day one without a single major hiccup – but simply because GeoCommons 1.0 was a very different beast from the GeoCommons we launched last week.

So while the GeoIQ engineering team began to build out GeoCommons 2.0′s features we undertook a parallel effort to migrate our infrastructure from a slowly melting half-rack in Washington, D.C. to the magical wonderland known as the Cloud – Amazon AWS, to be specific.

While the GeoIQ platform has been deployed in numerous environments and configurations and is not tied to any specific cloud service, Amazon was an obvious home for GeoCommons for a number of reasons – beyond the fact that it enables us to scale performance and costs as needed.  Firstly, like most startups we’re a very small team and we simply don’t have the resources to stand up, administer, and scale our own physical hardware, nor to manage most of the services that are essential in any scalable infrastructure.   Talent abounds at GeoIQ, but it usually abounds in 10 places at once – we can’t afford to waste our time cutting Cat 5 to length, swapping out bad drives, kicking faulty cooling systems, or calling REDACTED tech support because our overpriced Internet pipe is on the fritz.

Secondly, we’ve been using AWS for GeoIQ Cloud deployments for years and are comfortable with it from an operational standpoint.  Despite the recent outage in the US-East region we trust its reliability, and, perhaps more importantly, we’ve learned through years of trial and error how to effectively utilize AWS to ensure GeoCommons‘ reliability and get the best possible performance out of the GeoIQ stack.  In fact, the US-East outage could not have come at a better time for us.  We were in the midst of testing the new infrastructure and the outage afforded us an opportunity to test our failover plan during a real, unplanned, and uncontrolled situation.  I’m proud to say that (almost) everything went as planned, although some of us did lose more hairs than normal on that day.

Finally, we are consistently impressed by the breadth of Amazon’s services and the rate at which new services are being announced.  While no cloud is truly fire-and-forget, Amazon has come the closest by far.  I trust that they’ll continue to announce stellar services that allow us to further sunset our custom deployment solutions, which can only make our lives easier and allow us to focus on the things we know best – GeoCommons & GeoIQ.

I’ll go ahead and stop before this turns into a love poem to Amazon (too late?), but over the next week or so I’ll be writing posts on our developer blog detailing the architectural and technological changes that necessitated our migration to the cloud.  I will also describe how we’ve used Amazon’s services and outline some key lessons we’ve learned along the way.  Hopefully you’ll find them far more informative and beneficial to your work than this post turned out to be!

Stay tuned!