There seem to be two fundamental problems, of many, worsening our current economic quagmire. 1) The housing bubble pushed home prices to levels most working Americans could not afford and to keep the bubble going the financial community became very creative with mortgages and how the risk associated with them was calculated. The end result was lots of people in houses they could not really afford and very little transparency in the risk this created in the financial markets. There is a lot more to the story but for the sake of brevity we’ll leave it at that. 2) Credit liquidity in the current market has almost ossified causing our collective economic gears to come to a rattling halt. Wall Street freaks…the media freaks…the consumer freaks (no spending)…sales of goods plummet…Wall Street freaks again…media fuels more freaking…rinse and repeat.

To break the cycle it would seem logical that liquidity needs to be injected into the market. A lot of pundits have looked at this being solved by the government buying up the bad assets, giving capital to the banks in return for equity stakes, and several other derivative plans. While all these ideas have their merits and risks the idea I exposed on the plane was slightly different. Back to the core issues – I saw the biggest failing being lack of market transparency and a fundamental mismatch between supply and demand in the housing market. So how could we restore transparency to the market while getting people in homes they can actually afford thus freeing capital for consumer spending and financial investment.

My answer was a foreclosure clearing house. This may be Polly Anna and not feasible, but it made for a fun intellectual exercise. There has been lots of talk around providing bail outs to people whose homes are foreclosing, but even this will be short term and will not solve the fundamental problem that they are in a home they cannot afford. The only real solution is to put these individuals and families into homes they can afford. The easy credit and risk shell game that banks ran has created a basic mismatch of people buying supply with demand they did not really have.

The clearing house is a simple idea of providing a transparent market place where people can trade down to houses they can afford and have new loans guaranteed to do so. The loans could be guaranteed by the government but competed for by the banks. Banks that already have the mortgages on existing properties could have the choice of refinancing the house so the owner could afford the payments (that would be their own risk calculation) or entering the home into the clearing house. Also the home owner could have the choice to enter their home into the clearing house if they would like to trade down voluntarily.

The clearing house itself could run like many of the existing home real estate market places matching buyers and sellers (Zillow, Trullia. RedFin etc.). In fact the government could probably contract with one of the sites to run the technology side of the clearing house at a reasonable cost. Once a person’s home was identified for purchase they would then be free to look for a new home in the clearinghouse they could afford. The government backing would allow loans to be made so the individual, now free of the foreclosed home, could buy a new home they could afford. Banks would still compete to provide the best rate and terms to new owner, but the risk would all be transparent to the government since they would be providing financial backing and to the owners so they were not mislead into buying more house than they could afford (again).

In theory this should introduce liquidity back into the market and with a little time put liquidity back into the consumer market since the majority of a person’s paycheck would no longer be going to a mortgage. The market would be transparent again but not run or partially owned by the government. I would argue that it was not capitalism or the market economy that broke during this financial crisis, but a loss of transparency and a resulting hiding of risk. In fixing the crisis the government’s role should be ensuring transparency in the market place so that it can function effectively. My idea is most likely off the deep end, but I do hope government action is centered around restoring transparency and restoring liquidity to the market. If you were Sec. Paulson for a day what would you do with $700 billion? There are no shortage of smart people around the globe. Can we crowdsource an answer?

The following maps have been created in Maker to reveal pertinent election coverage and data that my fellow data colleagues and I thought would be helpful going into the big election tonight.

This map displays early voting data for selected states from yesterday:

Pay close attention to Virginia—a highly contested swing state —because if Barack Obama can win in the former Old Confederacy capital of Richmond then the odds of him winning the election will be in his favor.

The following is a map of active registered voters in Virginia:

Here is the latest polling data that shows Obama’s lead vs. McCain’s:

Here is a link to the data set in Finder: http://finder.geocommons.com/overlays/5911

And a link to the map in Maker: http://maker.geocommons.com/maps/1311?page=

So, if there indeed are Bush donors who now have become Obamacans, the data-team wanted to find out where they are spatially speaking. Below are the maps of our efforts showing locations of possible Obamacans in New York City and Washington D.C. Why use the term possible? Because what is mapped are the results based on spatial join and attribute join, the later being a variation of spatial join. And the accuracy of the results of such joins is subject to the limitations imposed by the accuracy of the original data (donor addresses) as well as limitations of the geocoding operation. More on this towards the end of this post. So what is mapped are donor address matches and not individual donors.

Attribute Join
The attribute join is based on an identifier “XY” constructed from the concatenation of X and Y location coordinates of the Bush-Cheney and Obama donors, where the X and Y location coordinates are obtained by geocoding donor addresses. The attribute join resulted in 250 records across the lower 48 states, mostly concentrated in major cities of North-East and West-Coast. The results are shown below for New York city (lower Manhattan) and Wash D.C., where blue circles represent Obama donors (1,415 in D.C. and 1,825 in New York city); red circles represent Bush-Cheney donors (294 in D.C. and 419 in New York). The purple squares colocated with Bush-Cheney red circles are the XY “attribute matches.” There were 32 such locations in D.C and New York City had 85.

New York City: “XY” attribute join of Bush-Cheney donors with Obama donors

Washington D.C.: “XY” attribute join of Bush-Cheney donors with Obama donors

Spatial Join
Yet another way was to carry out a “spatial” join between location of each Bush-Cheney donor with all of the co-located Obama donors, resulting in more than 9,200 Bush-Cheney records colocated with more than 42,000 Obama records in the lower 48 states. The results are shown below for New York City (lower Manhattan) and Wash D.C., where again blue circles represent Obama donors, red circles represent Bush-Cheney donors, and the purple circles with varying sizes represent count of Obama donors that are colocated with each of the “spatially” joined Bush-Cheney donor. There were more than 1,500 Obama donors colocating with 248 Bush-Cheney donors in D.C. while the comparable figures for NY city are more than 2,030 Obama donors colocating with 303 Bush-Cheney donors.

Bush-Cheney donor locations spatially joined with Obama donors in NY City

Bush-Cheney donor locations spatially joined with Obama donors in Wash D.C.

Donor Data
You may find/download the mapped as well as other supporting datasets from the Finder! by using the key-word “Obamacans“. The supporting datasets also include spatial join of all Bush-Cheney donors for each of the Obama donors.

A strict one-to-one Name/Address match between the Bush-Cheney and Obama donors based on uniquely generated ID to identify the real Obamacans resulted in zero matches. Unique IDs were constructed by concatenating the upper case fields of each donor record: “LAST NAME”, “FIRST NAME”, “STREET ADDRESS”, “CITY”, “STATE” and “ZIPCODE”. See below for the explanation of such disappointing results.

Geocoding, Mapping “join” and the results
Individual Bush and Obama donor lists were compiled from the publicly available donor records from the campaign finance reports filed by all presidential candidates. Federal Election Commission (FEC) rules permit an individual donor to contribute more than once with the condition that the total of all such contributions may not exceed $2300 per election cycle (primaries and general). Since both Bush-Cheney and Obama donor list include such multiple donation records, the resulting geocoded data inherits the many donor-multiple records problem.

The geocoding success rate depends not only on the accuracy of the address information, but if street address is not provided, then that donor’s location will be geocoded to the centroid of the zipcode/city/state areas mentioned in the record. Hence, we decided to limit our analysis to only those geocoded records that had street address and with the geocoding success score of 90 or more out of max possible score of 100. Next, multiple records per individual were eliminated by uploading the data to Access database, creating a unique ID based on donor’s name/address and by running sqls to find match between Bush and Obama donors based on the unique key. As was stated earlier, the output showed zero matches. Some of the possible reasons for zero matches are: donor names are spelled differently at different times, address changes over 4 year time period, some donor addresses have just PO boxes or only zipcode/city/state information; and/or combinations of these, which results in poor geocoding output.

The difference in number of matches with the “XY” attribute join and spatial join can be explained as follows: in case of the former, there may be more than one record for each of Bush-Cheney record, however, attribute join is between a Bush-Cheney record and the 1st of the multiple Obama records, while in the case of later, the multiple join occurs between spatially adjacent records of both Obama and Bush-Cheney. Multiple records with same location address often occur in cities where geocoding cannot distinguish between a multi-storied location from single family homes or where many donors live on the same city block(s).

For the above analysis, nearly 150 K Bush-Cheney individual donor records were extracted from the campaign finance reports filed with FEC between Oct, 03 and Sep, 04. Similarly, more than 560 K Obama individual donor records were extracted from the monthly campaign finance reports filed with FEC between Jan, 08 and Apr, 08. After geocoding both the Bush-Cheney and Obama donor records, only those with a geocoding score of 90 and more out of max of 100 were selected, further all those records that were geocoded to the centroid of zipcode or city/state were deleted. Additionally all those multiple records of many donors were aggregated to just single record, making the final tally for Bush-Cheney and Obama to be 15,742 and 38,042 donor records for the lower 48 states. “Spatial join” and “XY” attribute join between Bush-Cheney and Obama yielded a little over 9,270 and 250 records respectively.

So what is shown in the maps are address matches between Bush-Cheney and Obama donors rather than the donors themselves. As is evident from many of the articles in the MSM, there may indeed exist a few Obamacans but our analysis of FEC campaign finance reports fails to name the names. I would love to hear suggestions on how to improve on these results or better yet, a methodology to spatially identify/locate the Obamacans.

And why is that? Let’s look at the landscape of state level wins/losses. The 3-d map below shows the share of Obama vote in the lower 48 states. The height of raised polygons in the shape of the states with shades of blue/purple and green represent Obama’s share of votes, while the dark red/brown and beige/wheat hues represent Clinton states.

Considering the map above, Obama campaign would do well to realize the challenges that they now face in the race for the White House, both in the states that Obama lost to Clinton (CA, NY, PA, OH, FL, AR, TN, IN etc.), and the states that he won handily (SC, LA, MS, AL, GA, SC, NC, MT, NC, NE, ND, ID, OR, WA, VA, MD, MO, UT etc.). Obama campaign will have to compete with McCain in all those states that are shown in shades of green. It also does not bode well for him that the states that he won in the primary (blue and purple) traditionally vote for a Republican president.

The difficulties he will encounter become visible when one analyzes the landscape of voting patterns in the primaries at the sub-state level such as counties and congressional districts. The map below was constructed from dozens of data-sets available on the Finder!, while the data for the map was compiled from more than 45 on- and off-line sources.

Much of the county level data came from election divisions of each state’s office of SOS (Secretary of State), while the congressional district level data was scraped from from the mother of all election information sites, TheGreenPapers.Com and a few state Democratic parties and news papers. The map shows share of Obama votes by county and by congressional districts, a hybrid born out of the necessity of geocoding the default data available for different jurisdictions in the lower 48 states.

The sub-state level primaries voting patterns with bluish hues are where Obama did well. The areas with the orange and red hues where Clinton did well are the so called “Guns and Religion strongholds“. With Obama’s 50 state strategy, these are the same places where Obama campaign will have to spend enough time and money for the general election campaign. As is apparent from the map, the the rival campaigns will compete with Obama in blue hued areas, at the same time some of the independent 527s could swift-boat Obama by playing up Obama’s pastor problems in the red/orange hued Clinton country. In brief, what Obama campaign has experienced so far is just a primer to the challenges they will have to face during the General Election campaign.

All of the data for these maps, including county and congressional level data is available on Finder!, except for North Dakota and Wyoming, the really red states that appear blue in these maps, mainly because Obama’s win in the democratic primaries and whose respective SOS (Secretary of State) offices have not posted the primary results data yet. For the rest, you may search/download/map the state/county and congressional district level data on Finder! using tags such as ‘elections’, ‘primary’, ‘presidential primary’, ‘democrats’ and ‘politics’.

Romney vis McCain: Michigan primary vote

Note: Brighter hues = Higher vote count, Darker hues = Lower vote count

This is Romney’s first win (not including his win in the Wyoming primary) and probably saved him from dropping out of the primaries after finishing 2nd, both in Iowa and New Hampshire. McCain and Huckabee finished 2nd and 3rd respectively. With Romney’s win in Michigan, the GOP has no clear cut front runner. Romney got nearly 38.9% votes (~337,700), to McCain’s 29.7% (257,400), followed by Huckabee at 16% (~139,600).

DNC (Democratic National Committee) decided to punish Michigan for violating primary rules by moving forward its primary date. They stripped Michigan of all its delegates for the National convention. As a result, both Obama and Edwards withdrew their names, while Clinton’s name remained on the ballot and she won the primary. According to some, Clinton would have won the primary anyway.

Clinton vis Mr. Uncommitted: Michigan primary vote

Note: Brighter hues = Higher vote count, Darker hues = Lower vote count

Interestingly many who wanted to support either Obama or Edwards voted “Uncommitted”. Clinton won more than 55% vote (~327,300) compared to 40% vote for Mr. “Uncommitted” (~236,900). And much of the “Uncommitted” vote came from African Americans and young voters according to the exit polls. This does not bode well for the Clinton camp, as the Democratic primaries move south where African-Americans are a major constituency. More on this in the future blogs.

In the meanwhile you may want to explore all of the maps shown above at Geocommons. Search with keywords “Michigan” or “Primary” to discover dynamic, interactive maps with zoom-in and pan.