Two days, ago, I compared the average rate of error from final predictions to final election results for Pollster.com, fivethirtyeight.com, and my own predictions. Looking at 65 elections on November 4th, 2008, where all three sites made public final predictions / estimates, it turned out that Pollster.com and fivethirtyeight.com were equally accurate, and that I lagged about 8-10% behind.
While I was a bit further behind, I still wanted to see where I was less accurate. It turns out that when blowouts (final margin over 20%) and rarely polled elections (only one poll in the final eight days) are removed, my simple, rudimentary methodology was actually the equal of Pollster and 538. As long as there were at least two polls in the final eight days, simple poll averaging was just as good at predicting election outcomes as any other methodology around. Data in the extended entry.
Now that all of the counting is finally done for the 2008 elections, it is possible to compare how different election forecasters fared. The three I have long been most interested in comparing are:
My method, which takes the simple mean of all non-campaign funded, telephone polls that were conducted entirely within the final eight days of a campaign. My rationale for this method is described here: No Special Sauce Needed For Electoral Projections. This is an intentionally rudimentary "election forecasting for dummies" method that anyone can reproduce.
Pollster.com, which uses all polls ever conducted in a state, and creates a regression line based on those polls. This is the ultimate "don't cherry pick polls and don't argue with polls" method. It was developed by a professional pollster and a political scientist, and is explained here.
Fivethirtyeight.com, whose complicated methodology is essentially the opposite of Pollster.com's: adjust every poll based on demographics, previous house effects, and previous error rate.
How did these three distinct prediction methods fare against each other? Results in the extended entry.
Since final results have not, as of yet, been certified by the fifty Secretaries of State, it is still too early to compare my final poll averages of my Presidential and Senate forecasts to the final results in those states. The initial estimate seems to be that polling averages performed very well in states where there were a lot of polls, but did not do well in the less frequently polled states of Alaska (President, Senate and House showed massive, double-digit error), Iowa (6% error), Nevada (6-7% error) and North Dakota (6% error). Polling averages also seem to have been about 4-5% off in Arizona and New Mexico. Everywhere else, the averages seem to have nailed the final targets by 2.0% or less, even though inaccurate winners were projected in Indiana and North Carolina. Basically, it seems like the more polls in your averages, the more accurate the averages become. Makes sense.
While we wait for final results in the polling average states, I am happy to say that my House Forecast has, once again, done extremely well. If, as appears likely, MD-01 and VA-05 go to Democrats, and with CA-04, CA-44, LA-04, OH-15, and WA-08 still undecided, then Democrats will net 21-26 seats. My final projection was 21-27 seats, so I am feeling like I did pretty darn well. I also did well for each of the category projections (more in the extended entry):
President Electoral Vote: Obama 338--200 McCain
National Popular Vote: Obama 53.1%--45.4% McCain
You can see my final percentage projections here. I decided to go with Obama in North Carolina even though the state was exactly tied. The reason is that most of Obama's vote is already in, while McCain still has to get his voters to the polls. That's enough of a tie-breaker for me. If a final North Carolina poll comes out showing McCain ahead by any margin at all, I reserve the right to change my forecast for the state (Update: ARG poll of the state shows Obama up 1% in NC. It functions as a tie-breaker. Update 2: Zogby final tracking poll moves 2 points to McCain, so that breaks the tie in the other direction). Oh--and the national popular vote is just a guess based on the Pollster.com national average. I'm only banking my methodology on the state results.
Senate
Democratic Pickups: Alaska, Colorado, New Hampshire, New Mexico, North Carolina, Oregon and Virginia.
Run-off in Georgia where we fight for the Employee Free Choice Act
Extremely narrow loss in Minnesota
If I have some free time tomorrow, I'll spruce up the final percentages, and post them here. More likely, I will finish them after the election, to test how well my methodology worked.
House Best Guess: Democratic Pickup of 24 seats, for a total of D 260-175 R in the House.
Guess Range: 21-27
Read the entire final forecast here
Well, that was a fun year of forecasting elections. What have you got for your final numbers?
Obama at 338 minimum: Obama wins the Kerry states plus Colorado, Florida, Iowa, Nevada, New Mexico, Ohio, and Virginia.
Three toss-ups: Missouri, North Carolina and North Dakota are the only remaining toss-ups, in that polls yet to be released could still flip those states.
Four close states for McCain: Arizona, Georgia, Indiana and Montana are projected for McCain, even though there is still a small chance for Obama to win one or more of those four states.
This projection comes from looking at the simple mean of polling conducted during the final week of the campaign. I stand by this methodology despite it's simplicity, and even to a certain extent because of it's simplicity. I have come to believe that forecasting close (within single digits), statewide, general elections accurately is not the sole province of statistical wizards and those who wish to argue with polls. Instead, I believe it is something anyone can do as long as they have an Internet connection, and are willing to not argue with the polls they don't like. As long as you have the courage to look at all of the recent polls and avoid adding any special sauce of your own, then you can project close (within single digits), statewide, general elections as accurately as any "professional" forecaster around.
On November 5th, we will know if I am correct. However, if I turn out to be wrong, and Obama scores a larger than expected victory of around 400 electoral votes, the reason will be due to cell-phone only households. Within the last twenty-four hours, both fivethirtyeight.com and Pollster.com have released studies indicating that Obama leads by about 4% more in polls that include cell-phone only households, than he does in polls that do not. A shift of that magnitude has the potential not only to push all three of the toss-up states to Obama, but also all four of the narrow McCain states. This could give Obama up to 406 electoral votes, and a national popular vote victory of 9-10%.
Now, I'm sticking with the "use all polls" and "don't argue with polls" philosophy, and instead projecting Obama to win by around 7% nationally and to pull in between 340-360 electoral votes. With only one day left, I will not abandon my old mantra, "the truth is always in between." However, this cell-phone only discrepancy should still give hope to those of you working in those four "Lean McCain" states. Keep in mind that there could also be a discrepancy in the other direction due to a lingering "Bradley effect," that might push Obama's victory down to around 5% or so, and which could still tip Florida and / or Ohio toward McCain. There is no circumstance I can forsee where Obama drops below 291 electoral votes, meaning the Kerry states plus Colorado, Iowa, Nevada, New Mexico and Virginia.
I am forecasting 7% and 340-360, but I can see reasonable people pegging the range anywhere from 4% to 10%, with electoral votes ranging from 291-406 (or even 407, with Nebraska-02). What is your final prediction? Let's lay it on the line in the comments.
Update 3:00 p.m.: New Rasmussen swing state polls added. New Strategic Vision Pennsylvania added. New PPP Missouri added. Links can be found in quick hits.
Update 6:00 p.m.: Survey USA North Carolina, along with PPP Indiana and Nevada.
Update 8:00 p.m.: Strategic Vision Florida, Georgia and Ohio included.
Update 9:00 p.m.: New Florida, Georgia, Maine and Washington polls added.
Update 1:00 a.m.: New Florida, North Carolina, Pennsylvania and Virginia polls added.
Update 10:00 a.m. Tuesday: Final update completed. North Carolina flips to McCain.
Dark Blue (273): Obama +6.0% or more
Lean Blue (18): Obama +2.3%-+5.9%
White / Toss-up (87): Obama +2.2% to McCain +2.2%
Lean Red (28): McCain +2.3%-+5.9%
Dark Red (132): McCain +6.0% or more
Relevant States Chart
Polls 10/28-11/02 States not shown are further either direction
Dark Blue (252): Obama +6.0% or more
Lean Blue (66): Obama +2.3%-+5.9%
White / Toss-up (60): Obama +2.2% to McCain +2.2%
Lean Red (29): McCain +2.3%-+5.9%
Dark Red (131): McCain +6.0% or more
Relevant States Chart
Polls 10/26-11/01 States not shown are further either direction
Barack Obama has won the 2008 Presidential Election.
Yes, you read that right. And no, I am not joking.
People will probably say that I am calling the election too early, which could depress turnout. People might say that I am taking too much for granted, which is especially bad for a committeeperson in West Philadelphia. People might say that I am simply being foolish, because there is time left and a lot can change in four days. For these people, I have five quick points (more in the extended entry).
Dark Blue (291): Obama +6.0% or more
Lean Blue (62): Obama +2.3%-+5.9%
White / Toss-up (25): Obama +2.2% to McCain +2.2%
Lean Red (29): McCain +2.3%-+5.9%
Dark Red (131): McCain +6.0% or more
Swing State Chart
Polls 10/23-10/29 States not shown are further either direction
Update 7:45: Sorry for the delay in the update. New polls for Indiana, Iowa, Montana, New Hampshire, North Carolina and Wisconsin added. Next update at 3:00 a.m.
Update 1:30 p.m.: New polls from Colorado (2), Florida, Indiana, North Carolina, Ohio and Virginia (2) added. Also, in order to avoid a lag between state and national polls, I have only included polls taken entirely from October 23-29. Also in an attempt to reduce lag time, I have allowed for only one poll to be included in the averages.
As with yesterday, I will update the forecast twice during the day as new state polls come in. Methodology and analysis in the extended entry.
Current Forecaster Consensus: Obama 353, McCain 157, Toss-up 28
With six days and a couple hours before the polls start to close, I thought it would be helpful to put together a consensus of the six "small media" polling aggregation sites I frequent the most: Electoral-Vote.com, fivethirtyeight.com, Pollster.com, Open Left (of course), Real Clear Politics, and TPM Election Central. There is a consensus around the current status of 46 states, plus D.C. Here are the only four states where these six websites do not agree with each other:
Indiana: 538, Pollster, Open Left and TPM show McCain slightly ahead in Indiana, but Real Clear Politics and Electoral-Vote show Obama slightly ahead.
Missouri: 538, Pollster and RCP all have Obama slightly ahead in Missouri, Electoral-Vote and Open Left show an exact tie, TPM shows McCain slightly ahead.
Montana: 538, Pollster, RCP and TPM show McCain ahead in Montana, but Electoral-Vote and Open Left show an exact tie.
North Dakota: Open Left, Pollster and TPM show Obama slightly ahead. Electoral-Vote shows an exact tie. 538 and RCP show McCain ahead.
That's it. These six forecasters agree on every other state. North Carolina is on the edge, and losing it would knock Obama down to 338 electoral votes. In the next closest blue state, Florida, Obama leads by 2.7% to 3.4% in according to every forecaster. After that, in the remaining 311 blue electoral votes, Obama leads by at least 5.8% in every state according to every forecaster. This is what I meant earlier today about all forecasters showing a state by state Obama lead that is both deep and broad.
So, no matter which forecaster you choose, Obama has 311 solid electoral votes with only six days remaining. Is it closer? Sure, a little bit. Is Obama still solidly ahead? Yep.
There is no question that the tracking polls have tightened compared to where they were from October 21st through October 27th (though not from where they were from October 15th through October 20th). However, I just did the 2:00 p.m. update to the Presidential Forecast, and state polling has pushed Obama out to his most secure lead of the entire campaign. And I added a lot of polls:
Five new Pennsylvania polls, four new Ohio polls, two new Florida polls, plus one new poll each from Colorado, Nevada, New Hampshire, North Carolina, Virginia and Wisconsin added. Obama drops slightly in Florida, holds steady in Ohio, and improves everywhere else.
Obama's state by state situation is improving, even as the Pollster.com national tracker has dropped from Obama +8.8% on Saturday (his all-time peak) to Obama +5.8% today. However, according to my forecast, Obama now reaches 273 electoral votes in states where he leads by 8.3% or more, and hits 311 electoral in states where he leads by 6.5% or more. As I will discuss later in the day, I'm not alone in showing this vast Obama statewide lead, either. No doubt, on the surface it appears difficult to reconcile the tracking polls over the last couple of days with the state polling released over the same time period.
A partial explanation comes from the 2004 election, which Kerry lost nationally by 2.5%. Given 2004 results in the three most heavily polled swing states, Florida (Bush +5.0%), Ohio (Bush +2.1%) and Pennsylvania (Kerry +2.5%), and given an 8.3% national swing, those states are almost exactly where they should be right now. In my forecast, I have Obama +3.3% in Florida, which is 0.0% away from the expected result. In Ohio, I have Obama at +6.5%, only 0.3% away from the expected result. In Pennsylvania, I have Obama at +11.2%, only 0.4% away from the expected result. So, a partial answer is that there is no discrepancy between state polls and national polls, at least in the largest swing states. Earlier state and national polling diverged from one another in these states, but current state and national polling does not. This explanation also works for Wisconsin, where Obama is within 0.2% of his expected result given the national swing from 2004, and Missouri, where he is within 0.8%.
Still, that leaves a bunch of state polling in conflict with national polls. This week's polls from Colorado, Iowa, Georgia, Nevada, New Hampshire, North Carolina are all 2-4% more favorable to Obama than a simple 8.3% swing from 2004 can explain. In these cases, it can probably be chalked to a combination of several factors, including polling error, changing demographics, and improved Democratic campaign organization in these states. While 2-4% is a bit of a shift, it is not so large that a combination of such factors fail to provide a complete explanation.
This leaves Indiana, Montana, North Dakota and Virginia, where Obama is running between 7% and 12% of where he should be. However, Obama is equally under-performing relative to 2004 in a different handful of states, including Arkansas, Tennessee, and West Virginia. Rather than a broad difference, it seems to simply be a regional shift.
So, there is your explanation for the state poll and national poll discrepancy: there isn't much of one at all.
Dark Blue (311): Obama +6.0% or more
Lean Blue (27): Obama +2.3%-+5.9%
White / Toss-up (43): Obama +2.2% to McCain +2.2%
Lean Red (26): McCain +2.3%-+5.9%
Dark Red (131): McCain +6.0% or more
Targeting and Swing State Chart States not shown are 12.0% or more in either direction
Update (2:00 p.m.): Five new Pennsylvania polls, four new Ohio polls, two new Florida polls, plus one new poll each from Colorado, Nevada, New Hampshire, North Carolina, Virginia and Wisconsin added. Obama drops slightly in Florida, holds steady in Ohio, and improves everywhere else.
As with yesterday, I am posting the Presidential forecast early in the morning, and I will update during the day as new polls come in. Right now, except where you see asterisks, it only includes polls taken entirely after October 21st. I have to say, I have never seen a campaign that has ben tightening for so long that makes me so damn confident of victory.
(Dark Blue (291): Obama +6.0% or more
Lean Blue (47): Obama +2.3%-+5.9%
White / Toss-up (43): Obama +2.2% to McCain +2.2%
Lean Red (26): McCain +2.3%-+5.9%
Dark Red (131): McCain +6.0% or more)
Targeting and Swing State Chart States not shown here are further in either direction
I am going to try something different today. This is what the polling picture looks like at 2:00 a.m. eastern, before any of today's polls come out. I will continue to update it during the day, as new polls come out. We'll see how this goes.
Analysis and methodology in the extended entry.
Update (12:30 p.m.): New Montana, New Hampshire, North Carolina, Ohio, and Pennsylvania polls added.
Update 2 (2 p.m.): New Colorado, Indiana, Nevada, Pennsylvania and Wisconsin polls added.
STATE BLOGS
