No, it's not a toss-up: Coakley still leads

by: Chris Bowers

Fri Jan 15, 2010 at 16:09


(Just to be clear, in case I turn out to be way off, this forecast is mine, and does not necessarily reflect the opinion of anyone else at Open Left. - promoted by Chris Bowers)

The new consensus among election forecasters is that the Massachusetts special election is a "toss-up."  Stuart Rothenberg, Charlie Cook and Nate Silver have all described the election as such.  Pollster.com only shows Coakley ahead by 1.9%, which probably qualifies as a toss-up in their book.

This is a case where I am going to disagree with the consensus of election forecasters, and instead argue that the Massachusetts Senate campaign still shows a distinct lean toward Democratic nominee Martha Coakley.  I do this not based upon a desire to be contradictory, but instead upon a different reading of the empirical evidence available on the campaign.

As the only empirical data on voter preference available, polling data remains at the heart of any election forecast.  My research into the accuracy of various methods of reading polling data shows that the most accurate method still gives Martha Coakley a decided edge in the campaign.

Here are the polls I am looking at:

Massachusetts special election polling, 2010
Pollster Poll Mid-date Coakley Brown
Black Rock Jan 14 39 54
Research 2000 Jan 13 49 41
Suffolk Jan 12 46 50
Rasmussen Jan 11 49 47
Mellman Jan 09 50 36
PPP Jan 08 47 48
UNH Jan 04 53 36
Rasmussen Jan 04 49 41
Mean Jan 15 47.88 44.13
(While it is quite possible the Black Rock poll is juiced, until proof of that emerges, it will remain in the average.  Also, there are rumors of internal polling, but until those polls are released to the public, I am not including them.).

The simple mean of the eight polls conducted, and released to the public, on the campaign in 2010 still show Martha Coakley ahead by 3.75%.  That may not sound like a lot, but from 2004-2009, only 34 of the closest 143 campaigns for President (both state-level and national), Senate and Governor saw a swing of greater than 3.75% from the final, 15-day, simple polling mean to the final result.  Further, my research shows that the final, 15-day, simple polling mean was more accurate than any election website in predicting the results of these elections.  As such, I still see Martha Coakley as the strong favorite in this campaign. Even with the Black Rock poll included, I still give her an 88% chance to win.

It is pretty bold to claim that I have a more accurate means of predicting elections than websites like Pollster.com and fivethirtyeight, especially given that both websites took me to school in 2008.  In the extended entry, I explain the basis for this claim, and in so doing the basis for arguing that Martha Coakley is still the favorite in Massachussets.

More in the extended entry.

Chris Bowers :: No, it's not a toss-up: Coakley still leads
In order to justify still calling Martha Coakley the favorite in the Massachusetts special election, I have to explain my new methodology for forecasting elections. I developed this new method over the last three months, basically as a hobby in my free time:

1. What do you mean by "the 140 closest" campaigns?
Specifically, I mean the 140 Gubernatorial, Senatorial, Presidential swing state and Presidential national popular vote campaign from 2004 to 2009 where the final polling average predicted a margin of less than 18.5%.  I only looked at those campaigns because, well, I don't think election forecasters are needed for campaigns decided by 18.5% or more.  

I drew the line at 18.5% because I thought, ironically, that forecasting who would win Massachusetts in the 2008 Presidential election was pointless.  Further, every swing state and Senate seat that was seriously contested by the two major parties in 2008 and earlier fit inside the 18.5% range.  In short, 18.5% was the widest net I could throw and still argue I was looking at the meaningful races where an election forecaster might be useful.

From 2004-2009, there turned out to be 143 of these campaigns.  I did not look at House races.  I did not look at primaries--only general elections.  I would very much have liked to look at campaigns for 2002 and earlier, but I couldn't find a complete online resource for public polling on those campaigns.

2. What did you find?
I found, rather surprisingly, that the simple mean of (almost) all polls with the majority of their interviews conducted in the last fifteen days of a campaign was more accurate at predicting the final election margin than the far more statistically informed methodologies of Pollster.com and fivethirtyeight.com.  Further--and this was just as surprising--the final, 15-day average was more accurate than the final, 10-day (or 20-day, or 25-day, or 30-day) average, which meant that including older polls actually made the forecast more accurate, not less.

Across the 52 campaigns that Pollster.com (final numbers here) and fiverthirtyeight.com (final numbers here) both produced final averages for in 2008, here was the average error between the final predictions and the final result:

Error rates, final predicted margin to final vote margin, 2008
Pollster 538 Simple 15-day mean
Mean error 2.76 2.88 2.56
Median error 2.14 2.16 1.68
This is not a dramatic improvement on the Pollster.com and fivethirtyeight.com predictions, but it is large enough to be noticeable and (I think) significant.

To make sure it wasn't a fluke, I went back to 2004 and 2005-2006, and measured the average error rates for the Simple 15-day mean method.  The results were strikingly similar: a mean error of 2.46 in 2004, and a mean error of 2.56 in 2005-2006.  Overall, across all 143 campaigns, the mean error was 2.54, and the median error was 1.76.  The similarities of the numbers is striking--it performed just as well in the past as it performed in 2008.

Further, I checked to see if the simple polling mean would perform better if a different date range than 15-days was used.  Here is what I found when looking at some other date ranges, across the same 143 campaigns:

Mean error rate, various date ranges
30-day 25-day 20-day 15-day 10-day
Mean error 2.63 2.60 2.56 2.54 2.59
the 15-day performed (very) slightly better, but really there is no significant difference.  This is perhaps the most important finding of all: including older polls in the averages, including those up to one month old, does not signficantly affect the overall accuracy of the averages.

3. What does this mean?
I drew a couple of conclusions from all of this:

  1. Special sauce has no effect. If Pollster.com and fivethirtyeight.com performed equally well in 2008, then the differences between their two methodologies is not significant.  This means that the extra weights 538 puts into the mix--a demographic regression, weighting by past pollster accuracy, adjustments for pollster "house effects," and weighting by poll sample size--don't seem to have a positive impact on the overall accuracy of the forecast.  Pollster.com has none of those weights, and performed (very) slightly better.  Pollster.com came to the same conclusion in December 2008.

  2. Recentness doesn't matter, either. One "special sauce" adjustment Pollster.com's regression estimates do make is that more recent polls have more impact on the overall estimate.  Fivethirtyeight adjusts for recentness, too.  However, the simple mean estimate does not adjust for recentness, and seems to produce more accurate results.  Further, the simple mean itself does not appear impacted by recentness, given there is virtually no difference in the overall accuracy of the 30-day, 25-day, 20-day, 15-day and 10-day polling averages.
Now, this is all very counter-intuitive, since one would think that polls taken closer to an election are more accurate than polls taken further out from an election.  And, in fact, there is strong empirical evidence demonstrating this.  However, it appears that including less recent into a polling average actually improves the accuracy of the overall average, even though the older poll are less accurate than more recent polls.

My explanation for this is that, according to the study that showed older polls are less accurate (see page three of this PDF), the older polls were not that much less accurate. At the same time, polling averages become more accurate when more polls are included in the average (I don't have specific numbers on this at this time, but a quick glance at my work suggests this).  So, what is happening is that the inclusion of slightly less accurate polling is improving the accuracy of the overall polling average simply by adding more data.  The increase in the amount of data in the system more than cancels out the inclusion of slightly less accurate data.

There appears to be a "sweet spot" for the accuracy of simple mean polling averages at around 15-days before an election.  The difference is minor, but I am going to run with it until the numbers suggest a different date.  Far out from an election (such as, say, nine months from the 2010 midterms), I will use a much wider range of dates for my polling averages (90 days for my Senate forecast, and 30-days for the National House ballot)

4. Can I see your data?
Sure. You can download the zip folder with all 21 spreadsheets here:

Election forecast study

Some notes on the data:

  • For 2004-2005 all polls were taken from Real Clear Politics.  For 2006-2009, all polls were taken from Pollster.com
  • All election results are taken from Dave Leip's atlas.
  • Zogby Interactive polls and Columbus Dispatch polls were not included, due to their horrendous past performance and questionable methodologies.
  • Strategic Vision polls, as it seems likely those were never real polls.
  • Partisan and Campaign-funded polls are included.  Since this methodology works because it equally weights as much scientific, relevant data as possible, the more polls, the better.
  • For the same reason, if there is more than one poll from a single organization in the given date range for a campaign, I include all of those polls.
  • Polls need to have 50% or more of their interviews conducted in the given date range to be included in the averages.
  • At least two polls per campaign, even if that means including polls that are older than the date range in question.  Otherwise, I'm not forecasting--I am just reporting on a single poll, or throwing my hands up in the air saying that forecasting is impossible in this case.  Both are unacceptable.
  • The date range for the polls included do not include election day.  For example, the 10-day average for 2008 includes polls where a majority of their interviews were conducted on October 25th or later, given that Election Day in 2008 was on November 4th.  The 15-day averages for 2008 included polls with a majority of their interviews conducted on October 20th or later.  The 20 day averages included October 15th or later, etc.
And basically, that is it.  It took me months to put all this together, but a surprisingly simple answer to accurate election forecasts emerged.  Just take the simple mean of (almost) all the polls conducted over the last 15 days.

5. What does this mean for Massachusetts?
What this means for the Massachusetts special election is that I will include all of the polls that conducted the majority of their interviews on January 4th, or later, into the average.  This produces the following result:

Massachusetts special election polling, 2010
Pollster Poll Mid-date Coakley Brown
Black Rock Jan 14 39 54
Research 2000 Jan 13 49 41
Suffolk Jan 12 46 50
Rasmussen Jan 11 49 47
Mellman Jan 09 50 36
PPP Jan 08 47 48
UNH Jan 04 53 36
Rasmussen Jan 04 49 41
Mean Jan 15 47.88 44.13
As I noted above, in the 143 campaigns I looked at in this study, the final, 15-day simple poll mean differed from the final margin by more than 3.75 on 34 occasions.  Given that the polls could just as easily be favoring Coakley as they could be favoring Brown, this comes out to only a 12% chance that Scott Brown will win.  As such, I still consider Martha Coakley to be the clear favorite, and this campaign far from being a toss-up.

Now, it is possible that special election polling is more like primary polling, and making turnout modelling much more difficult for pollsters.  In fact, the wide range of results among the polls suggests that is actually likely.  However, the truth is that I don't have numbers anywhere approaching the level of detail for primary and special elections that I have for general elections.  I want to base my forecasts on thorough, empirical research, and I just don't have that for a special election.  Truth is, there are so few special elections, there wouldn't be enough data points for a convincing study, anyway.

So, I am going to stick with the general election research I have conducted.  That research suggests that Martha Coakley is still the clear favorite, and this campaign is not a toss-up.  The 144th test of this theory takes place on Tuesday, January 19th.


Tags: , , , , , , , (All Tags)
Print Friendly View Send As Email

Hmm, how are the bookies seeing the race? (0.00 / 0)
If the odds are good, OpenLefters should place bets now. Looks like we have an information advantage over the public!

And, no, I'm not joking. Chris has been right with his predictions before, and a 88% chance is a quite sure bet.


Well, I don't bet on elections (4.00 / 3)
There is already enough at stake in them, anyway. :)

[ Parent ]
intrade at 61% for the dem winning (4.00 / 1)


[ Parent ]
To which I should add (4.00 / 3)
There is no guarantee at all that this method is better in every campaign. It isn't. It has just proven slightly better, on average, across several dozen campaigns.

It is entirely possible that this method will not prove more effective for this specific election. Still, I thought it was worth noting that there is a poll averaging method--one that I think is the most accurate--that outlies significantly from other poll averaging sites at this time.

An interesting test case, even if it will end up just being the 144th test case in a long line.


[ Parent ]
I don't believe MA will elect a repug, but (0.00 / 0)
would feel more confident if Chris got similar numbers for a set of special elections and primaries only, excluding normal general elections.

There is no such thing as a free market.

[ Parent ]
Everyone who will be surprised ... (0.00 / 0)
when Brown gets smoked, raise your hand.  

Not to presume to argue with Nate Silver, but as a non-poll-wonky MA Dem, I simply note the following.  Fear-mongering on this race serves literally everyone's interests.  Let us enumerate them:

The media (it's a horse race!)
The pundits (Dems in trouble!)
The Brown campaign (we can win!)
The Rs (we're relevant!)
The DLC'ers (see - you have to tack to the middle - HCR will be electoral death for us!)
The Obama administration (toe the line on this or I won't come campaign for you like I did for MA D(!!!) Coakley!)
The Coakley campaign (more money for us!)
The pollsters (See - you need to poll even unpollable races cause you never know what might happen!)

Coakley by 12.  MA is not that different from the rest of the country.  There is a hard core %30 that would vote for a ham sandwich over Jesus H Himself if the sandwich promised to hold the line on taxes and spending.


This is great news for John McCain!!! (4.00 / 1)


[ Parent ]
Unfortunately, right now this feels a lot like... (0.00 / 0)
Hillary in WI.  It seemed like she was going to do well there, and she got smoked by the "celebrity".  If we had more time, I'd suggest that the campaign write up a, "He's the biggest celebrity in Massachusetts, but is he ready to lead" commercial.

Unlike Hillary, there is no hard core group of supporters ready to run the gauntlet for her.  We've been begging people to go out and vote for her.  It seems that no one really likes her very much, except maybe for a few old women.

Like Hillary at the time, the media is gunning for her and trying to sink her, she's being overwhelmed by superior forces and $$$ and nothing she seems to do makes a dent, while everything he says is golden.  

Hillary was able to eventually knock Obama off his perch a bit, but Coakley hasn't seemed to be able to do that.  IF we can get some dems excited, we might be able to squeak this one out.

I certainly hope that the Globe has been int he field the last week... We could use some good news.  

REID: Voting against us was never part of our arrangement!
SPECTER: I am altering the deal! Pray I don't alter it any further!
REID: This deal keeps getting worse all the time!


[ Parent ]
Just out of curiosity (0.00 / 0)
Do you think Coakley's problems are more her own lack of vitality, or her complacency in campaigning (which is something that probably any Democrat would've fallen into)?

In other words, would this race be any different if a more liberal and colorful candidate like Mike Capuano were the Democratic nominee?


[ Parent ]
I don't know... (4.00 / 1)
People are attacking her for not responding to Brown earlier, but that would have elevated her stature.  According to her web site, she didn't go totally AWOL, she had bunches of campaign events through the month and ads as well.

Brown made himself into a celebrity, and people like celebrities.  I always figured that MA voters were smarter than that, but I guess they can be just like anywhere else in the country.

She seems totally powerless... she doesn't have the charisma of Brown, nor can she dole out the one liners.  She's competent and that usually would have been enough, but not this time.

Capuano noticed the voter anxiety when he campaigned for the primary... perhaps he would have been more aggressive, but he could have flubbed things up as well.  He's a bit of a hothead and could have gaffed.

REID: Voting against us was never part of our arrangement!
SPECTER: I am altering the deal! Pray I don't alter it any further!
REID: This deal keeps getting worse all the time!


[ Parent ]
Let's say that Capuano had kept the same kind of schedule that Coakley had between the primary and when the race started picking up a few weeks ago (0.00 / 0)
because honestly I think any Democrat would've been lulled into complacency.

If it were Capuano campaigning for the last three weeks instead of Coakley, do you think he'd be doing better than she is?  Assume that he'd be himself, and he might make a few minor gaffes but nothing that would kill his campaign.

I ask because conventional wisdom holds that the more tepid, centrist candidate is the "safer" choice, and I wonder if this race can finally begin to end that argument.


[ Parent ]
I don't know him that well... (0.00 / 0)
He apparently made an ass of himself at the primary debates which is what killed him there...

REID: Voting against us was never part of our arrangement!
SPECTER: I am altering the deal! Pray I don't alter it any further!
REID: This deal keeps getting worse all the time!


[ Parent ]
I dunno (4.00 / 1)
like lordmike said, Capuano didn't do well in debates and he really didn't have any support outside of Boston.  

[ Parent ]
I also think she's gotten a lot of bad luck.... (0.00 / 0)
Just sheer bad luck, and the matchup was bad from the get go....  The guy made a splash for himself, was something different, and it snowballed from there!

REID: Voting against us was never part of our arrangement!
SPECTER: I am altering the deal! Pray I don't alter it any further!
REID: This deal keeps getting worse all the time!


[ Parent ]
Except she's not (4.00 / 1)
her own internals have her trailing.

You didn't even read the post, did you? (n/t) (4.00 / 4)


[ Parent ]
I read the post... (0.00 / 0)
...it ignores reality.

[ Parent ]
PS didn't mean to sound harsh, if it did,... (0.00 / 0)
...anyway, apologies in advance if saying your post ignores the reality of the economy and the huge losses of hundreds of thousands of Democratic voters since Obama came into office and the debacle the Democratic congress has made of their supermajorities. Numbers are nice, all else being "normal," but when they ignore that there's nothing normal this mid-January one year into the great recession, it's fair to question numbers.

[ Parent ]
this post is reality (0.00 / 0)
Its based on a thorough examinationof empirical evidence. That is reality. Subjective feeling are the opposite.

[ Parent ]
Chris... (0.00 / 0)
What about newest ARG poll? That probably puts your number a bit closer to scary territory I would think.

[ Parent ]
10% unemployment is not subjective feeling (0.00 / 0)
etc. You know what I meant.

[ Parent ]
Just more questionable "inside" news... (0.00 / 0)
...from our self described media staffer, I guess. Nothing that can be verified. The stuff you read in the papers, starting with "a campaign staffer, who wants to stay anonymous, explained that...". D'oh.

[ Parent ]
One person reporting that insider tidbit (0.00 / 0)
Is Marc Ambinder at The Atlantic

Things You Don't Talk About in Polite Company: Religion, Politics, the Occasional Intersection of Both

[ Parent ]
That's interesting. Marc Armbinder, huh? (0.00 / 0)
Hmm...

[ Parent ]
Pls note: Instead of the usual brouhaha about Campaign insiders... (0.00 / 0)
..not willing to put their name on record, Armbinder simply avoids saying anything about his sources! Hmm, what are the Atlantic's guidelines about anonymous sourcing anyway? Guess they don't have any rules...

[ Parent ]
Always ask (4.00 / 2)
when you see a "leaked" internal poll why someone would leak it.  In my experience I have seen these "leaks" used for a number of purposes.  In New Hampshire one campaign I worked on would "leak" one night's results to create the impression that they were behind (and lower expectations) .  In other instances I have seen consultants/advisors do it because they lost an internal campaign battle and were trying to get even or, alternatively, were trying to get the strategy changed.

Take such leaks with a grain of salt.



[ Parent ]
hope your right (4.00 / 1)
i am not ready to lose the super majority quite yet...

Hmm. Why not? (4.00 / 1)
Really, aiko, what's it good for anyway? I'm not at all sure if it wouldn't be better if Oakley loses. It would be a pity for her, she doesn't seem to be a bad Dem, but it would force the Dems to finally talk business and do something against the damn filibuster.

[ Parent ]
I supported Capuano in the primary (0.00 / 0)
and I think we can get a better, more liberal Democrat out of MA than Coakley (and we should, given how safe that seat normally is).  Of course I'd rather Coakley win but if she loses we can at least get a better liberal in 2012 rather than having to wait another couple of decades for Coakley to finish her service.

[ Parent ]
Stakes are much bigger than that... (4.00 / 2)
We lose this now and a gigantic number of things get shut down until 2012 and perhaps even longer than that, and we probably lose even bigger come November since HCR and anything else on the docket for this year will be toast, unless Democrats grow a pair and start using reconciliation or kill the filibuster (and a loss in MA will basically guarantee Democrats running with their tails between their legs, so we can pretty much forget about either of those things happening this year too).

[ Parent ]
I know (0.00 / 0)
I was just joining Gray in looking for silver linings in a Coakley defeat.  Obviously I'd much rather see her win.

Setting all that aside, if Coakley wins and ends up being another timid Democrat (e.g. one who does not support Medicare for All, who weakly gives into anti-liberal Beltway conventional wisdom, etc.), I really do hope Capuano or some other strong liberal runs against her in the primary in 2012.  An MA Senate seat is too valuable to be wasted on anyone other than a proud, feisty, true-blue liberal Democrat.


[ Parent ]
it'd be nice if we had a supermajority (0.00 / 0)
In terms of getting reform of, well anything, passed, it seems we barely have 53 or 54 votes in favor at any given time. Several Democratic Senators are completely unreliable Democratic votes, as is Lieberman's (unless he gets to extort concession for his vote). So, the Democratic majority slips to 52 for two years; there's a good chance a better Democrat will win the seat back in 2010.

[ Parent ]
2004 to 2008 was a radically different economy (0.00 / 0)
than what we've got today. Even throughout most of 2008, voter denial continued that the economy was in freefall. Today I think everyone knows they're worth 20% or more less than they were two years ago. Many of them vote, or as the case will be in the most liberal state in the nation, will choose not to vote next Tuesday for Democrat Coakley.

PS Really enjoy your number analysis, but numbers alone fail to account for massive shifts in voter sentiment like we've seen over the past 12 months since Obama began to bury us all.


Obama's economy (0.00 / 0)
than what we've got today. Even throughout most of 2008, voter denial continued that the economy was in freefall. Today I think everyone knows they're worth 20% or more less than they were two years ago. Many of them vote, or as the case will be in the most liberal state in the nation, will choose not to vote next Tuesday for Democrat Coakley.

PS Really enjoy your number analysis, but numbers alone fail to account for massive shifts in voter sentiment like we've seen over the past 12 months since Obama began to bury us all.


nice work (0.00 / 0)
And it looks like Coakley would have a 6-7% lead if you don't count the Black Rock poll, which it seems there are good reasons to disclude.

That poll is a joke... (0.00 / 0)
They gave Brown a 5% lead in December when every other pollster had her at 30% or more... Nate says they have a R+25 house effect...  

They've done other whackanoodle polls before and they are all crazy...

REID: Voting against us was never part of our arrangement!
SPECTER: I am altering the deal! Pray I don't alter it any further!
REID: This deal keeps getting worse all the time!


[ Parent ]
Nice disclaimer (0.00 / 0)
(Just to be clear, in case I turn out to be way off, this forecast is mine, and does not necessarily reflect the opinion of anyone else at Open Left. - promoted by Chris Bowers)

But if you do turn out to be right on target, can we all share some credit? :D


You know what I'd like to see? (0.00 / 0)
I'd like toe see her "registered" voter numbers, not just likely... that would be good to see...

REID: Voting against us was never part of our arrangement!
SPECTER: I am altering the deal! Pray I don't alter it any further!
REID: This deal keeps getting worse all the time!


wtf... (4.00 / 1)
I hate sports shit from politicians anyway, but this is just stupidity. I wonder how much dumb sports statements actually cost politicians anyway.  I certainly don't have the highest opinion of voters in general (as others on this site have troll-rated me for), but I do think people are probably capable of separating out sports stuff from political stuff... I doubt this will flip any votes, but maybe people who were on the fence about voting (who would vote for her) will be less willing to now, and in her position she can't afford this now.

Man, could Democrats have picked a shittier candidate? Her career is over after this if she doesn't somehow pull it out.


[ Parent ]
She is a gaffe machine... (0.00 / 0)
It's obviously a joke and a brain dead moment, but the only people who care aren't the ones who will be voting for her, anyways...

I thought Kerry was bad... this woman is terrible!

REID: Voting against us was never part of our arrangement!
SPECTER: I am altering the deal! Pray I don't alter it any further!
REID: This deal keeps getting worse all the time!


[ Parent ]
I think 2004 and 2008 (0.00 / 0)
were unusually stable elections over the last 10 days.  As I posted about a week out here in 2008 (and got creamed for it) I think analysis based on this data set will eventually show that when used to predict subsequent elections.  

In short, I believe the probabilities I read at 538  are too confident, and there will come an election where this is seen.  The reason is that the probabilities are based on polling error, and do not account for a late event that shifts all of the races in one direction (eg 1980 and the hostage negotiations with Iran, the bombing halt in 1968 and the Cuban Missle Crisis in 1962).  These exogenous events objectively happen with greater frequency than these estimates suggest.  

There are academic studies based on pre '06 data that suggest that the median is actually better than the mean.  I haven't run the data since '06 to see if that is still the case.

I think Coakley is going to win.  When I look at the internals of some of these polls I get suspicious.  Suffolk, eg, has Obama's job approval rating in Mass at 48-43.  Yet if I take the national percentages and plug that into the Mass electorate, I can't see how those numbers can be right .  Suffolk has 41% of Likely Voters as Democrats and 39% as Independents.  There is no way to take those percentages and square that job approval rating.  


Hope you're right... (0.00 / 0)
Last few polls and rumors making me very nervous...I hope Ras and PPP give us better news over the weekend, but the trend looks like its gotten worse, not better.  I don't know if Obama and Clinton swamping the state in the final days will really do anything.

[ Parent ]
As anyone who was (4.00 / 2)
here during the NH primary will remember, take my predictions as worth less than zero...

Democrats wind up voting for Democrats.  In the end I think that fact will win out.

But if we lose this seat, the Democratic Party may blow itself apart.

I am haunted by the thought that Geithner and Bernanke are the  political equivalent for Obama of what Katrina was for Bush - a defining event that could simply not be recovered from.

I am nervous too.


[ Parent ]
two predictions: (0.00 / 0)
coakley wins by 1 or 2 points. And, obama gets mad (i.e.,scared)and we see a little more fight, with a populist streak in 2010.  

[ Parent ]
Certainly won't help with (0.00 / 0)
progressives.

My blog  

[ Parent ]
Those who won't turn out because of Clinton and Obama (4.00 / 1)
were never turning out anyway, period, no matter who the nominee was.

Another words, barely anybody.  


[ Parent ]
You're just looking for the next rationalization (0.00 / 0)
for keeping the fuckin centrists in control.  That is why I advocate writing in instead of kos's stupid not voting solution.

My blog  

[ Parent ]
and your just looking for a pyrrhic victory (0.00 / 0)
because it annoys you no one as far as left as you can win elections.  

[ Parent ]
Obama ran as far left as I am (0.00 / 0)
and would be a good president if he governed that way.

My blog  

[ Parent ]
You mean... (4.00 / 1)
The progressives that approve of Obama almost 90%? In any case, I don't think that that's the issue. More likely is simply the fact that people don't care about endorsements, and the people this targets are probably voting anyway.

[ Parent ]
self identified "liberal democrats" (0.00 / 0)
are not the same as progressives.

My blog  

[ Parent ]
and pumpkins aren't the same as squash (0.00 / 0)


[ Parent ]
most people probably just think (0.00 / 0)
a liberal dem is a kneejerk dem voter.

My blog  

[ Parent ]
Outcomes vs margins? (4.00 / 2)
It's interesting that everyone uses average margin-of-victory error rate to measure the success of their poll-aggregating system.  Fundamentally, we're interested in who will win, not in accurately predicting their margin; say if Chris's system beats Nate's, but only because it gets the margin more accurate on the large-margin races (say, wins by 10 or 15 points), it may still not be actually better in terms of what we're really interested in: predicting who will win X election.

The real test is how many errors in predicting the winner a given system generates.  If system A generates more winner errors but has a low average margin error, and system B generates fewer winner errors but has a higher average margin error, we should prefer to use system A in predicting who will win election X.  

The problem, of course, is that for almost all races, the winner is correctly predicted.  So we need a large dataset -- but the one Chris has is pretty big!  I spent about 15 minutes with the 2008 data,* and lo, the 10, 20, 25, and 30 day sets all make two winner errors, while the 15 day set makes only 1.  But of course we'd need to check that for the other years, and to check it against Nate and pollster.com's successes.  

As a intermediate position, you might try looking not at races within 18 points, but races within 5-10 points, say.  Those are the harder ones to predict, and the ones where there is more of an overlap between win prediction accuracy, and margin accuracy.  We already know this race is close (as it is for all races we're worried about), so what we want to know is which system is best at predicting the close races.  It does us no good if your system is better at getting the margin right on the 10-18 point races, and less good on the 0-10 point races.

*Just paste into an excel cell "=IF(SIGN(predmargin)=SIGN(realmargin),0,1)" to get a 1 for an incorrect prediction and 0 otherwise.


doesn't everyone agree that a) she has been leading and it's close now and b) the trendline is bad for her (0.00 / 0)
this method, like all others, just gives a snapshot of where we are today - it doesn't predict where we're going to end up because to do that, you would need to know what will happen in the next few days, as pointed out above.  personally, i'd prefer a qualitative account that amalgamate the opinions of long time massachusetts politicos to supplement the numbers which are all going to say what i said in the subject.

I'll be damned. This actually makes perfect sense to me. (4.00 / 1)
I think I may even be able to explain why this works. I'm no statistician but I know something about digital signal processing (DSP) and a lot of the same concepts apply. In DSP terms, the primary objective for all polling meta-analysis would be signal identification. The enemy of course is noise.

Basically, you have a signal that you want to measure -- in this case, the level of support for a political candidate at a particular point in time. Data is acquired via opinion polls. The data includes a certain noise component from various sources, e.g., quantization error (MoE), background noise due to temporary effects of moods, weather, current events, emotional responses to advertising... whatever. You also have potential biases to deal with (DC offset in DSP terms) due to possible house effects, when evaluating data from multiple pollsters.

The object is to cut through that thicket to find truth. Two of the three most common approaches to this problem in DSP are low-pass filtering or oversampling. Both rely on averaging over some period and work in principle because the noise component tends to be random. That means that you can expect as many sampling errors to the high side of your signal as to the low side. Averaging multiple samples should therefore tend to produce regression to the mean in the noise component, leaving you with more signal, less noise. Of course if that requires measuring over a longer period of time, then it tends to work best in identifying fairly steady-state signals.

What pollster.com almost certainly does (and probably 538 as well) is an exponential rolling average. When applied to a moving signal (e.g., a trend line) it's exactly like turning down the treble control on your stereo. Since the noise is assumed to be random, the thinking is that it should be fluctuating faster than the part of the signal you care about -- as when you're just listening to someone talk, stuff happening up in the cymbal-crash range is likely noise. Exponential averaging can work well for finding trends in moving signals because it allows fine adjustment in the trade-off between recency and reduction in the high-frequency component. It can also be adjusted on the fly to accommodate irregular sampling intervals.

An evenly weighted, "flat-top" average (what you're doing), makes a crappy low-pass filter by comparison, particularly if samples are not evenly spaced in time. But it should tend to give you somewhat better regression to the mean for a given data set if errors can be assumed complimentary. In fact as long as the averaging period in shorter than the rate of change in your signal, you should get a 25% better signal-to-noise ratio for every doubling of the number of samples (polls) going into the average. Simple averaging might also tend to give you somewhat superior cancellation of biases in samples from multiple sources if those tend to be random relative to each other. But if you average over too long a period, relative to the rate of any changes in your signal, then you would begin to introduce errors of your own.

So looking at your results here, it appears to me that you have observed a tendency for candidate preferences to calcify in the final 2-3 weeks of even fairly close elections. The fact that you hit a point of diminishing returns beyond that suggests that the window gets too wide for the rate of change in the signal beyond that. At that point the benefits of including more polls begin to be canceled or outweighed by errors being introduced by the averaging process.

We might also infer that undecideds may tend to break toward either candidates in similar proportions as decideds do, but that it would take more analysis to confirm. Of course there are probably still a ton of caveats and special elections might prove to be special cases. But it's pretty fascinating stuff (if you happen to be some kind of geek, I guess).

Anyway, apologies for the loooong comment. I couldn't think of a shorter way to explain it. And BTW, nice work!


Well done. Now, here's your next assignment... (0.00 / 0)
I think there's a slight tweak you could introduce to your study that might add to its value. I suggest you look at the historical data you've gathered and evaluate the accuracy of polling averages excluding the final week of polling data.

Why do that? Because, if I'm working on a campaign (or supporting a candidate from outside the campaign), what I want to know is not just who's likely to win, but how polling can inform my efforts to increase my favored candidate's chances. While polls get more and more accurate as the election approaches, there is also less and less time for a campaign to tailor its actions to shape the world of opinion described by the polls. So as a political activist, I'd be more interested in what technique for averaging polls I should rely on when the election is still a week (or a month, or three days) away. For example, is the simple mean average of the polls ranging from 7-22 days before the election better or worse than the prediction of Pollster.com seven days out? It sounds like you have most of the data you would need to perform this calculation (although it might be a bit of work to isolate the Pollster & 538 projections at specific pre-election-day intervals), and I think it might prove to be a valuable exercise.


USER MENU

Open Left Campaigns

SEARCH

   

Advanced Search

QUICK HITS
STATE BLOGS
Powered by: SoapBlox