Here is the new polling chart for the Massachusetts special election:
Massachusetts special election polling, 2010
Pollster 
Poll Middate 
Coakley 
Brown 
PPP 
Jan 16 
46 
51 
InsideMedford 
Jan 15 
41 
51 
Black Rock 
Jan 14 
39 
54 
ARG 
Jan 13 
45 
48 
Research 2000 
Jan 13 
49 
41 
Suffolk 
Jan 12 
46 
50 
Rasmussen 
Jan 11 
49 
47 
Mellman 
Jan 09 
50 
36 
PPP 
Jan 08 
47 
48 
UNH 
Jan 04 
53 
36 
Rasmussen 
Jan 04 
50 
41 
Mean 
Coakley +1.1 
46.82 
45.73 
Based on a study I conducted of the 143 closet, statewide general elections from 20042009, I have been making some bold predictions in the Massachusetts Senate race of late. Even though virtually every other election forecaster has moved this campaign to "tossup" or "lean Brown" arguing that it is still "lean Coakley."
According to that study, Coakley would still have a 67% chance to win. This is because the final, 15polling average erred by at least 1.09% in 94 of the 143 elections in the study, and since there are equal chances the polls could be wrong in favor of either candidate. Given how few elections ever come so close to even odds, that is close enough for me to describe the election as a "tossup."
However, this is a special election, not a general election. Even though some local election officials are seeing signs that turnout will be very high, perhaps it is best not to apply a general election study to this campaign, but instead to use a study of primary and special election results. As such, today I looked at the 37 closest (final polling margin within 18.50%) Presidential primaries in 2008, along with the two general, House special elections in 2009. The study compared Pollster.com's final estimates for those 39 campaigns with the simple polling mean of the last15 days of those campaigns. Here are the results:
Primary and special election error, 39 campaigns

Pollster.com 
Simple 15day 
Mean error 
7.03 
6.41 
Median error 
5.76 
4.80 
Closest 
18 
20  Two things jump out from this chart. First, the simple, 15day polling mean again proved slightly more accurate. Second, the error rate from both methods is catastrophicmore150% the error for general elections.
If this election is more akin to a Presidential primary and a special election for the U.S. House, then Coakley's chances of winning are 56% in my methodology (which is extremely close to Nate Silver's 57% estimate), and only 12% according to Pollster.com's current trendline (Brown +7.9%). That is a clear tossup on my end, and a strong lean toward Brown at Pollster.com.
Next, I took the comparison a step further Looking only at the 13 campaigns for which Pollster.com used its ''Loess' iterative locally weighted least squares regression" (which it only does with campaigns with eight or more polls), and where Pollster.com differed from the 15day, simple mean by 2.1% or more, I ran some more numbers:
Primary and special election error, 13 late trending campaigns

Pollster.com 
Simple 15day 
Mean error 
7.46 
8.38 
Median error 
7.36 
6.46 
Closest 
8 
5  These 13 cases compare apparent latetrending campaigns. These are important, because my 15day simple methodology should miss the late trends (because I weight older polls more heavily) while the more sensitive, Loess regression should better pick up the trend. Although the error rate for both methods is terrible, although there are few cases, and although the simple, 15day mean does better in terms of median error, the Loess regression still does a slightly better job overall of predicting the final results. In 8 out of 13 cases, putting more weight on later polls through a regression method was more accurate.
Now, it should be noted that there are a couple of famous cases where the simple 15day mean accurately predicted that an apparent, massive late trend was overstated: the 2008 New Hampshire Democratic primary, and the 2009 New York 23rd special election. In both cases, which happen to be the two most prominent of the 13 in the final "study," the simple 15day mean picked up a much smaller amount of movement toward Barack Obama and Doug Hoffman respectively. It still predicted the wrong winners (just like everyone else), but the elections were predicted to be very close and the eventual error rate was less than catastrophic.
Add it all up, and the campaign is, from a Democratic perspective, a tossup at best. My gut, and my research, tells me that it won't be a runaway Scott Brown victory, ala the Pollster.com trendline. If I were to go out on a limb without any hard data, I would drop the Pajamas Media poll, and weight the preJanuary 12th polls at only 85% of their value. That shows Coakley by 2.3%. If for no other reason, I like that as a prediction (barring more polls), because it still allows me to be contradictory with everyone else. As an election forecaster, when you are lacking a definitive conclusion, it is best to go against the herd. There is just more of an upside to being right in those cases.
However, even I can't deny that the numbers are improving for Brown seemingly everyday, and that he has a good chance of winning right now.
