Continued Presidential Poll Analysis

DJ Drummond has a (for once) non-wordy post that analyzes the current set of polls that have come in post-conventions for both parties. His motivation for doing the analysis was:

Maybe I'm just a suspicious type, but from what I could tell, a lot of voters had made up their minds before the conventions, and while I think Sarah Palin is going to make a big difference in the election, I did not expect a lot of voters to come over right away; I figure they want to find out more, first. The reason I say this, is that I found it strange to see poll numbers change so quickly. At least part of the answer, I found, was that the results were spun by fooling around with the weighting.

As (I hope) you are aware, polling agencies conduct a poll in their raw data they have percentages for those polled that break down party affiliation. The odds that these percentages match the party affiliation among actual voters on election day is of course infinitesimally small. So the polling agencies are forced to adjust their raw numbers to what they predict the election-day breakdown will be. That is where the science of polling ends and the art begins. It is impossible to know for certain what this weighting should be.

Drummond took an interesting approach. He averaged the breakdown over the last five national elections and used that as the expected. He then adjusted the raw numbers from some recent polls to these expectations. He arrived at the following numbers:

ABC News/Washington Post
August 21: 49-43 Obama original report. Historical weights indicate 49-48 Obama.
September 7: 47-46 Obama original report. Historical weights indicate 50-47 McCain .

CBS News
August 19: 45-42 Obama original report. Historical weights indicate 44-44 tie.
September 7: 46-44 McCain original report. Historical weights indicate 48-43 McCain.

The quick summary is that the current polls may be overestimating Republican percentages and as such be showing too large of a lead for John McCain. Read his whole post for more details. It is interesting to note that even with this adjustment, both polls show a ~4% lead for McCain.

Regardless of the actual adjusted numbers Drummond arrived at, one key take away point is to be very careful of overall national polls without looking at the internals. For example, this latest Gallup result is getting some press:


A 52% to 41% advantage to McCain on the question of "Who is a strong and decisive leader?" seems like a strong point for the Republican campaign. But if Republican responses are being over-weighted then the results are far less meaningful.

One way to see past such "mistakes" in weighting is to look deeper into the internals and see how one particular group--Republicans, Democrats, Independents--is responding. For example, pulling a chart I linked earlier:


Since this particular result is just what independents are saying, the weighting question doesn't enter into the equation. Until you want to determine the affect of this result on a national average, in which case you are back to the art of picking the correct weighting.

Polling results are always more than just two numbers scrolling by on a CNN headline ticker. If you are looking to do more than just feel good (or panic) about your candidate, it is always necessary to dig deeper and see that the internals might be trying to tell you.