Biased Polling: Who Is Being Polled?

In heated elections periods (and sadly now just in general) it is common for a state to be labeled simply blue or red. But of course that is not the case; there are often red and blue areas in a state. For example, California is of course left-leaning, but only in the urban areas; the rural areas of CA are conservative. Philadelphia and Pittsburgh are really, really blue but the rest of Pennsylvania is quite red. While these divisions usual follow urban/rural lines it is not always the case. Vermont is pretty uniformly blue, rural or otherwise.

This is an important realization to make when you consider polls and potential biases. When a pollster says that independents are skewed towards one candidate or the other, which independents are being polled? If one were to poll independents in Detroit, for example, you'd see a markedly different result than if you poll independents in the rest of lower state Michigan.

Of course, a reputable polling agency would take this distinction into account, but are they? It's tough to know because not every polling agency releases the county by county numbers. Here is one example, though.

In Virginia, Mason-Dixon released a poll giving Obama a 49-38 lead among independents. This seems odd in VA, considering how many ex-military personnel live in VA along with evidence that this group supports McCain heavily. A commenter at Hedgehog Report notes the following oddity in the county breakdown in the internals.

Northern Virginia: 170 interviews
Shenandoah/Piedmont: 95 interviews
Richmond Metro: 80 interviews
Hampton Roads: 140 interviews
Lynchburg/Southside: 70 interviews
Roanoke/Southwestern Virginia: 70 interviews

Last I checked, there are more people in Hampton Roads than in NOVA. Yet they have NOVA more interviews.

Northern Virginia is of course the most liberal area and overpolling here will skew your results.

My other data points are anecdotal. We'll start with what a commenter from PA noted, below.

I live in Bucks County Pa and there is nothing but McCain/Palin signs, which is why I'm always shocked to hear that BO is "so far ahead" in the polls. In Montgomery County, I see mostly McCain signs also.

This mirrors what I am hearing from family members in rural southern Ohio. It is a "sea of McCain-Palin signs". So much so that it is hard to find even a single Obama sign in the rural areas.

Where you live and the people that surround you can't help but influence you. I live and work outside of Boulder, CO. The Obama-Biden signs are overwhelming here. (I saw a car the other day that had an Obama bumper sticker proudly displayed next to a John Kerry one. Now that's dedication to a cause!) A friend in Austin, TX reports the same thing--it is all Obama, all the time. If you live in such an environment and you hear that Obama is way ahead in polling, it is easy to be forced to agree.

But if you live in an area like southern Ohio or western Pennsylvania, where the situation is opposite, it is hard to understand the national and state polling. If the pollsters are under sampling these areas, they are generating highly erroneous results.

And in some ways, this isn't just anecdotal evidence. You are influenced by the environment around you. DJ Drummond notes the locations of the major polling agencies.

ABC News: 77 W 66th St, #13, New York City, New York
CBS News: 524 W 57th St, New York City, New York
FOX News: 1211 Avenue of the Americas, New York City, New York
Gallup: 901 F St NW, Washington DC
Hotline: 88 Pine St, 32nd floor, New York City, New York
IBD: 12655 Beatrice St. Los Angeles, California
The Los Angeles Times: 202 W 1st St, Los Angeles California
Marist Institute: 3399 North Rd, Poughkeepsie, New Jersey York
Mason-Dixon: 1250 Connecticut Ave #200, Washington DC
Newsweek: 251 W 57th St, New York City, New York
The New York Times: 1 City Hall, New York City, New York
Pew Research Center: 1615 L St NW, #700, Washington DC
Quinnipiac: 275 Mount Carmel Ave., Hamden Connecticut
Rasmussen: 625 Cookman, #2, Asbury Park, New Jersey
Reuters: 3 Times Square, New York City, New York
Survey: USA 15 Bloomfield Ave., Verona New Jersey
TIPP: 690 Kinderkamack Rd, Oradell, New Jersey
Washington Post: 1150 15th St NW, Washington DC
Zogby: 901 Broad St, Utica, New York

Think about the people making the decision on who to poll and how to word questions. They are all living in an environment where calling someone a "conservative" is a dirty insult. How does that effect their day to day decisions about polling?