Grasping at poll samples

Mika Brzesinski and Joe Scarborough in makeup. ()
Tweet Share on Facebook Share on Tumblr Print

Blake Zeff

Follow: feed

Let's start here, at the source of the whole "unskewed" cottage industry: most pollsters do not actually decide how many Democrats or Republicans to include in their sample.

They typically compile their sample based on demographic information (like race, age, and gender) to make sure it reflects accurate proportions of the populace.

Then they ask respondents a number of things, including which party they currently identify with, which in turn says something about how popular the parties currently are.

In a poll that’s not weighted by party, the affiliation of the respondents isn't a choice made by the pollster. It’s a finding.

MORE ON CAPITAL

ADVERTISEMENT

There are in fact some polls that do weight their samples by political party. But these are the exception, not the norm, and the practice is not done by the majority of respected polling organizations we read about.

For example, Quinnipiac weights its polls not by party but by demographic features, using Census data. Similarly, PPP weights for race, age, and gender “to ensure its surveys properly represent the population.” Pew surveys are “statistically adjusted to represent the proper proportion of Americans” with respect to geography, age, race, and use of cell phones versus landline phones. And Gallup’s polls are “weighted to known population parameters on demographic and phone use variables.

So the people taking issue with polls samples may prefer to focus not on the party breakdowns (which the polling outfit does not directly determine), but on the underlying demographic ones, since these are the variables that are being adjusted.

If a poll, for example, oversampled young people, that could compromise its accuracy and sway the results in one direction or another. Or if it undersampled cell phone users, that could have an effect.

But complaints that pollsters are making assumptions about the number of Democrats or Republicans are misdirected; this isn’t the methodology most of them are using.

Other potential sources of error are pollsters' judgments about which voters are more “likely” to vote than others. Gallup, for example, bases its “likely voters” algorithm on seven key questions pertaining to the individual’s voting history and interest.

This is in fact a pollster-generated way of adjusting the sample. But it would be a hard way to skew the findings in the direction of one candidate or another; the “likely voter” algorithm doesn't ask a single question about party identification.

Also: party identification is not the same as party registration. 

Gallup asks the question this way: “In politics, as of today, do you consider yourself a Republican, a Democrat, or an independent?”

As Pew explains, party identification “is an attitude, not a demographic. To put it simply, party identification is one of the aspects of public opinion that our surveys are trying to measure, not something that we know ahead of time.”

This has implications, as illustrated by an exchange on "Morning Joe" on Wednesday, in reaction to a Quinnipiac poll that had just come out.

How could Mitt Romney possibly be leading by double digits among independent voters, host Joe Scarborough asked, but be trailing overall? It doesn’t seem to make sense, panelist Mark Halperin agreed.

The problem here is that the term "independent," as Steve Kornacki put it, "is a lot more fluid than people realize."

Independents are not, for polling purposes, people who aren't registered with a party. They're people who right now tell pollsters they don’t identify with either one, for whatever reason. This may include people who aren’t registered to a party. But it may also include registered Republicans currently disgusted with their party, or Democrats momentarily disenchanted with politics. 

So when Romney leads among independents, but not overall, there may be several factors at play. Obama voters previously less inclined to identify as Democrats may suddenly feel more enthusiastic about the party. If former self-described independents now ID themselves as Democrats, that would increase the size of Democrats in a sample. And it would decrease Obama’s share of the independent vote (since he would have subtracted independents from his vote share).

Another possibility is Romney’s voters are now identifying as independent, rather than as Republicans, for whatever reason. This would boost his standing among independents, and would also reduce the number of self-identifying Republicans in a poll.

In either instance, Romney's lead among "independents" does not necessarily mean he’s leading among centrists, swing voters, the formerly undecided, or, actually, among voters who aren't registered with either party. 

It doesn't matter, in other words. What's left are the topline numbers. Believe them or not.

Blake Zeff is a former presidential campaign aide to Barack Obama and Hillary Clinton and a former aide to Chuck Schumer and Eric Schneiderman.