Sure the Poll is Bogus, But Why Exactly?
As a former science teacher, I think a lot of the scientific method for finding answers to most questions. I believe in polls, when done correctly. But I also know there are many details that you have to get right for a poll to be accurate.
The KTHV poll for example, got a lot of things right, but they got one big thing wrong. I suspect (they are hazy on their methodooy) they got 473 out of 509 voter opinions from the just 2nd Congressional district. Naturally that is going to produce a flawed result. Hutchinson is stronger in the third distirct, and Beebe in the 2nd. Their poll showed Asa Hutchinson down 48-38 to AG Mike Beebe. Other polls that used better geographical methodology showed the race within the margin of error.
The new FOX 16 Oakleaf poll did not make the same mistake, they sampled from all four districts equally (which still somewhat under-represents Hutchinson's 3rd district). Sure enough, in Oakleaf's poll Hutchinson had a big lead in the 3rd district, 51-33%. But there is more than one way to mess up a poll. Hutchinson was so far down in the other disticts that overall Beebe was up by 21 points. This is way out of line with the other polls, even the flawed KTHV poll. Again, the other polls show that the race is close. Something stinks here, what is it?
(to find out what stinks, click TUESDAY below and scroll down, or if sent straight here, just scroll down)
The KTHV poll for example, got a lot of things right, but they got one big thing wrong. I suspect (they are hazy on their methodooy) they got 473 out of 509 voter opinions from the just 2nd Congressional district. Naturally that is going to produce a flawed result. Hutchinson is stronger in the third distirct, and Beebe in the 2nd. Their poll showed Asa Hutchinson down 48-38 to AG Mike Beebe. Other polls that used better geographical methodology showed the race within the margin of error.
The new FOX 16 Oakleaf poll did not make the same mistake, they sampled from all four districts equally (which still somewhat under-represents Hutchinson's 3rd district). Sure enough, in Oakleaf's poll Hutchinson had a big lead in the 3rd district, 51-33%. But there is more than one way to mess up a poll. Hutchinson was so far down in the other disticts that overall Beebe was up by 21 points. This is way out of line with the other polls, even the flawed KTHV poll. Again, the other polls show that the race is close. Something stinks here, what is it?
(to find out what stinks, click TUESDAY below and scroll down, or if sent straight here, just scroll down)
posted by Mark Moore (Moderator) at Tuesday, August 22, 2006
2 Comments:
Well, comparing the two polls, we see some striking contrasts. Oakleaf took care to sample each congressional district, he did not mess that up, but there are many ways to mess up a poll. Oakleaf messed this one up in two ways. 1) He oversampled Democrats, and 2) He oversampled older voters, who are the most Democratic voting age group in any poll, and undersampled younger voters who- and this may surprise some- lean far more Republican than any other age group.
Let's talk about the age breakdown first. For example, here is the break down in agre groups from the Oakleaf poll: a whopping 36 percent were over 65 years of age, 20 percent 55-64, 20 percent 45-54, 12 percent 35 to 44, 7 percent 25 to 34, and 4 percent 18 to 24.
The KTHV poll had 24% age 65 and over- so the Oakleaf poll had a 50% greater representation than the KTHV poll in this age group. A large over-representation of older voters would explain a lot of where the mistake was made in this poll. Older voters are set in their ways. In the third district they are set in their ways to vote Republican, and in the rest of the state set in their ways to vote Democrat, but tend to be "conservatives" on social issues. Their massive over-representation could explain why the poll track conservative on social questions but still showed the more liberal Democrat with a big lead- older voters tend to vote Democrat regardless of the facts or whether they agree more with the Republican or the Democrat on the actual issues of the race.
The FOX 16-Oakleaf poll also under-represented the most Republican leaning group- younger voters. They only had 11% from age 18-34 while the KTHV poll had the same group at 18%.
The age mis-sampling is probably what led to the party mis-sampling. Oakleaf had 41% Democrats and 20% percent Republicans in their sample. Now, there are more Democrats than Republicans in this state, but not that many more. Not over 2-1. The KTHV poll has 42% Democrats and 30% Republicans.
In other words in the KTHV poll the Republican sample was a whopping 50% greater proportion of the numbers than it was in the Oakleaf poll. And the Democrats polled in the FOX16-Oakleaf poll were older Democrats who are the least likely type to cross over.
In conclusion, polls are only as good as the methodology used to conduct them. Once upon a time the liberal media could pysche conservatives out with polls that were taken in such a way as to give a biased result which they then reported as fact.
After examining the methodology, I gaurantee you that the FOX 16 poll is fundamentally flawed, and should not be used for anything except as an example of how failure to control for all important variables can produce skewed results in polling.
I still have faith in the scientific method, including polling. Don't lose faith in polling, but lose faith in bad methodology. Don't lose faith in sciences ability to discover truth. Lose faith in the idea that every scientific study was conducted with proper methodology.
The poll also has to be a bit of good news for Green Jim Lendall and Independent Rod Bryan. The rumors that Bryan was in double digits were ridiculous, but considering the poll was done in all districts and that it greatly over-represented older voters I am amazed that it showed them getting five percent of the vote combined.
If KTHV, with their more realistic age samples but less realistic geography (overweight to 2nd District) dares to include them in the next poll I would not be surprised if together they got double digits in that poll.
Post a Comment
<< Home