KTHV Polling Methodology Invalidates Results
UPDATE: A representative of SURVEY USA has responded to our questions about their poll methodology. Please read the comments in the thread for their account.
********************************************************
KTHV has released another SURVEY USA poll for state races. For the Republicans, the bad news is they are all behind, and except for Lt. Governor the gap is growing. The good news is that flawed polling methodology invalidates the results of the poll. The poll says something, but as a prediction of the final outcome the results are meaningless.
Their major mistake was to view Arkansas as a pie with two equal halves, "Little Rock area" and "Rest of State". They never define what "Little Rock area" means. Is it just Pulaski County, or is it the entire 2nd congressional district? They don't say, but even if it is the entire 2nd district, that is only 1/4th of the real pie, yet they count it as 1/2.
Arkansas has four congressional districts and it is nuts to make one of them (the 2nd) equal to the other three. In fact, it is not even the largest district. The GOP-leaning 3rd district is the most vote-rich district, but it still is not HALF the state, more like a third. That is why Republicans should not be discouraged by these fraudulent polls. The Republican's strategy is to split rural conservatives from the Democratic fold, a strategy that will not show up much in a poll that overweights the Little Rock area.
Their poll is a good measure of feelings in the Little Rock area, but not the state as a whole. What would be the results of a poll that made the 3rd district (Northwest Arkansas) worth half of the state and the other three congressional districts the other half? Our guess is it would show the Republicans winning every race.
Look, I am willing to accept a poll result that is done with correct methodology, but these pollsters are making mistakes than a good 8th grade science fair project would avoid. I taught 8th grade and I can tell you that I had a lot of kids who would have known better than to oversample one region like that. Why are they doing it? Is it part of a deliberate strategy to undermine the GOP ticket? I dunno, but I know that the GOP should do its own poll, using correct methodology, and release the results.
********************************************************
KTHV has released another SURVEY USA poll for state races. For the Republicans, the bad news is they are all behind, and except for Lt. Governor the gap is growing. The good news is that flawed polling methodology invalidates the results of the poll. The poll says something, but as a prediction of the final outcome the results are meaningless.
Their major mistake was to view Arkansas as a pie with two equal halves, "Little Rock area" and "Rest of State". They never define what "Little Rock area" means. Is it just Pulaski County, or is it the entire 2nd congressional district? They don't say, but even if it is the entire 2nd district, that is only 1/4th of the real pie, yet they count it as 1/2.
Arkansas has four congressional districts and it is nuts to make one of them (the 2nd) equal to the other three. In fact, it is not even the largest district. The GOP-leaning 3rd district is the most vote-rich district, but it still is not HALF the state, more like a third. That is why Republicans should not be discouraged by these fraudulent polls. The Republican's strategy is to split rural conservatives from the Democratic fold, a strategy that will not show up much in a poll that overweights the Little Rock area.
Their poll is a good measure of feelings in the Little Rock area, but not the state as a whole. What would be the results of a poll that made the 3rd district (Northwest Arkansas) worth half of the state and the other three congressional districts the other half? Our guess is it would show the Republicans winning every race.
Look, I am willing to accept a poll result that is done with correct methodology, but these pollsters are making mistakes than a good 8th grade science fair project would avoid. I taught 8th grade and I can tell you that I had a lot of kids who would have known better than to oversample one region like that. Why are they doing it? Is it part of a deliberate strategy to undermine the GOP ticket? I dunno, but I know that the GOP should do its own poll, using correct methodology, and release the results.
posted by Mark Moore (Moderator) at Thursday, August 31, 2006
17 Comments:
Its a conspiracy.
Hey, if there was no such thing as a conspiracy, they would not have laws on the books against it.
The crosstab results show, I kid you not, that the Republicans are winning among ONE ethnic group- HISPANIC VOTERS!
Maybe the legal ones are getting sick of the illegals coming up and causing trouble, giving them a bad name, and taking jobs.
Another thing, the need to mention Jim Lendall and Rod Bryan by NAME to get an accurate poll. Why won't they include half the candidates on the ballot for Governor in the poll? I would think it would be news since Lendall was just part of a big story on ballot access. I would think they would like to measure how much his entry effected the race, but NNOOOOOO. They are not curious enough to even mention his name in the poll.
Finally some sense out of this ludicrous poll! I just KNEW they had to be polling in Little Rock. For a poll to have Asa losing ground, I also knew it had to be false because that just isn't true. And yes, I agree that they need to add Jim and Rod to the mix of gubernatorial candidates. What I find really interesting is that since Jim Holt is ahead in the heavily polled Little Rock area, he should win his race handily. Thanks, Mark!
Did I read the poll right? Doesn't Jim have the narrowest margin between him and his opponent than all the other Republcian candidates? And haven't the other Republican candidates been trying to keep their distance from Holt becasue the major media wants to paint him extreme. Seems like those other candidates might want to get a little closer to Senator Holt. I don't believe the polls either and think Mark did a good job showing that, but I do believe that Senator Holt is probably closer to his opponent than any of the other candidates.
Now with all those Greens running for state-wide office, The Dems should drop about 2% in the polls. But I am sure they will be ignored in the polls.
Holt is likely AHEAD of his opponent in a poll done with correct methodology, and the others are close. They have not been running from Holt, that is just what the Demo-Zette WANTED them to do.
The sad truth is that President Bush is the drag on this ticket, not Holt. The swing voters have soured on him and that is hurting all Republicans somewhat.
The results of any scientific poll can be pre-arranged by fiddling with the methodologies to achieve a desired result. I have an offer for Gilbert Baker and Clint Reed. For a paltry $5,000 I will give them a poll methodology that will show the Republicans winning every race in the KTHV poll. If it shows the Democrats winning even one race, I will refund $4,000. If it shows them winning two of the four, I will refund it all.
If they want me to actually conduct the poll, I will do that too, but it will take a much larger fee. Still, they should not take me up on that because I don't care which organization does the poll, Zogby, Gallup, Survey USA even, if they follow the methodology that I will present then it will show the GOP dominating.
to see the crosstab numbers on this poll, stick this URL in your browser....
http://www.surveyusa.com/client/PollReport.aspx?g=964a450e-9253-4ada-bb0c-2c5106a6f23d
they nailed the primaries pretty well
This is the same method they always use, and they are usually right. Republicans are in trouble.
surveyusa,
I am going to trust but verify. I have asked to see if the News Director of Survey USA does indeed have a blogspot. If your LR area does include 50% of the state then my criticism of your methodology would be unfounded. Since you are here, I would hope that you could answer the question I had abot the methodology of your prior KTHV poll. In that one, you sampled 509 likely voters statewide, but you also polled the 2nd distirct congressional race at the same time. In that one you polled 473 voters. My question is "are 473 of the 509 voters you polled for statewide races the same people that responded to your 2nd district congressional poll conducted simultaneously?"
And 1:40 the primaries are really different. For one thing, the female vote in primaries in homogenous where as in a general you have single women basically married to the state who vote democrat and women with good husbands and children who vote Republican. That is a generalization of course, but the trend holds. If they over sample one group or the other it will skew the poll. That is not a factor in primaries. Also, on the Republican side at least, the numbers from Little Rock were close to the ones in NWA and elsewhere- Holt won pretty much everywhere. That is not the pattern we will see for the general election. Expect the 3rd to go for the Republicans, the 2nd to go for the Democrats, and the north part of the 1st to go Republican and the south part for Democrats. That leaves the 4th as the battleground. If the GOP's campaign to woo rural conservatives works, they can stay close enough in the 4th to win overall, otherwise they will lose.
1:45- I am not sure they used the same method they "always use". I don't think they even used the same method they did in the previous poll, that is why i asked their newsdirector what I did.
And my offer to Mr. Reed and Sen. Baker still stands, though it is made tongue in cheeck. In fact, I will make the same offer for a poll methodology that takes equal shares from all four congressional districts. It will show the Republicans winning at least three of four races or your money back! This will show how easy it is to make a poll say whatever you want it to say.
Thank you for your answer, but perhaps I was unclear in my question. Bear with me as I make the journey of changing my mind on this poll business...
I understand that they were RELEASED on different days, what I would like to know is if they were CONDUCTED at the same time (I have heard that they were. People in the 2nd got calls about the Congressional race and the statewide races.) So I want to know if they were CONDUCTED at the same time and how many of the 473 people interviewed in the 2nd district poll were included in the 509 people in the statewide poll.
The problems is that the media isn't necessarily engaged in a conspiracy of partisan politics, though they obviously skew liberal. They are engaged in a conspiracy of cheapness. They hire pollsters like Earnie Oakleaf who are willing to cut corners in order to do their polls on the cheap. Obviously, if you weight all four congressional districts the same, you're going to get a skewed poll, and every one of them will favor Democrats. Bradley County, for example, simply does not have the same percentage of the vote as Pulaski County or Benton County.
In response to the SurveyUSA guy, you can't use media market numbers as a basis for weighting polling. Media markets are very different things than voting districts. The Little Rock "media market" includes half the state, this is true. But that includes counties like Dallas County, which produced barely a blip of the percentage of voters in all of the recent elections.
Correct methodology would require that you examine the exit polling of the previous elections based on voter turnout of counties. If you're not willing to spend the money to do this, then you should at least base it upon voter turnout of congressional districts.
Basing polling numbers on respondents equally divided among the congressional districts is flawed, and no professional pollster who is concerned about his or her reputation would do this. The percentage of voters in the fourth district who turned out in the last election in the Fourth District is simply far lower than in the second district or third district. That's not opinion, it's just fact. Look it up.
"but they were two separate jobs, using two separate and non-overlapping samples. Respondents who got the 2nd CD poll weren't asked about statewide races, and those who got the statewide poll weren't asked about the 2nd CD"
OK, that is a clear statement. I have a higher opinion of your methods than I did when I wrote the article, assuming I hear on Tuesday that no one is on blogspot trying to impersonate you. The great thing about blogs is that every side can get on and tell their side of the story.
I still think it would be a better poll if you sampled CD's by 04 turnout, and I am especially leery about the 3rd CD being undercounted in the "rest of state" pile, but I am beginning to think your poll is reasonably accurate.
"This is the same method they always use, and they are usually right. Republicans are in trouble."
No we're not. It's just that the sampling size was half Pulaski County and well, Pulaski County is nothing but a bunch of socialist libs. It doesn't bother me if the poll gives Democrats a false sense of security.
Post a Comment
<< Home