A university is just a group of buildings gathered around a library. ~Shelby Foote

Tuesday, February 28, 2006

Could Be Plus or Minus Three Percentage Points

That's the disclaimer on that CBS News poll that has Bush's approval rating down to 34%. It's at the bottom of page nine of the raw poll data. What that disclaimer-- and the extensive article on the poll-- does not mention, is that of the 1018 randomly sampled respondents, 409 identified themselves as Democrats, while only 279 identified themselves as Republicans. Nearly 50% more Democrats than Republicans.

That little tidbit is to be found on the very last page of the 18 page report. WAAAAAY at the bottom.

Anybody think that the fact that less than 30% of the sample were Republicans, while over 40% were Democrats, could bias the results at all? Especially given that at the top of the poll, it acknowledges that a large percentage (72%) of Republicans still approve of the President, while very few (9%) of Democrats approve of the President's performance.

+/- 3% my left butt cheek.


Although I think he is a dumb unqualified f-up... a poll of 1018 people out of what, ~300 million adults is it???? has a +/- of about 99% not 3%. Did we learn nothing from exit polls? how is this poll random? you have to either feel very very stongly about being in the poll or have absolutely nothing to do to even participate in a poll like this which immediatly eliminates most emotionally reasonable hard working people. also although I am not a statistics expert I am a scientist and a clearly biased sampling of 0.0003% is so silly it is not even worthy of my time to talk about it.
Unfortunately, Inconceivable, both you and Nick are working Richard Cohen math.

First of all, the reflection of numbers of self-identifying Republicans and Democrats are usually intended for tracking, not to provide what you believe is the right proportion of one to the other. It is entirely possible that fewer people are self-identifying as Republicans because it's just not that attractive to do so these days.

In any case, the methodology employed by these companies is well established, and the margin of error is based not upon how the sample size relates to the entire population, as inconceivable and his calculator worked out up there, but described in that bad old Wikipedia:

A poll with a random sample of 1,000 people has margin of sampling error of 3% for the estimated percentage of the whole population. A 3% margin of error means that 95% of the time the procedure used would give an estimate within 3% of the percentage to be estimated. The margin of error can be reduced by using a larger sample, however if a pollster wishes to reduce the margin of error to 1% they would need a sample of around 10,000 people. In practice pollsters need to balance the cost of a large sample against the reduction in sampling error and a sample size of around 500-1,000 is a typical compromise for political polls. (Note that to get 500 complete responses it may be necessary to make thousands of phone calls.)

It's not a function of having the right number of Republicans answering. As a matter of fact, if the balance of Republicans to Democrats was used to manipulate and weight the results, it would actually invaldiate the results.

As both of you note, you are not statisticians; neither, I admit, am I. But I've been through enough engineering to know that statistics is not an imprecise discipline, and see no need to discount the results of this poll.

Unless, of course, it disrupts your worldview.
actually I am very anti-bush so the poll doesn't offend me as a person at all, it does however offend me greatly as a scientist as do most polls/population studies not because of the conclusions but because people believe in them as factual when they are not... just because the procedure is accepted dose not make it valid. As I said I am not a statistician but I deal with data interpretation every day. I can tell you in general scientists either laugh or ignore poll type studies like this because they are ridiculous in their very nature. when dealing with chemical/biological signaling pathways the standard of deviation would be much much less than any poll type assay because of the human element involved (where there is no worry about trusting the responders or attracting a biased group of responders). a sample of .0003% would not even be looked at as evidence at all much less to say it is accurate to +/-3%. But maybe I am biased as I will also not acknowledge most medical population surveys either as they are based on the same flawed logic and false assumptions as this survey. BTW if "well established" techniques and results were the bastion of truth then real science wouldn't exist and you would still be scratching your response to me on a cave wall. it used to be well established that bleeding people with leeches cured them from blood diseases and drilling holes in your skull let out the evil sprits... I tend to think those practices are more valid than trying to prove a sampling of 1000 people taken at their word is an accurate measure of the true feelings of the entire country.
Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?