|B A R E L Y B A D W E B S I T E|
It is bad to make mistakes in logic, because you can get fooled and make mistakes. Here are a few Web pages explaining the various ways you can get fooled. The more of these you recognize when they arise, the fewer mistakes you will make. Also, the more of these you understand the better you can use them to fool all those people who don't recognize them when they arise.
As I see it you should pretty much not even bother to read or listen to survey results, because unless the survey was done properly -- and most aren't -- those results are almost completely meaningless.
Let me start with an extreme example of a worthless survey, called the "self-selected survey." You've seen hundreds of them, maybe thousands. USA Today is famous for their moronic surveys captured graphically because colorful pictures sell newspapers, and because nonsensical "news" of that sort sells newspapers.
Here's an example of a self-selected survey: Ann Landers asks her readers to weigh in on the question whether animals should be used for testing vaccines, and a week later she reports that a whopping 76% are against it.
Now, I honestly can't tell you the actual ratio of all people who are against it, but, honestly, this survey doesn't tell you either. It doesn't provide any useful information at all, because it's a self-selected survey, i.e., the people who cast a vote got to choose not only how they voted but whether they voted.
On an issue such as animal-testing of drugs, a minority of people have strong feelings against it, and most people don't have any strong feelings either way. Naturally, some of the people who are against animal testing are going to go out of their way to vote, whereas the majority of people who have less strong feelings simply aren't going to bother. If they did bother, they might vote for animal testing of vaccines.
Self-selected surveys violate the primary rule of statistically valid sampling, which is this:
The odds of any member of the population
In this case, the population consists of all people who are capable of understanding the question, not just those few who have a special reason for "voting."
So you can now see why a survey in which the respondents get to select themselves is deeply suspect, and why if the question engenders strong feelings in a minority, it's quite likely way wrong.
Here's another example of a self-selected survey, but before I tell you the results, let me ask you the question. The question, asked of Kansas Citians, was "Have you ever seen a drag show in Kansas City?" Now, my question to you, dear reader, is what you think the results of this survey were?
Do you think as many as 1 in 10 answered Yes? If I had to guess I'd say that might be on the high side, but I don't get out a lot. Do you think as many as 4 in 10 said Yes? I would find that almost incredibly high.
According to the survey, a full 80% of respondents said they'd seen a drag show in Kansas City.
I don't care what you think about the popularity of drags shows in Kansas City or anywhere else, the figure of 80% flies in the face of common experience.
But I have no reason to doubt the survey votes were counted right. 80% is undoubtedly the result from this survey, but it's meaningless. It's meaningless partly because it's a self-selected survey, but it's particularly misleading when you consider the nature of the question. People who wish to support drag shows might well be eager to vote Yes, whereas most people who haven't been to one aren't going to be interested in voting at all.
Incidentally, had I voted I would have voted Yes. In roughly 1990 a friend of mine managed a snazzy restaurant, and several of the waiters were gay. I took to going there after work and staying till closing at midnight, at which time KJ and I and the waiters and any other random partiers who were around went out to the gay bars, some of which had drag shows.
I don't know about any restaurants your friends manage, but in this one the bar was basically, for all intents and purposes, open to employees, and full advantage was taken. By the time we hit the first place we were in fine kettle, a kine fettle of fish.
For the 20% of you who don't know, gay bars -- at least the wild and crazy ones I went to -- are fun for the whole adult family, no matter what your predilection is. I like extreme behavior, and that's what I got. Extreme costumes, extreme hairdos, extreme personalities, and lots of fun, lots of gaiety.
Anyway, not only are self-selected surveys meaningless but so are most surveys you hear about in the popular press. For a sampling of a population to result in useful conclusions -- conclusions you can rely on -- a number of rules must be followed, as they have been when you read proper scientific papers. If even one rule is broken even a little bit, then that's the extent to which the conclusions are suspect.
As I said, an essential rule is that the sample of respondents be chosen randomly, and a purely self-selected survey is almost the antithesis of randomness.
Another rule is that the smaller the sample is the less reliable it is. You should want to know how many respondents there were compared to the size of the entire population. If it's 10%, you can rely on it to a much greater degree than if it's only 1%.
|B A R E L Y B A D W E B S I T E|