Chapter 4: Statistics: Collecting Data
Learning Outcomes
- Identify methods for obtaining a random sample of the intended population of a study
- Identify ineffective ways of obtaining a random sample from a population
- Identify types of sample bias
- Identify the differences between observational study and an experiment
- Identify the treatment in an experiment
- Determine whether an experiment may have been influenced by confounding
As we mentioned previously, the first thing we should do before conducting a survey is to identify the population that we want to study. In this lesson, we will show you examples of how to identify the population in a study, and determine whether or not the study actually represents the intended population. We will discuss different techniques for random sampling that are intended to ensure a population is well represented in a sample.
We will also identify the difference between an observational study and an experiment, and ways experiments can be conducted. By the end of this lesson, we hope that you will also be confident in identifying when an experiment may have been affected by confounding or the placebo effect, and the methods that are employed to avoid them.
Sampling Methods and Bias
Selecting a Population
Suppose we are hired by a politician to determine the amount of support he has among the electorate should he decide to run for another term. What population should we study? Every person in the district? Not every person is eligible to vote, and regardless of how strongly someone likes or dislikes the candidate, they don’t have much to do with him being re-elected if they are not able to vote.
What about eligible voters in the district? That might be better, but if someone is eligible to vote but does not register by the deadline, they won’t have any say in the election either. What about registered voters? Many people are registered but choose not to vote. What about “likely voters?”
This is the criteria used in much political polling, but it is sometimes difficult to define a “likely voter.” Is it someone who voted in the last election? In the last general election? In the last presidential election? Should we consider someone who just turned 18 a “likely voter?” They weren’t eligible to vote in the past, so how do we judge the likelihood that they will vote in the next election?
In November 1998, former professional wrestler Jesse “The Body” Ventura was elected governor of Minnesota. Up until right before the election, most polls showed he had little chance of winning. There were several contributing factors to the polls not reflecting the actual intent of the electorate:
- Ventura was running on a third-party ticket and most polling methods are better suited to a two-candidate race.
- Many respondents to polls may have been embarrassed to tell pollsters that they were planning to vote for a professional wrestler.
- The mere fact that the polls showed Ventura had little chance of winning might have pr