17 Experiments
So far, we have primarily discussed observational studies – studies in which conclusions would be drawn from observations of a sample or the population. In some cases these observations might be unsolicited, such as studying the percentage of cars that turn right at a red light even when there is a “no turn on red” sign. In other cases the observations are solicited, like in a survey or a poll.
In contrast, it is common to use experiments when exploring how subjects react to an outside influence. In an experiment, some kind of treatment is applied to the subjects and the results are measured and recorded.
Observational studies and experiments
An observational study is a study based on observations or measurements
An experiment is a study in which the effects of a treatment are measured
Here are some examples of experiments:
Example 20
a. A pharmaceutical company tests a new medicine for treating Alzheimer’s disease by administering the drug to 50 elderly patients with recent diagnoses. The treatment here is the new drug.
b. A gym tests out a new weight loss program by enlisting 30 volunteers to try out the program. The treatment here is the new program.
c. You test a new kitchen cleaner by buying a bottle and cleaning your kitchen. The new cleaner is the treatment.
d. A psychology researcher explores the effect of music on temperament by measuring people’s temperament while listening to different types of music. The music is the treatment.
Try it Now 6
Is each scenario describing an observational study or an experiment?
a. The weights of 30 randomly selected people are measured
b. Subjects are asked to do 20 jumping jacks, and then their heart rates are measured
c. Twenty coffee drinkers and twenty tea drinkers are given a concentration test
When conducting experiments, it is essential to isolate the treatment being tested.
Example 21
Suppose a middle school (junior high) finds that their students are not scoring well on the state’s standardized math test. They decide to run an experiment to see if an alternate curriculum would improve scores. To run the test, they hire a math specialist to come in and teach a class using the new curriculum. To their delight, they see an improvement in test scores.
The difficulty with this scenario is that it is not clear whether the curriculum is responsible for the improvement, or whether the improvement is due to a math specialist teaching the class. This is called confounding – when it is not clear which factor or factors caused the observed effect. Confounding is the downfall of many experiments, though sometimes it is hidden.
Confounding
Confounding occurs when there are two potential variables that could have caused the outcome and it is not possible to determine which actually caused the result.
Example 22
A drug company study about a weight loss pill might report that people lost an average of 8 pounds while using their new drug. However, in the fine print you find a statement saying that