Statistics M11/Economics 40
Lecture 2
I. CONTROLLED EXPERIMENTS
The researcher ASSIGNS a treatment and observes the outcomes.
Sildenafil handout (example of a classic medical experiment)
1. Principles of Experimentation
The goal is to CONTROL FOR (or eliminate) potential confounding factors.
A. ASSIGN some patients to get the new treatment and others not.
a. The best method is RANDOM assignment, or randomization, to eliminate potential bias.
b. Subjects getting the new treatment form the TREATMENT GROUP; subjects not getting the new treatment form the CONTROL GROUP.
B. Make the experiment BLIND.
a. Give the control group a PLACEBO (a do-nothing treatment).
b. Do not let the patients know whether they are getting the new treatment or the placebo.
c. This eliminates the "PLACEBO EFFECT", when subjects report a change or improvement just because they think they are getting the new treatment.
C. If possible make the experiment DOUBLE BLIND.
a. Do not let the people in contact with the subjects know which treatment they are getting.
b. This eliminates any bias in the administrator's judgments.
c. Sometimes double blinding is impossible.
D. Once the experiment is concluded, COMPARE the treatment and control groups.
E. If possible, REPLICATE the experiment.
Don't be too afraid. Even if a study is not a randomized double-blind controlled experiment, it may not be worthless; it depends on how the study was done and what sources of bias may exist. The upshot of doing all of this is to eliminate all possible forms of contamination.
II. OBSERVATIONAL STUDIES
The researcher collects the data as they currently are.
1.Definition
An OBSERVATIONAL STUDY is an assessment of treatments, policies, or exposures and their associated outcomes. For example, studies on the effects of smoking are always observational.
Observational studies differ from experiments in that the researcher has no control over which subjects are assigned the treatment. He or she collects data as they currently are. These studies are common when questions are asked that cannot be easily answered with an experiment (e.g. smoking, dangerous or harmful behaviors).
Surveys are a typical example of observational studies. For example see some of the surveys collected by the Federal Reserve
2.Remarks
a. Observational studies are the most useful for descriptive statements of fact . Example from the handout on the Consumer Price Index from the WSJ.
b. Observational studies are dangerous to use for cause-and-effect conclusions because of CONFOUNDING FACTORS (outside factors which can interfere with outcomes).
Cannot be certain if it is something an Ivy League education and manager performance or something about the type of people who attend Ivy League schools that determines manager performance.
c. Some confounding factors (e.g. personality traits, gender) can be eliminated by CONTROLLING FOR that factor by dividing the population under study into subgroups based on those factors.
3.Other Examples
a. According to insurance records, people who drive red cars are more likely to get into accidents than people who drive white cars.
(This is an oservational study; as a statement of fact, it is true that drivers of red cars are more likely to be in accidents; but painting a person's red car white will not make them a better driver.)
b. According to medical records, the disease pellagra was associated with flies: where there were flies, there was more likely to be pellagra.
(Again, this is observational; flies are a useful marker of pellagra; but flies are also an indicator of poverty, and pellagra is a disease of malnutrition, not infection.)
III. Comparing Experimental and Observational Studies
1.Randomized, controlled experiments are more expensive and more difficult to do than observational studies.
2.Experiments can also be unrealistic (in an artificial setting) and unethical (smoking studies).
3.Controlled Experiments are better than observational studies in that a researcher can begin to eliminate confounding and pin down cause and effect. Researchers can impose a treatment on randomized subjects. This is not true of observational studies.