A pharmaceutical company thinks that it has a drug that will lower blood pressure. They do a small pilot study on 10 people, and, by flipping a coin, assign 6 people to the "treatment" group and 4 people to the "placebo" group. The treatment is, of course, an injection of the drug. The placebo is an injection of something harmless and ineffective. Systolic blood pressure of everyone is measured before and after the injections. Here are the changes (blood pressure after injection minus blood pressure before; a negative sign means improvement.):

Treatment: -10,-9,-5,-3 ,0,2

Control: -4,-4,2,6

a) Using whatever descriptive statistics you think appropriate and/or helpful, what evidence is there either for or against the pharmaceutical company's claim? (You can use graphics, if you'd like.)
 

There are many choices here to support the drug company's claim.  Here are some examples:
The Treatment has 4 out of six improvements, (66%), the Control only 2 out of four (50%).
The average change in the treatment is -4.16 points, as compared to 0.0 in the Control
The median treatment change is -4 and in the control group is -1.

You might also draw a dot plot that shows that the treatment scores are, as a group, below the control scores.

b) Perform a "six step" simulation of a median test to help us decide if the drug was effective:

i) Probability Model:

The Null hypothesis is that you can't really tell the two groups apart, i.e. they all come from the same box.  So our model is a box with tickets whose values are
-10,-9,-5,-3,0,2,-4,-4,2,6

ii) Define a Trial:(continued next page)
A trial consists of pulling out six tickets withOUT replacement, and declaring them the "treatment group."
 
 
 

iii) Define a successful trial

The median of the entire group is -3.5.  In the drug company's study, they found that 3  observations were below the group median.  The more observations in the control group that are below the group median, the stronger the drug company's claim is (because they claim that their drug works better than the placebo, so we'd better see more of their people below the median, percentage wise, than we see placebo people.)  Thus, the drug company will be happy if 3,4,5, or all 6 people fall below the group median.  So a successful trial consists of one in which 3 or more of the six tickets are below -3.5.

Note what the two sides are saying: the null hypothesis side claims that the type of outcomes that make the drug company happy (3,4,5, or 6 people below -3.5) occurs quite often even when the drug has no effect.  (We model "the drug has no effect" by putting all of the tickets in the same box.)
In other words, they're saying that the outcome we witnessed is due to chance, the luck of the draw, and not to the drug.

The drug company, on the other hand, claims that while such an outcome might occur by chance, this happens only very rarely, and so it is more likely due to their drug.
 
 
 

iv) We'll repeat this 100 times.

v) How do you compute the probability of interest?
Divide the number of successful trials by 100.

vi) Make a decision: Well, we can't do this here, but stay tuned....
We'll decide in favor of the drug company if, in fact, this probability is "small".  Let's say that by "small" we mean 5%.

I did this using DataDesk.  (See an earlier handout where I explained how to use DataDesk to do the median test.  You can also use the 5step program, which is a little easier.)

I was unable to get the histogram I made to print in this document.  But to do this yourself, you just make a histogram of the sums of each trial.  (A sum represents the number of tickets that were below the median.)  The resulting histogram tells you how many trials had x "successes".

By opening the menu associated with the histogram window (click and hold on the "arrow" in the upper left corner), you can select the option that says "Compute Bar Counts", and you'll get the number of observations appearing in each bar.  We can use this to count the number of successful trials.  There were 70, so the "pvalue", or probability of success, is 70%.

This is such a large percentage, that it seems that the drug company is wrong.  This sort of outcome can easily occur by chance alone.