1. This class has been about 4 basic ideas--and teaching you the skills to begin to apply these ideas

1. The world is composed of what we see and what we expect--the real and the hypothetical

2. All things exist within distributions, some of which have handy properties

3. We can move between observations and expectations, contrasting and comparing, borrowing facts about one and applying it to another, using logic, in order to reduce our uncertainty about the world

4. Research is conducted within a never ending loop of decision making

2. We have divided the world into what we observe and we cannot observe but expect is so.

1. We learned that things that we observe are sets of observations, or sample data.

1. Sometimes we have only one variable

2. Sometimes we have more than one and are concerned about their joint distribution

2. We have further divided our observations into being comprised of three logical components though the precise percentage assigned to any of the three parts is not always knowable. Anything we observe is part truth, possibly part bias (or due to other factors than what we are interested in), and part chance variation. No matter what we do, our observations are always part truth, part bias, and part random variation

3. When it comes to things we cannot see, our expectations about we would see, if we could, can be quantified if we can define our population, its elements, and how we sample from it. Indeed, in the first part of the course, you learned how to generate via probability theory expected distributions from sampling from a population. So if I asked you to flip a fair coin 50 times, you could tell me (without even doing it) how many heads on average you would obtain, and how much on average you might expect your sample to vary from that.

3. The world is filled with all sorts of distributions and these can be summarized in terms of their central points and their spread or variance. In this class, we have learned about three types of distributions:

3. We also learned to use these expectations to create estimates of the "chance error" part of the equation.

1. Here we learned about the law of large numbers which states that in the long run, over repeated trials, random fluctuations occur but that the center of the distribution comes to center itself on the population parameter. That large negative chance error is balanced out by large positive chance error.

2. So from this, we were able to calculate both the expected center value of a random variable and also it standard error (or size of chance variation). Both of these became important later on. The expected value as a point of comparison for our observed center in our sample and the standard error as a way of weighting the size of the difference between observed and expected to the amount of chance error we thought we might have in the situation.

3. Test statistics: The probability density distributions of the normal curve, the z-curve (which is the normal curve), the t-distribution (closely similar to the normal curve), and the chi-square distribution.

1. These too have centers and average spreads--but the most important thing is that the area under the curves sums to 100% and so a value on these distributions gives information about the probability of obtaining this score or larger (or this score or smaller).

2. The other interesting thing about all of these test statistics is that their equations are very similar. Each one is the order of saying divide the difference between what we observe in our data and what we expect to observe by the size of chance error that we expect to observe.

3. All of these test statistics generate scores that are tied to probability density or percentiles. So obtaining a particular value can be directly translated into a statement about the percentage of the distribution that is bigger or smaller than that score.

1. From this we could translate individual observed scores into percentiles. Z-scores which make use of the normal distribution weight the deviation from average of a score by the average deviation within the distribution. 2 units out to the right and we know that 97.5% of the sample observations are to the left.

2. We could also make statements about how likely the deviation of what we observed from what we expected was due to chance alone in our statistical tests of the null hypothesis.

4. We can use what we know about theoretical distributions to reduce our uncertainty about what we observe.

1. Our observations are always imprecise and comprised of 3 components. By tackling each of the three components in turn we can assume that we have minimized one (bias) so that it is zero through good sampling and measurement, estimate the size of the other (chance), and so indirectly develop an estimate ot the size of the component we are really interested in (true score). How do we do this?

1. Getting bias to be zero

1. We found that there are many forms of bias. Some can arise from how we select our sample. And some can arise from how we measure the elements we have decided are to be in our sample.

2. Selection bias occurs when when the sample of observations is not drawn from the population of all possible events in a way that is consistent with our model.

1. Example: if we believe that a coin is fair (and it is not) and we toss it 100 times, getting heads each time, there is nothing wrong with the coin--our problem is in our model. The model states that the probability of flipping the coin and getting heads is .5. If instead we had a model, correct this time, that the probability of getting heads is 1, then we could perfectly predict the behavior of the coin.

2. Simple random sampling is sampling where every element in a population has an equal chance of ending up in our sample.

1. This means that on average, our sample is representative of our population--we can use our sample statistics to estimate our population parameters.

1. Even then, we learned that our sample statistics vary by chance from our population parameters and so we developed confidence intervals to express our degree of certainty that the population parameter was near our sample statistic.

2. 95% CI--an interval around the mean of the sample. We can state that 95% of the time, we have created an interval that includes the population parameter

2. Although SRS is the ideal, it requires that we have precisely defined our population (enumerated every element) and that we are certain our methods of selection have a known selection probability. This is rarely actually true--but most statistics used today assume this. This has been emphasized not because we want you to walk away saying, "Statistics is hogwash!" but rather because we want you to become an informed consumer of statistics, to ask questions such as, does this sample reflect the population?, how important are the differences from the ideal? Sometimes it’s important; other times through thinking carefully about the issues we can reassure ourselves that the violation is not enough to torpedo our faith in the results.

3. Finally, sampling always introduces concerns about bias.

1. Convenience sampling is sampling without defining the relationship between the sample and the population

2. Survey sampling uses a variety of methods to draw a sample with known relationship to the population--still this is not always done well.

3. Bias can also occur after we have selected our sample.

1. Randomly assigning subjects to treatment conditions is a means of controlling bias. Random assignment means that each subject has an equal probability of being assigned to a condition. The effect of random assignment is to randomly assign random differences within subjects to both conditions equally

2. We also learned that applying treatments to one group (the experimental group) and having an identical group that did not receive the treatment (control group) was another method of achieving control of bias.

3. Observational studies are designs where the probability of subjects being assigned to one condition or another can confound uncontrolled differences in subjects with our causal variables of interest--that is subjects are already assigned to conditions when we begin the study and so may differ in unknown ways.

4. Bias can arise for many other reasons including that generated by researchers and subjects who are not blind to study hypotheses, and the difficulties of measuring constructs accurately.

2. Estimate the role of chance

1. To paraphrase, "Chance happens." It simply is. Things vary and all we can do is estimate the extent of variation that occurs. Why should we care?

2. When we fail to understand chance, then we make mistakes. If we think, using gambler’s fallacy, that 4 coin tosses of heads must be followed by tails and we bet all our money on that, we will lose half the time. In science, it means that sometimes we will see something that appears true, but it is nothing other than chance.

3. And so, in this class you have learned how to calculate variance estimates. Variance within a sample of observations, the square root of which is a standard deviation. Variance within a population or a distribtution of a discrete random variable, the square root of which is a standard error. Variance within repeated samplings from a population (the sampling distribution of the means), the square root of which is the standard error of the means. Variance in a joint distribution around a regression line, the squareroot of which is the standard error of the estimate.

4. Along the way, you learned about correction factors to control for the effects of sampling without replacement

5. The hope is that you have come to develop a sense about the size of average deviations around a value, much as you already had a sense of averages or means.

3. And finally, by assuming that the true value part of the observation (observation = true value + bias + chance) is exactly what we expect under the conditions of chance alone--the situation that occurs when the null hypothesis is correct--effectively setting differences of what we observe from what we expect to zero--then we found that we could attach a probability or percentile to that result.

1. Example: In a z-test contrasting the means of two groups, the null hypothesis is that the means of the two groups are exactly equal. The equation becomes difference due to chance alone divided by a weighting for the amount of average chance error we expect to see. The result is a z-value, which has a percentile attached to it. So we can say that the probability of this result occuring when the mean difference is zero is P = some value.

2. Along the way, you had to learn about degrees of freedom in order to choose the correct t or chi-square probability density function for establishing your P value.

3. And hopefully, you have learned that a small P value means it is very unlikely that the result you observe is due to chance

5. Finally research is conducted within never ending loops.