STAT 200A: Applied Probability

Logic flow

basic concepts --> one random variable --> two random variables --> three random variables --> many random variables --> transformations --> multivariate normal --> expectation and variance --> limiting theorems ---> conditional expectation and variance

Homework I

1) Work out the rare disease example.
2) Work out the smoking habit example.
3) Derive the exponential distribution and Poisson distribution from Poisson process.
4) Derive the normal distribution from random walk. Check out quincunx and Einstein's random walk
sample HW

Homework II

1) Work out the mixture model example.
2) Work out the rejection sampling example.
3) Work out the bivariate normal example. Assuming that [Y |X ] is a normal distribution with mean X, derive the marginal distribution [Y] and the conditional distribution [X | Y].
4) Work out the Galton's example. Assuming that Z, epsilon1, and epsilon2 follow independent normal distributions, and that epsilon1 and epsilon2 have the same distribution, derive the joint distribution [X, Y], and the conditional distribution [Y | X]. Check out Francis Galton's account of the invention of correlation
by Stigler.
5) Work out the Markov example. Derive the t-step transition matrix of the Markov chain from the one step transition matrix. Check out the seminal paper on Google pagerank algorithm by Larry Page, Sergey Brin, R. Motwani, and T. Winograd.
sample HW

Homework III

1) Work out the "Asia" example. In particular, assuming we are given all the necessary probabilities and conditional probabilities, please calculate p(lung cancer | smoking, dyspnea, positive x-ray),
2) Work out the chain example of belief propagation, assuming that we observe the head and tail of the chain. Check out Pearl's seminal paper on belief network.
3) Work out the forward and backward algorithms for hidden Markov model. Compare these algorithms with belief propagation by identifying pi and lambda messages. Check out Rabina's classical tutorial on HMM.
4) Prove the Markov property of the Ising model.
sample HW

Homework IV

1) Derive the random number generators for exponential and normal.
2) Derive the general formula for the change of density under one-to-one differentiable transformation.
3) Derive the density of multivariate normal by transforming independent normal random variables. Explain the principal component analysis.
4) Derive the conditional distribution from a joint multivariate normal.
5) Work out the calculations in Kalman filtering.
sample HW

Homework V

1) Explain the geometric meaning of correlation.
2) Show that correlation measures the strength of linear relationship.
3) Derive the Markov, Chebechev, Chernoff inequalities.
4) Prove the weak law of large number.
5) Prove the central limit theorem using characteristic function.
sample HW

Homework VI (Final)

Due next Friday 5pm 8971 MS.
1) Edit your class notes (no need to turn in). Write down a detailed list of topics that we have covered according to the logic flow (see above).
2) Explain the asymptotic equipartition property and the concept of entropy.
3) Explain importance sampling.
4) Prove the Adam and Eve formulas and provide geometric interpretation for the Eve formula.
5) Explain Galton's regression.
6) Derive the rate of convergence of Markov chain in terms of maximal correlation.
7) Work out the explanations of the Metropolis algorithm and the Gibbs sampler as discussed in class. Check out the classical paper of Metropolis et al.
sample HW