STAT 200B: Theoretical Statistics

Logic flow

(1) Euler's astronmy example, Laplace's combination of equations, Legendre's least squares, Gauss-Markov theorem.
(2) Model complexity, testing error versus training error, bias and variance, Stein's estimator, VC dimension.
(3) Discriminative versus generative, Pearson's method of moments, Fisher's maximum likelihood.
(4) EM algorithm, unsupervised learning, mixture model, logisitic regression.
(5) Fisher information, asympotic optimality of MLE, Cramer-Rao bound.
(6) Kullback-Leibler divergence, Akaike information criterion, exponential family model.
(7) Likelihood ratio test; Neymann-Pearson lemma, generalized likelihood ratio test.
(8) Duality between hypothesis testing and confidence inerval.
(9) Bayesian decision theory, empirical Bayes, Gibbs sampler, data augmentation.