STAT 426: Introduction to Theoretical Statistics

Fall 2012 – Section 2
Last update: Tue 11 Dec 2012, 18:13 (EST)

General information

Lectures: Tues – Thurs, 14:30 – 16:00, 120 DENN (Dennison Building)

Instructor: Arash A. Amini
aaamini at umich dot edu
Office Hours:
Tue 4 – 5p in 274 West Hall
Wed 12 – 1p in 274 West Hall
Fri 2 – 2:45p in 274 West Hall

Grader: Yingchuan Wang
yingcw at umich dot edu
Office Hours:
Wed 2:30 – 3:30p in 274 West Hall
Fri 9:30 – 10:30a in 274 West Hall

Textbook: “Mathematical Statistics and Data Analysis” by John Rice, 3rd ed.

Final Exam: Wednesday, Dec 19, 1:30-3:30p.

Midterm Exam: Tuesday, Oct 30 in class.

Announcements

• 10/2: Notes and slides for Lecture 9 posted.
• 10/4: Lecture 10 notes posted.
• 10/4: Homework 5 posted.
• 10/9: Lecture 11 notes and slides posted.
• 10/11: Lecture 12 notes posted.
• 10/11: Homework 6 posted.
• 10/18: Lecture 13 notes posted.
• 10/18: Homework 7 posted.
• 10/23: Lecture 15 posted.
• 11/1: Homework 8 posted.
• 11/5: Lecture 16 and a note on Asymptotic Normality of MLE posted.
• 11/8: Lecture 17 posted.
• 11/8: Homework 9 posted.
• 11/8: Office hours updated.
• 11/13: Lecture 18 posted.
• 11/15: Lecture 19 posted.
• 11/15: Homework 10 posted.
• 11/18: Homework 10 updated.
• 11/20: Sildes for lecture 20 posted.
• 11/27: Sildes for lecture 21 posted.
• 11/29: Sildes for lecture 22 posted. Slides for Lecture 21 updated.
• 11/29: Homework 11 posted.
• 12/4: Sildes for lecture 23 posted.
• 12/6: Homework 12 posted.
• 12/6: Sildes for lectures 23—24 updated.
• 12/11: Sildes for lectures 25 updated.

Syllabus

• Probability review (Chapters 1 – 5 except section 4.6)
  • Basics, probability calculus, counting, conditional probability, Bayes rule, independence
  • Random variables, distributions (joint, conditional, etc.)
  • Expectation
  • Limit theorems (LLN and CLT)
• Distributions derived from normal distribution (Chapter 6)
• Estimation of parameters (Chapter 8)
  • Method of moments
  • Maximum likelihood (and sketch of its asymptotic theory)
  • Elementary decision theory, mean squared error (MSE), bias-variance decomposition
  • Bayesian approach
  • Cramer-Rao bound
  • Sufficiency
• Hypothesis testing (Chapter 9 sections 1 – 5)
  • Neyman-Pearson lemma
  • Duality with confidence intervals
  • Generalized likelihood ratio tests

Required background

Knowledge of basic probability (at the level of STAT 425) and multivariate calculus.

Grading

Homework 25%, Midterm 35%, Final 40%.

Exams are closed book. You can bring one sheet (8×11.5) of handwritten notes with you.

Homework is assigned on Thursdays (posted below), due next week in class (before the lecture). Late homework is not accepted. The least homework grade will be dropped.

You can collaborate on homework assignments, but each of you must write up his/her homework on an individual basis and must indicate with whom they discussed the homework.

Homeworks

• Homework 1
• Homework 2
• Homework 3
• Homework 4
• Homework 5
• Homework 6
• Homework 7
• Homework 8
• Homework 9
• Homework 10
• Homework 11
• Homework 12

Lecture notes

These notes are not for distribution and might have errors.

• Lecture 2 Combinatorics, conditional prob. (hand-written)
• Lecture 3 – 4 Random variables and distributions (slides)
• Brief note on the distribution of polar coordinates (hand-written)
• Lecture 5-6 Expectation (hand-written)
• Lecture 6-7 Variance, Covariance (hand-written)
• Lecture 7-8 Corr. coeff, Cauchy-Schwarz, MSE, MGF (hand-written)
• Brief note on conditional expectation. (hand-written)
• Lecture 8 Limit theorems (hand-written)
• Lecture 9 LLNs and CLT (hand-written) + Slides
• Lecture 10-11 More on conv. and chi-square, t, F, etc. (hand-written)
• Lecture 11-12 sample mean/var., elements of decision theory + Slides
• Lecture 12 Method of moments, consistency
• Lecture 13-14 Maximum likelihood estimation.
• Lecture 15 Consistency of MLE, Fisher information
• A note on the proof of asymptotic normality (AN) of MLE.
• Lecture 16 Fisher info and AN of MLE, confidence intervals
• Lecture 17 Bias-variance decompostion, efficiency, ARE, Cramer-Rao lower bound
• Lecture 18 Sufficiency
• Lecture 19 Exponential families, Rao-Blackwell
• Lecture 20 More on sufficieny, exponential family and Rao-Blackwell (Slides)
• Lecture 21-22 Bayesian inference (Slides)
• Lecture 22 Hypothesis testing (Bayesian formulation) (Slides)
• Lecture 23—24 Hypothesis testing (Neyman—Pearson formulation) (Slides)
• Lecture 25 Generalized likelihood ratio tests and review (Slides)