Stat 231--- CS 276A

Pattern Recognition and Machine Learning

MW 9:30-10:45 pm Fall 2008, PAB 2434.
 
www.stat.ucla.edu/~yuille/Courses/UCLA/Stat_231/Stat_231.html.
 
Office Hours: Tuesdays 2.00-3.00 pm (8967 Math/Science).
EXCEPT: 7& 14/October.
Or by appointment: email Yuille@stat.ucla.edu
 

---

Course Description

This course introduces the basic concepts, theories, and algorithms for pattern recognition and machine learning. These are used in computer vision, image processing, speech recognition, data mining, statistics, and bioinformatics. Topics include: Bayesian decision theory, parametric and non-parametric learning, clustering, component analysis, support vector machines, and boosting..

Prerequisites

• Math 33A Linear Algebra and Its Applications, Matrix Analysis
• Stat 100B Intro to Mathematical Statistics,
• CS 180 Intro to Algorithms and Complexity.

The appendix in Duda, Hart, and Stork introduces much of the mathematics required.

Textbook

• R. Duda, P. Hart, D. Stork. "Pattern Classification", second edition, 2000. [Required]
• T. Hastie, R. Tibshurani, and J.H. Friedman, "The Elements of Statistical Learning: Data Mining, Inference, and Prediction", Spinger Series in Statistics, 2001. [Reference]
• E. Alpaydin " Introduction to Machine Learning", MIT. Press, 2004. [Reference]
• C. Bishop. “Pattern Recognition and Machine Learning”. Springer. 2006.

Instructors

• Prof. Alan Yuille, yuille@stat.ucla.edu, 310-267-5383, office Math Sciences 8967.

Grading Plan: 4 units, letter grades

2 Homework Assignments: Homework 1: Stat231-Fall08\home1-08.pdf Answer 1: Stat231-Fall08\Answer-Home1.pdf Homework 2: Stat231-Fall08\home2-08.pdf	30%
Two computer projects: 1. Project I. Stat231-Fall08\project1-fall08.pdf 2. Project II. Due Monday 15/December: Stat231-Fall08\Project II.doc 3. Stat231-Fall08\face.zip 4. Stat231-Fall08\non_face.zip	20% 20%
Final Exam	30%

Tentative Schedule. All readings are from Duda, Hart and Stork. (May be replaced by readings from Bishop).

Lecture	Date	Topics	Reading Materials	Handouts
1	9-20	Introduction to Pattern Recognition: Issues, Applications, and Examples.	Ch 1	Stat231-Fall08\Lecture1.pdf
2	10-01	Bayesian Decision Theory. I. Basic Concepts.	Ch 2.1-2.6	Stat231-Fall08\Lecture2.pdf
3	10-06	Bayesian Decision Theory. II. Bayes Error, Empirical Error, ROC curves.	Ch 2.1-2.6	Stat231-Fall08\Lecture3.pdf
4	10-08	Learning Probability Distributions. I. Maximum Likelihood Estimation	Ch 3.1-3.6	Stat231-Fall08\lecture4.pdf
5	10-13	Learning Probability Distributions. II. Exponential Distributions and Sufficient Statistics.	Ch 3.1-3.6	Stat231-Fall08\lecture5.pdf Stat231-Fall08\Pietra.pdf
6	10-15	Dimension Reduction. I: Principal Component Analysis.	Ch 3.8.1 Ch 10.13.1	Stat231-Fall08\lecture6.pdf
7	10-20	Dimension Reduction II: Fisher's Linear Discriminant.	Ch 3.8.2 Ch 10.14	Stat231-Fall08\lecture7.pdf
8	10-22	Non-Parametric Learning. I Parzen windows.	Ch 4.1-4.5	Stat231-Fall08\lecture8.pdf
9	10-27	Non-parametric Learning. II K-nearest neighbour classifier.	Ch 4.6	Stat231-Fall08\lecture9.pdf
10	10-29	Data Clustering: K-means clustering, EM algorithm.	Ch 10.1-10.4	Stat231-Fall08\lecture10.pdf
11	11-03	Decision Trees: Design principles and examples.	Ch 8.1-8.4	Stat231-Fall08\lecture11.pdf
12	11-05	Learning Theory: Generalization and Memorization. PAC Theory.	Ch 5.1-5.5	Stat231-Fall08\lecture12.pdf
13	11-10	Linear Separability and Perceptrons.	Ch 5.11-5.12	Stat231-Fall08\lecture13.pdf
14	11-12	Margins and Support Vectors. Linear Non-Separability.	Ch 5.11-5.12	Stat231-Fall08\lecture14.pdf
15	11-17	Kernel Methods. I. The Kernel Trick.	Ch 9.6 & course notes	Stat231-Fall08\lecture15.pdf
16	11-19	Kernel Methods. II. Non-separability with Kernels.		Stat231-Fall08\lecture16.pdf
17	11-24	Boosting I: Basic AdaBoost.		Stat231-Fall08\lecture17.pdf Stat231-Fall08\viola01rapid.pdf Stat231-Fall08\lecture18.pdf
18	11-26	Boosting II: Probabilistic AdaBoost.		Stat231-Fall08\Extra.1.pdf
19	12-01	Boosting III: Generalized AdaBoost.		Stat231-Fall08\Extra.2.pdf
20	12-03	General Review
21	00-00	Final exam	*