Stat 231: Pattern Recognition and Machine Learning

Stat 231--- CS 276A

Pattern Recognition and Machine Learning

MW 12:30-1:45pm Fall 2006,     Math/Science 5147.

www.stat.ucla.edu/~yuille/Courses/UCLA/Stat_231/Stat_231.html.

Office Hours: Tuesdays 2.00-3.00 pm (8967 Math/Science).

EXCEPT: 24/October.

Or by appointment: email Yuille@stat.ucla.edu

Course Description

This course introduces the basic concepts, theories, and algorithms for pattern recognition and machine learning. These are used in computer vision, image processing, speech recognition, data mining, statistics, and bioinformatics. Topics include: Bayesian decision theory, parametric and non-parametric learning, clustering, component analysis, support vector machines, and boosting..

Prerequisites

Math 33A Linear Algebra and Its Applications, Matrix Analysis
Stat 100B Intro to Mathematical Statistics,
CS 180 Intro to Algorithms and Complexity.

The appendix in Duda, Hart, and Stork introduces much of the mathematics required.

Textbook

R. Duda, P. Hart, D. Stork. "Pattern Classification", second edition, 2000. [Required]
T. Hastie, R. Tibshurani, and J.H. Friedman, "The Elements of Statistical Learning: Data Mining, Inference, and Prediction", Spinger Series in Statistics, 2001. [Reference]
N. Cristianini and J. Shawe-Taylor, "An Introduction to Support Vector Machines", Cambridge Univ. Press, 2000. [Reference]
C. Bishop. “Pattern Recognition and Machine Learning”. Springer. 2006.

Instructors

Prof. Alan Yuille, yuille@stat.ucla.edu, 310-267-5383, office Math Sciences 8967.

Grading Plan: 4 units, letter grades

2 Homework Assignments:

Homework 1: Stat231-Fall06\home1-fall06.pdf

Answer 1: Stat231-Fall06\Answer-Home1.pdf

Homework 2: Stat231-Fall06\home22-fall06.pdf

Stat231-Fall06\fall06-ans2.pdf

30%

Two computer projects:

1. Project I. Stat231-Fall06\project1-fall06.pdf

2. Project II. Due Wednesday 6/December:

Stat231-Fall06\project2.zip

Stat231-Fall06\face.zip Stat231-Fall06\non_face.zip

20%

Final Exam

30%

Tentative Schedule. All readings are from Duda, Hart and Stork. (May be replaced by readings from Bishop).


Introduction to Pattern Recognition:
Bayesian Decision Theory. I.
Bayesian Decision Theory. II.
Learning Probability Distributions. I.
Learning Probability Distributions. II.
Dimension Reduction. I:
Dimension Reduction II:
Non-Parametric Learning. I

Data Clustering:   
Decision Trees: 
Stat231-Fall06\lecture12.pdf
Linear Separability and Perceptrons.
Margins and Support Vectors.
Kernel Methods. I.
Kernel Methods. II.
Boosting I:  
 
Boosting III:  



Lecture	Date	Topics	Reading Materials	Handouts
1	10-02	Introduction to Pattern Recognition: Issues, Applications, and Examples.	Ch 1	Stat231-Fall06\Lect1_intro_to_PR.ppt
2	10-04	Bayesian Decision Theory. I. Basic Concepts.	Ch 2.1-2.6	Stat231-Fall06\Lecture2.pdf
3	10-9	Bayesian Decision Theory. II. Bayes Error, Empirical Error, ROC curves.	Ch 2.1-2.6	Stat231-Fall06\Lecture3.pdf
4	10-11	Learning Probability Distributions. I. Maximum Likelihood Estimation	Ch 3.1-3.6	Stat231-Fall06\lecture4.pdf
5	10-16	Learning Probability Distributions. II. Exponential Distributions and Sufficient Statistics.	Ch 3.1-3.6	Stat231-Fall06\lecture5.pdf Stat231-Fall06\Pietra.pdf Stat231-Fall06\TutorialGY.pdf
6	10-18	Dimension Reduction. I: Principal Component Analysis.	Ch 3.8.1 Ch 10.13.1	Stat231-Fall06\lecture6.pdf
7	10-23	Dimension Reduction II: Fisher's Linear Discriminant.	Ch 3.8.2 Ch 10.14	Stat231-Fall06\lecture7.pdf
8	10-25	Non-Parametric Learning. I Parzen windows.	Ch 4.1-4.5	Stat231-Fall06\lecture8.pdf
9	10-30	Non-parametric Learning. II K-nearest neighbour classifier.	Ch 4.6	Stat231-Fall06\lecture9.pdf
10	11-01	Data Clustering: K-means clustering, EM algorithm.	Ch 10.1-10.4	Stat231-Fall06\lecture10.pdf
11	11-06	Decision Trees: Design principles and examples.	Ch 8.1-8.4	Stat231-Fall06\lecture11.pdf
12	11-08	Learning Theory: Generalization and Memorization. PAC Theory.	Ch 5.1-5.5	Stat231-Fall06\lecture12.pdf
13	11-13	Linear Separability and Perceptrons.	Ch 5.11-5.12	Stat231-Fall06\lecture13.pdf
14	11-15	Margins and Support Vectors. Linear Non-Separability.	Ch 5.11-5.12	Stat231-Fall06\strang_nonlinear_optimization.pdf Stat231-Fall06\lecture14.pdf
15	11-20	Kernel Methods. I. The Kernel Trick.	Ch 9.6 & course notes	Stat231-Fall06\lecture15.pdf
16	11-22	Kernel Methods. II. Non-separability with Kernels.		Stat231-Fall06\lecture16.pdf
17	11-27	Boosting I: Basic AdaBoost.		Stat231-Fall06\viola01rapid.pdf
18	11-29	Boosting II: Probabilistic AdaBoost.
19	12-04	Boosting III: Generalized AdaBoost.
20	12-06	General Review
21	12-12	Final exam, 8-11am	*