Stat 231: Pattern Recognition and Machine Learning

Stat 231--- CS 276A

Pattern Recognition and Machine Learning

MW 9:30-10:50pm Fall 2007,     Math Sci. Bldg 5203

www.stat.ucla.edu/~sczhu/Courses/UCLA/Stat_231/Stat_231.html

Course Description

This course introduces fundamental concepts, theories, and algorithms for pattern recognition and machine learning
which are used in computer vision, image processing, speech recognition, data mining, statistics, and bioinformatics.
Topics include: Bayesian decision theory, parametric and non-parametric learning, Clustering, Model complexity,
Component analysis, Boosting techniques, Support vector machine, and various bound analysis.

Prerequisites

Math 33A Linear Algebra and Its Applications, Matrix Analysis
Stat 100B Intro to Mathematical Statistics,
CS 180 Intro to Algorithms and Complexity.

Textbook

R. Duda, P. Hart, D. Stork, "Pattern Classification", second edition, 2000. [Required]
C.M. Bishop, "Pattern Recognition and Machine Learning", Springer, 2006 [Reference]
T. Hastie, R. Tibshurani, and J.H. Friedman, "The Elements of Statistical Learning: Data Mining, Inference, and Prediction", Spinger Series in Statistics, 2001. [Reference]
N. Cristianini and J. Shawe-Taylor, "An Introduction to Support Vector Machines", Cambridge Univ. Press, 2000. [Reference]

Instructors

Prof. Song-Chun Zhu, sczhu@stat.ucla.edu, 310-206-8693, office BH 9404. Office Hours: Tuesday 4:30pm-5:30pm

Grading Plan: 4 units, letter grades

3 Homework Assignments	20%
Two experiments (projects): Lab schedule at 9413 Boelter Hall 1. Human face modeling by PCA with a second PCA for morphing 2. Object detection (Human face, cars) detection from images by AdaBoost Some more detailed description of project II is here.	15% 15%
Middle-Term Exam	20%
Final Exam	30%

Tentative Schedule for 2007


Introduction to Pattern Recognition
[Problems, applications, examples]
Bayesian Decision Theory I
[Bayes rule, discriminant functions]
Bayesian Decision Theory II
[loss functions and Bayesian error analysis]
HW1.pdf
Lect4-5.pdf

FisherFace.pdf
 
Component  Analysis and Dimension Reduction III:
[Fisher Linear Discriminant,   Multi-dimensional scaling (MDS)]
Lect7.pdf
 LLE paper 
Parametric Learning
[    Maximum Likelihood Estimation (MLE)
         Sufficient Statistics and Maximum entropy  ]

Non-parametric Learning II:
[K-nn classifer and Error analysis]
 Data Clustering I:   
[K-mean clustering,  EM]
 
Veteran Day, Holiday
Boosting Techniques I:
[ Adaboost ]
Boosting Techniques II:
[ Example on face detection, project II]
Boosting Techniques III: 
 [Probabilistic analysis, logit boost]
Non-metric method I:
 Design and classification trees: principle and example

Non-metric method II:
Syntactic pattern recognition
Support vector machine I: 
 Kernel-induced feature space



Lecture	Date	Topics	Reading Materials	Handouts
1	10-01	Introduction to Pattern Recognition [Problems, applications, examples]	Ch 1	syllabus.pdf Lect1.pdf
2	10-03	Bayesian Decision Theory I [Bayes rule, discriminant functions]	Ch 2.1-2.6	Lect2.pdf
3	10-08	Bayesian Decision Theory II [loss functions and Bayesian error analysis]	Ch 2.1-2.6	Lect3.pdf
4	10-10	Component Analysis and Dimension Reduction I: [principal component analysis (PCA), face modeling]	Ch 3.8.1, Ch 10.13.1	HW1.pdf Lect4-5.pdf FisherFace.pdf
5	10-15	Component Analysis and Dimension Reduction II: [Explanation of Project 1: code and data format]	Project 1	Assignment example
6	10-17	Component Analysis and Dimension Reduction III: [Fisher Linear Discriminant, Multi-dimensional scaling (MDS)]	Ch 3.8.2, Ch10.14	Lect6.pdf
7	10-22	Component Analysis and Dimension Reduction IV: Local Linear Embedding (LLE), Intrinsic dimensions of data)	paper	Lect7.pdf LLE paper
8	10-24	Parametric Learning [ Maximum Likelihood Estimation (MLE) Sufficient Statistics and Maximum entropy ]	Ch 3.1-3.6	Lect8.pdf
9	10-29	Non-parametric Learning Ing I [Parzen window]	Ch 4.1-4.5	HW2 Lect9.pdf
10	10-31	Non-parametric Learning II: [K-nn classifer and Error analysis]	Ch 4.6, handout	Lect10.pdf
11	11-05	Data Clustering I: [K-mean clustering, EM]	Ch 101-10.4	Lect11.pdf
12	11-07	Data Clustering II: [EM, mean-shift]	Handout on a mean-shift paper
13	11-12	Veteran Day, Holiday
14	11-14	Boosting Techniques I: [ Adaboost ]	Ch 9.5	Lect12-14.pdf
15	11-19	Boosting Techniques II: [ Example on face detection, project II]	Tutorial paper on Adaboost Handout on face detection paper
16	11-21	Boosting Techniques III: [Probabilistic analysis, logit boost]
17	11-26	Non-metric method I: Design and classification trees: principle and example	Ch 8.1-8.3	Lect15.pdf
18	11-28	Non-metric method II: Syntactic pattern recognition	Ch 8.5-8.8	Lect16.pdf
19	12-03	Support vector machine I: Kernel-induced feature space	Tutorial paper
20	12-05	Support vector machine II: Support vector classifier	Ch 5.11
		Final exam December 11, 3-5:00 PM	*