Stat 231--- CS 276A

Pattern Recognition and Machine Learning

MW 9:30-10:50pm Fall 2009,     Geology Bldg 4660

www.stat.ucla.edu/~sczhu/Courses/UCLA/Stat_231/Stat_231.html

Course Description

This course introduces fundamental concepts, theories, and algorithms for pattern recognition and machine learning,
which are used in computer vision, speech recognition, data mining, statistics, information retrieval, and bioinformatics.
Topics include: Bayesian decision theory, parametric and non-parametric learning, data clustering, component analysis,
boosting techniques, kernel methods and support vector machine, and fast nearest neighbor indexing and hashing.

Prerequisites

• Math 33A Linear Algebra and Its Applications, Matrix Analysis
• Stat 100B Intro to Mathematical Statistics,
• CS 180 Intro to Algorithms and Complexity.

Textbook

• R. Duda, P. Hart, D. Stork, "Pattern Classification", second edition, 2000. [Required]
• C.M. Bishop, "Pattern Recognition and Machine Learning", Springer, 2006 [Reference]
• T. Hastie, R. Tibshurani, and J.H. Friedman, "The Elements of Statistical Learning: Data Mining, Inference, and Prediction", Spinger Series in Statistics, 2001. [Reference]
• N. Cristianini and J. Shawe-Taylor, "An Introduction to Support Vector Machines", Cambridge Univ. Press, 2000. [Reference]

Instructors

• Prof. Song-Chun Zhu, sczhu@stat.ucla.edu, 310-206-8693, office BH 9404.
  Office Hours: Monday 3-4:30pm
  
• TA Brandon Rothrock, rothrock@cs.ucla.edu, office BH 9414,
  Office hours: Thursday 4-6:00pm

Grading Plan: 4 units, letter grades

Grading policy

• Homework policy:
  Homework must be finished independently. Do not discuss with classmates.
• Project policy:
  You are encouraged to work and discuss in a group, but each person must finish his/her own project. Hand in
  (i) a brief description (hard copy), (ii) your results and plots (hard copy), (iii) you code (e-copy) to the reader.
• Late policy:
  You have a total of three late days for the class, but after using the three late days, no credit will be given for late homework/project

Tentative Schedule for 2009

Introduction to Pattern Recognition
[Problems, applications, examples, and projects]

Bayesian Decision Theory I

[Bayes rule, discriminant functions]

Bayesian Decision Theory II 
[loss functions and Bayesian error analysis

Component Analysis and Dimension Reduction I:
[principal component analysis (PCA)]
[face modeling and AAM model ]

[Explanation of Project 1: code and data format]

HW1


Lect4-5.pdf

Component  Analysis and Dimension Reduction II:
[Fisher Linear Discriminant ]
 Multi-dimensional scaling (MDS)]

FisherFace.pdf


Lect5-6.pdf

Component  Analysis and Dimension Reduction III:
[Local Linear Embedding (LLE), Intrinsic dimension ]

LLE paper 

Boosting Techniques I:
[ Adaboost ]

Lect7-9.pdf

Boosting Techniques II:
[ Example on face detection ]

[ Explanation of project II ]

Boosting Techniques III:
[Probabilistic analysis, logit boost]

Non-metric method I:
 [ tree structured Classification: principle and example ]

Non-metric method II:
Syntactic pattern recognition

Support vector machine I: 
 Kernel-induced feature space

Lect12-15.pdf

(xerox handout)

Support vector machine II: 
[ Support vector classifier ]

[ Explanation of project III ]

Veteran Day, Holiday 

Parametric Learning
[    Maximum Likelihood Estimation (MLE) ]
        [ Sufficient Statistics and Maximum entropy  ]

Non-parametric Learning I
  [ Parzen window ]

Non-parametric Learning II:
[K-nn classifer and Error analysis]

Non-parametric Learning III:
[K-nn fast approximate computing:  KD-tree and Hashing ]

 Data Clustering I:   
[K-mean clustering,  EM]

Data Clustering II: 
[EM,  mean-shift]