Stat 232B-CS266B: Statistical Computing and Inference in Vision and Image Science
MW 2-3:20 pm Spring 2013, Math Sci. Bldg 5203 [syllabus]
This graduate level course introduces a broad range of advanced algorithms for statistical inference and learning on graphical structures. These algorithms could be used in vision, pattern recognition, speech, bio-informatics and data mining. Topics include heuristic search and relaxation algorithms, parsing algorithms, stochastic PDEs, advanced Markov chain Monte Carlo methods, hierarchical clustering methods for learning. These algorithms work on two types of underlying representations.
Reference books The lectures will be mainly based on papers and book chapters.
Grading Plan: 4 units, letter grades
The grade will be based on four parts 2 homework 20% 3 small projects 45% Project 1: Matching pursuit and Least angle regression (15%) Project 2: Inside-outside algorithm for learning and inference on and-or tree (15%) Project 3: C4 for line drawing intepretation with positive and negative edges (15%) Final exam 35%
Chapter 1 Introduction [ch1_introdction.pdf] 1. Problems, objectives, and applications 2. Basics in algorithm design state spaces, operators, constraints, metrics and heuristics. 3. Algorithms on various graphical structures. Chapter 2 Classical algorithms: heuristic search and relaxation 1. Heuristic searches in and-or graphs [ch2_part1_search.pdf] Reading materials: (Pearl heuristics Chapter1, Chapter2, Chapter4) 2. Relaxation-Labeling for line drawing interpretation [ch2_part2_relaxation.pdf] Reading materials: (Winston_AI Ch12, tutorial, Leclerc) Chapter 3 Inference in the sparsity land [ch3.pdf] 1. Matching pursuit for image coding (matchingPursuit) 2. Basis pursuit and Lasso (Least absolute shrinkage and selection operator) (LASSO) 3. Least angle regression (LARS) and (group lasso) Chapter 4 Classical algorithms: Parsing for grammars [ch4_chart_parsing.pdf] 1. Bottom-ip/top-down parsing: CYK, Earley 2. Best First Chart parsing (FoM_chart_parsing) 3. Inside-Outside: inferring and learning SCFG [ch4_inside-outside.pdf]
Chapter 5 Optimization by differential equations [ch5_PDE_optimization.pdf] 1. SNAKE/Balloon, Curve evolution 2, Geometric heat flow 3, Green's theorem and region competition equations 4. stochastic diffusion and Langevins 5. Jump-diffusion Chapter 6 Inference in the Gibbs fields [ch6_Advanced_MCMC.pdf] 1. Critria for finite state MCMC design 2. Data driven Markov Chain Monte Carlo for segmentation and parsing 3. Swendsen-Wang cut and its variants 4. C4 and Computing multiple distinct solutions [ch6_multi_solutions.pdf] Chapter 7 Image parsing algorithms [ch7_human_pose.pdf] 1. Top-down / bottom-up parsing of attributed grammar 2. Alpha-beta-gamma processes 3. Discussions on scheduling and decision policy 4. Example on human pose parsing 5. Multi-Armed Bandit problem: exploration vs exploitation Chapter 8 Clustering and learning hierarchical models [ch8.pdf] 1. Structure Learning: Bi-clustering and Block pursuit 2. Disconnectivity graphs for neergy landscape 3. Curriculum design (discussion)