```
Stat 232B-CS266B:
Statistical Computing and Inference
in Vision and Image Science
```

MW 2-3:20 pm Spring 2013, Math Sci. Bldg 5203 [syllabus]

Course Description

This graduate level course introduces a broad range of advanced algorithms for statistical inference and learning on graphical structures. These algorithms could be used in vision, pattern recognition, speech, bio-informatics and data mining. Topics include heuristic search and relaxation algorithms, parsing algorithms, stochastic PDEs, advanced Markov chain Monte Carlo methods, hierarchical clustering methods for learning. These algorithms work on two types of underlying representations.

- Flat graphs where the nodes/vertices represent states of the same semantic level. E.g, constraint-satisfaction, relaxation-labeling, and Swendsen-Wang Cut, and C4.
- H
*ierarchical graphs*where one level of vertices semantically generate the nodes at the level below. E.g. matching pursuit, heuristic search, language parsing (Earley, Inside-Outside), search on And-Or graphs, Data-Driven Markov Chain Monte Carlo, and compositional inference.

Prerequisites

- Basic statistics, linear algebra, MCMC, Stat232A or equivalence, programming skills (matlab, C++) for a project.
- Knowledge and experience on images will be a plus.

Reference books The lectures will be mainly based on papers and book chapters.

- Mumford and Desolneux,
*Pattern Theory: the stochastic Analysis of Real-World Signal*, 2010. - J. Pearl,
*Heuristics:Intelligent Search Strategies for Computer Problem Solving*, 1984.

Instructors

- Prof. Song-Chun Zhu, sczhu@stat.ucla.edu,
310-206-8693, office BH 9404.
*Office Hours: Monday 3:30pm-5:00pm*

`Grading Plan: 4 units, letter grades`

The grade will be based on four parts 2 homework 20% 3 small projects 45% Project 1: Matching pursuit and Least angle regression (15%) Project 2: Inside-outside algorithm for learning and inference on and-or tree (15%) Project 3: C4 for line drawing intepretation with positive and negative edges (15%) Final exam 35%

Tentative List of Topics

Chapter 1 Introduction [ch1_introdction.pdf] 1. Problems, objectives, and applications 2. Basics in algorithm design state spaces, operators, constraints, metrics and heuristics. 3. Algorithms on various graphical structures. Chapter 2 Classical algorithms: heuristic search and relaxation 1. Heuristic searches in and-or graphs [ch2_part1_search.pdf] Reading materials: (Pearl heuristics Chapter1, Chapter2, Chapter4) 2. Relaxation-Labeling for line drawing interpretation [ch2_part2_relaxation.pdf] Reading materials: (Winston_AI Ch12, tutorial, Leclerc) Chapter 3 Inference in the sparsity land [ch3.pdf] 1. Matching pursuit for image coding (matchingPursuit) 2. Basis pursuit and Lasso (Least absolute shrinkage and selection operator) (LASSO) 3. Least angle regression (LARS) and (group lasso) Chapter 4 Classical algorithms: Parsing for grammars [ch4_chart_parsing.pdf] 1. Bottom-ip/top-down parsing: CYK, Earley 2. Best First Chart parsing (FoM_chart_parsing) 3. Inside-Outside: inferring and learning SCFG [ch4_inside-outside.pdf]

Chapter 5 Optimization by differential equations [ch5_PDE_optimization.pdf] 1. SNAKE/Balloon, Curve evolution 2, Geometric heat flow 3, Green's theorem and region competition equations 4. stochastic diffusion and Langevins 5. Jump-diffusion Chapter 6 Inference in the Gibbs fields [ch6_Advanced_MCMC.pdf] 1. Critria for finite state MCMC design 2. Data driven Markov Chain Monte Carlo for segmentation and parsing 3. Swendsen-Wang cut and its variants 4. C4 and Computing multiple distinct solutions [ch6_multi_solutions.pdf] Chapter 7 Image parsing algorithms [ch7_human_pose.pdf] 1. Top-down / bottom-up parsing of attributed grammar 2. Alpha-beta-gamma processes 3. Discussions on scheduling and decision policy 4. Example on human pose parsing 5. Multi-Armed Bandit problem: exploration vs exploitation Chapter 8 Clustering and learning hierarchical models [ch8.pdf] 1. Structure Learning: Bi-clustering and Block pursuit 2. Disconnectivity graphs for neergy landscape 3. Curriculum design (discussion)