Stat 232B-CS266B: 
 Statistical Computing and Inference
      in Vision and Image Science
                MW 2-3:20 pm Spring 2013, Math Sci. Bldg 5203
                     
               [syllabus]

Course Description

This graduate level course introduces a broad range of advanced algorithms for statistical inference and learning on graphical structures. These algorithms could be used in vision, pattern recognition, speech, bio-informatics and data mining. Topics include heuristic search and relaxation algorithms, parsing algorithms, stochastic PDEs, advanced Markov chain Monte Carlo methods, hierarchical clustering methods for learning. These algorithms work on two types of underlying representations.

Prerequisites
Reference books
   The lectures will be mainly based on papers and book chapters.
Instructors
Grading Plan: 4 units, letter grades
    The grade will be based on four parts
        2 homework                                       20%
        3 small projects                                  45%            
             Project 1:  Matching pursuit and Least angle regression (15%)
             Project 2:  Inside-outside algorithm for learning and inference on and-or tree (15%)
             Project 3:  C4 for line drawing intepretation with positive and negative edges (15%)
        Final exam                                         35%

Tentative List of Topics
  Chapter 1   Introduction                                                             [ch1_introdction.pdf]
   1. Problems, objectives, and applications 
   2. Basics in algorithm design
         state spaces, operators, constraints, metrics and heuristics.
   3. Algorithms on various graphical structures.
          
 Chapter 2  Classical algorithms: heuristic  search and relaxation          
   1. Heuristic searches in and-or graphs                                      [ch2_part1_search.pdf]   
          Reading materials: (Pearl heuristics Chapter1, Chapter2, Chapter4)  
   2. Relaxation-Labeling for line drawing interpretation                  [ch2_part2_relaxation.pdf]
          Reading materials: (Winston_AI Ch12,  tutorial, Leclerc)

  Chapter 3  Inference in the sparsity land                                                         [ch3.pdf]
   1. Matching pursuit for image coding  (matchingPursuit)
   2. Basis pursuit and Lasso (Least absolute shrinkage and selection operator) (LASSO)
   3. Least angle regression (LARS) and (group lasso)

  Chapter 4  Classical  algorithms: Parsing for grammars                [ch4_chart_parsing.pdf]
   1. Bottom-ip/top-down parsing: CYK, Earley
   2. Best First Chart parsing     (FoM_chart_parsing)
   3. Inside-Outside: inferring and learning SCFG            [ch4_inside-outside.pdf]
Chapter 5 Optimization by differential equations [ch5_PDE_optimization.pdf] 1. SNAKE/Balloon, Curve evolution 2, Geometric heat flow 3, Green's theorem and region competition equations 4. stochastic diffusion and Langevins 5. Jump-diffusion Chapter 6 Inference in the Gibbs fields [ch6_Advanced_MCMC.pdf] 1. Critria for finite state MCMC design 2. Data driven Markov Chain Monte Carlo for segmentation and parsing 3. Swendsen-Wang cut and its variants 4. C4 and Computing multiple distinct solutions [ch6_multi_solutions.pdf] Chapter 7 Image parsing algorithms [ch7_human_pose.pdf] 1. Top-down / bottom-up parsing of attributed grammar 2. Alpha-beta-gamma processes 3. Discussions on scheduling and decision policy 4. Example on human pose parsing 5. Multi-Armed Bandit problem: exploration vs exploitation Chapter 8 Clustering and learning hierarchical models [ch8.pdf] 1. Structure Learning: Bi-clustering and Block pursuit 2. Disconnectivity graphs for neergy landscape 3. Curriculum design (discussion)