Stat 232B-CS266B: 
 Statistical Computing and Inference
      in Vision and Cognition

This graduate level course introduces a broad range of advanced algorithms for statistical inference and learning on hierachical models. More specifically, this course will focus on grammatical models in the form of probabilistic And-Or Graphs, including i) Spatial AOG for object recognition and scene understanding; ii) Attribute-AOG for human pose and attribute inference; iii) Temporal-AOG for event understanding and behavior predictions; iv) Causal-AOG for causal-effects in human-object/scene interactions; and v) the joint STC-AOG for comprehensive scene and event parsing across multi-cameras. the lecture will covers topics on:

• Representations: data structures and models.
• Inference algorithms: parsing algorithms, bottom-up / top-down processes, and scheduling.
  
• Learning algorithms: parametric learning, structure learning, pursuit algorithms.
  

Prerequisites

• Stat232A, programming skills (C++, Python) for a project.
• Knowledge and experience on computer vision.

Reference books

• Stochastic Grammars in Computer Vision, by S.C. Zhu, a draft book (pdf distributed in CCLE).

Instructors

• Prof. Song-Chun Zhu, office BH 9404.Office Hours: Tuesday 3:30pm-5:00pm (on Zoom, notify me ahead of time)
• TA: Luyao Yuan, Project discussion sessions, Wednesday 6-7PM (on Zoom).

Grading Plan: 4 units, letter grades

• Grading will be based on 5 projects below measured by code, quility of results, and reports submitted on CCLE.
• Each project will take 2 weeks including the final exam week, and each person has 3 free late days (work day). After you run out of the late days, late submissions won't be accepted.

Tentative List of Topics    
[draft textbook, lecture notes, reading materals are distributed in CCLE] 

  Chapter 1   Introduction                                                             
   1. Overview of regimes of models from Stat232A
   2. Hierarchical STC-AOG representation and applications
   3. Project design and requirements
          
 Chapter 2  Spatial And-Or Graph            
   1. Terminology: basics of grammars, vocabulary, relations, parse graph, language   
   2. Characteristics of image grammars  
   3. And-Or graph for knowledge representation
   4. Some examples 

  Chapter 3  Learning And-Or Graph
   1. Parametric learning: EM algorithm, pursuit of contextural relations
   2. Structure learning: Block pursuit, AOG fragment pursuit
   3. Structure-parametric learning: Full-AOG and pruning

  Chapter 4  Inference and parsing algorithms 
   1. Traditional parse algorithms: CYK, Earley parser, chart parsing
   2. Inside-Outside: inferring and learning SCFG
   3. Alpha-beta-gamma scheduling
   4. Examples on object parsing             
  
 Chapter 5  Attributed And-Or Graph 
   1. Attribute grammar 
   2. Example I: parsing man-made object and scenes 
   3. Example II: geometric attribute for scene parsing
   4. Example III: appearance attributes for human parsing

  Chapter 6 Temporal And-Or Graph 
   1. Atomic actions 
   2. Representing events by T-AOG
   3. Learning and pursuit T-AOG from videos and demonstrations                         
   4. Event parsing and intent prediction with generalized Earley parser.
    
 Chapter 7 Fluent and Causal-And-Or Graph
   1. Fluents of objects and scenes, and causal relations
   2. Perceptual causality
   3. Pursueing causal relations
   4. Learning the causal-AOG: Pursuit and transfer 
 
 Chapter 8 Joint parsing and integration
   1. Scene centric parsing of object, scene and event
   2. Examples: cross-view parsing of scenes and humans
   3. Advanced topics: explanation and exploration with Logic + AOG + DNN.