Image Representation by Active Curves

Representation & Computation

We propose a hierarchical composition or grouping of the active Gabor elements in to simple geometric shapes, which are curves and corners, which help us find sparse representation of natural images. We call these geometric shapes as active templates, because they are frequently seen geometric prototypes in image, and are designed to be deformable so that they can be adapted to specific shapes in images. Image below is an example that an active curve template can be deformed according to different poses of deers.
Because of the compositional relationship, a very efficient algorithm can be designed to compute all the possible active template hypothesis in an image. template hypothesis scores are computed recursively by combining best hypothesis scores of component templates, and in parallel for templates of same shape type but placed at different positions in an image. The combination is done by sum operations, and the best score searching operation is done by max operations. Results for these operations are saved as sum (S) and max (M) maps as shown below for an input image.
As we can see from example above, at corner template level (S3 maps), high template responses are very sparse and highly clustered on real image corners. It will be much safer to make decisions at this level, since evidence for template hypothesis are collected at a much larger image neighborhood. More details about the representation and computation can be seen in the paper.

More Results on Natural Images

Click the "i" button for more information about each image.

Sample Detection Results on ETHZ Data:

Blue box is the ground-truth, green box is reported box. Score of each reported window is shown on its top-left corner. For each window, deformed template is also imposed. On each image, we report 5 windows for each category. Correctness of each window is evaluated according to the PASCAL criteria.

Code and Data

Here is the link (48MB) . More comments in the code are on the way, and the most recent update can always be checked out by svn from the following repository:

svn checkout http://image-representation-by-active-curves.googlecode.com/svn/trunk/ image-representation-by-active-curves-read-only

The ETHZ dataset is included, which can also be downloaded from this link hosted by ETHZ. All the other images used in paper are in folder "data". Code is placed in folder "code".

• To run experiment in Section 7.2:

  Go to folder "code/singleImage/", and run "main.m" in MATLAB.

• To run the adaBoost experiment in Section 7.3:

  Go to folder "code/adaBoost/", and run "main.m" in MATLAB.

• For the object detection experiment in Section 7.3:

  Download the original ETHZ dataset (29MB).

  Unpack the five object folders to "data/ETHZ/"

  Go to folder "code/ETHZ/", and run "main.m" in MATLAB.

A C++ language compiler supported by MATLAB is required to compile and run the code. Read text below for more details.

Running Environments and Miscellaneous details of code

For All Experiments

• All the results are saved under directory "results". This folder will be created when the code is run.
• Algorithms are implemented in Matlab with Mex-C routines, so that a C compiler is required.
• In computing M1, S2 maps, SIMD (Single Instruction Multiple Data) intrinsics are used.
• Code for all experiments is tested under Ubuntu 10.04 with MATLAB R2010a and GCC 4.4.3.
• The single image experiment is also tested under Windows 7 with MATLAB 2009b and VC2005 or later.

Single Image Experiment

• Parameters are listed in the "default_param.m" file, in folder "code/singleImage".
• This experiment can NOT be run in Ubuntu terminal mode, because only in desktop mode can MATLAB draw the corners and points on result images.
• Results are saved in "results/singleImage/".
• To run the edgelink code, go to folder "code/edgelink/" and run "main.m". Results are also saved in "results/singleImage/".

The AdaBoost Experiment

• Parameters are listed in the "default_param.m" file, in folder "code/adaBoost".
• To save time on multiple runs, computed features for each image are stored in folder "data/cache/adaBoost".
• For each image, about 9 million active templates features are proposed, so we use MATLAB parallel toolbox to expedite computing of the weak classifiers.
• We recommend running this experiment on a machine with at least 4 core CPU and 8 GB memory. With recommended setting, a single run on active template features would cost about 2 hours.
• Results are saved in "results/adaBoost/".

The Object Detection Experiment

• Parameter files are "categoryName_param.m", located in folder "code/ETHZ". For example, "Applelogos_param.m" is the parameter file for category applelogos.
• Results are saved in "results/ETHZ/". Results contain learned templates, proposed window on each testing image, and the performance curve shown in paper.
• After running "main.m", cropped positive and negative samples are stored in "data/cache/EHTZ"
• Intermediate results such as score maps are stored in "data/cache/EHTZ".
• Learning templates cost about 20-30 minutes for each category. Testing costs about 8 hours for each category. On a multi-core machine, you can run several matlab threads simultaneously, and each detects a different category. In this way, the total time cost for detecting five categories is still about 8 hours.

Acknowledgement for Using Code and Data from Other Projects

• Code for evaluating detection performance curve is from vision lab of ETHZ. Specifically, it is from this package.
• Code for l-2 regularized logistic regression is adapted from liblinear. We changed the corresponding code so that bias term is not regularized.
• Non-maximum suppression code for reporting object detection window and some auxiliary code segments are from part based object detecting project by Pedro F. Felzenszwalb et al.
• Part of the code is from active basis project.
• The leaves dataset used is from vision lab of Caltech.

Acknowledgment

This project is supported by NSF DMS 1007889 and NSF IIS 1018751.