iFRAME (inhomogeneous Filters Random Field And Maximum Entropy)

Experiment 4.2: Evaluation of Object Clustering (EM with sparse FRAME v.s. k-means with HOG)

Code and dataset

Experiment setup

To evaluate the clustering quality, we introduce two metrics: conditional purity and conditional entropy. Given the underlying groundtruth category labels X (which is unknown to the algorithm) and the obtained cluster labels Y , the conditional purity is defined as the mean of the maximum category probabilities for (X, Y ),

and the conditional entropy is defined as,

where both p(y) and p(x|y) are estimated on the training set, and we would expect higher purity and lower entropy for a better clustering algorithm.

We compare clustering by sparse FRAME with k-mean with HOG features. The methods are evaluated in terms of conditional purity and conditional entropy. All the results are obtained based on 10 repetitions. Table 1 show a description of the dataset we used. And Table 2 and 3 show the results of two methods. It can be seen that our method performs significantly better than k-mean with HOG feature, especially on the datasets in which different categories are distinguished by texture.

Parameters setting

Method 1: EM with sparse FRAME

General Parameters: nOrient = 16; sizeTemplatex=100; sizeTemplatey=110; GaborScaleList=[1.4, 1, 0.7]; useDoG = false; isLocalNormalize = false; sigsq=10; #sketch=300; locationShiftLimit=5; orientShiftLimit=1; 
HMC Parameters: lambdaLearningRate = 0.1/sqrt(sigsq); epsilon = 0.03; L = 10; nIteraton =20; 12x12 chains 
Clustering Parameters: #EM iteration = 10; isSoftClassification = false.

Method 2: k-mean with HOG

General Parameters: sizeTemplatex=100; sizeTemplatey=110;
HOG Parameters: #HOG windows per bounding box =6x6; #histogram bins=9;

Table 1. Datasets for Clustering

	Name	# Clusters	Examples	# Images
Exp1	dalmatian & weimaraner	2		15/15
Exp2	beef cattle & milk cattle	2		15/15
Exp3	butterflies, dragonflies & bugs	3		15/15/15
Exp4	Cars, motorbikes, scooters & bikes	4		15/15/15/15
Exp5	Human faces, butterflies, & horses	3		30/30/30
Exp6	Horses, pigeons, eagles, swans, & fishes	5		15/15/15/15/15
Exp7	Cats, deers, & wolves	3		15/15/15