L1 Regularized Logistic
Regression
1. Methods
Learn basis elements by L1 regularized logistic regression with MAX1
scores after
sigmoid transformation. After
the regression, each feature will get a regression
coefficient. The coefficient describe the size of the contribution of
that feature. A positive regression coefficient means the element
increases the probability of the outcome, while a negative coefficient
means the element decreases the probability of that outcome. In other
words, the learning method divides the learned template to
two groups. After dimension
reduction by deleating the basises with 0 pooled MAX1 scores, the
computation is fast for discriminative methods.
2. Classification Experiments
2.1 Dataset. Number of training negatives 160, testing negatives
160. 5
numbers of positive training examples (5, 10, 20, 40, 80) are tested.
We tried
3 different number of tuning psrameters: C = 1, 10, and 100.
2.2 Results of testing AUC.
L1
regularized Logistic regression performs better when training
positive sample size increases. The tuning parameters didn't affect the
result very much, but when the value was bigger, the regression
performs better.
2.2 Results of number of learning features. After L1
Regularized Logistic
Regression, some coefficients of the features will be set to 0.
Therefore, the
features with non-zero coefficients will be the features
learned/selected by the method. The plot shows the average number of
selected features(includes features with positive or negative
coefficient). Compared with the 80 features learned by active basis, L1
Regularized Logistic
Regression choose more features, and especially when learning with
large training positives.
2.3 Results of number of learning features with positive coefficient.
Although features with positive coefficient(positive features) are
usually less than features with negative coefficient(negative
features), the ratio between each other didn't change much when the
traning positives increase. The features with positive
coefficient(positive features) can be viewed as the sketch of the
object.
2.3 Compare templates.
Number of training positive is 80;
number of elements is 80;
Length of Gabor = 17 pixels; Local normalization or not = 1; Range
of displacement = 3 pixels; Subsample rate = 2 pixel; Image height and
width = 85 by 127 pixels
Active Basis:
Number
of sketches in the above active basis templates are, respectively, 5,
10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80
L1 regularized logistic regression: number of training negative
is
160; C =100
Number
of sketches in the above active basis templates are, respectively, 5,
10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60
=========================================================================================================================
Number of training positive is 80;
number of elements is 80;
Length of Gabor = 17 pixels; Local normalization or not = 1; Range
of displacement = 3 pixels; Subsample rate = 2 pixel; Image height and
width = 85 by 127 pixels
Active Basis:
Number
of sketches in the above active basis templates are, respectively, 5,
10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80
L1 regularized logistic regression: number of training negative
is 80; C =100
Number
of sketches in the above active basis templates are, respectively, 5,
10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65
