Gradient Projection Algorithms

Gradient Projection Algorithms and Software for Arbitrary Rotation Criteria in Factor Analysis

Index

1. Introduction
2. Algorithms using explicit derivatives
3. Derivative free algorithms for factor analysis
4. General GP algorithms
5. How to produce a random starting matrix?

1. Introduction

Gradient Projection algorithms are a general class of algorithms that can be used to optimize virtually any rotation criterion. Here algorithms and rotation criteria are available for download in Matlab, Splus and R, SAS PROC IML, and SPSS matrix language format.

The following rotation methods have been implemented:
Bentler's invariant pattern simplicity, biquartimax (oblimin with gamma=.5), biquartimin (oblimin with gamma=.5), Crawford-Ferguson, equamax (oblimin with gamma=p/2), factor parsimony (Crawford-Ferguson with kappa=1), geomin, infomax, minimum entropy, McCammon minimum entropy ratio, oblimax, oblimin, orthomax (orthogonal oblimin), parsimax (Crawford-Ferguson with kappa=(k-1)/(p+k-2)), partially specified target rotation, promax, quartimax, quartimin, simplimax, tandem principle I and tandem principle II, target rotation, varimax.

A list of references to the criteria is [PDF] here.

Gradient projection algorithms are designed to optimize any rotation criterion. See the programs and subroutines for examples.

Details of the algorithms, and the correct reference for the software is:
Bernaards, C.A. and Jennrich, R.I. (2005) Gradient Projection Algorithms and Software for Arbitrary Rotation Criteria in Factor Analysis. Educational and Psychological Measurement, 65 (5), 676-696. (abstract)

The software may be distributed free of charge and used by anyone if credit is given. It has been tested in several situations, but it comes with no guarantees and the authors assume no liability for its use or misuse. By downloading and/or using this software you agree to these statements.

2. Algorithms using explicit derivatives

The correct reference for this software is:
Bernaards, C.A. and Jennrich, R.I. (2005) Gradient Projection Algorithms and Software for Arbitrary Rotation Criteria in Factor Analysis. Educational and Psychological Measurement, 65 (5), 676-696. (Email one of the authors for a copy, abstract).

A quick overview is available on [PDF] slides from JSM 2003.

Theory of gradient projection algorithms may be found in:
Jennrich, R.I. (2001). A simple general procedure for orthogonal rotation. Psychometrika, 66, 289-306.
Jennrich, R.I. (2002). A simple general method for oblique rotation. Psychometrika, 67, 7-19.

Matlab
The Matlab source code consists of one file with both gradient projection algorithms in it. Also, a number of popular and less known rotation criteria are provided. Here is the Matlab implementation.

Splus
The Splus code can be sourced which will give the main algorithms, one for orthogonal and one for oblique rotation. Also, a number of popular and less known rotation criteria are provided. Here is the Splus implementation.

R
An R library is available from CRAN. The library is clean, and has help files. Alternatively, this older code may be sourced but it is less clean.

SAS PROC IML
The SAS PROC IML code is divided over two files. Orthogonal.SAS has the orthogonal rotation algorithm and a number of criteria. Oblique.SAS has the oblique rotation algorithm with its criteria.

SPSS Matrix language
The SPSS code is divided over two files. Orthogonal.SPS has the orthogonal rotation algorithm applied to a number of criteria. Oblique.SPS has the oblique rotation algorithm applied to a number of criteria.
Note that in SPSS each criteria has its own implementation of the GPF algorithm. Also, within each algorithm the criteria appears at two spots. Adding a new criterion requires that its implementation is changed at both places. The implementation for simplimax appears to work but relies on round-offs in order to serve as an artificial SIGN function. If you have a better implementation email one of the authors.

Stata
Stata 9 appears to have the GPF algorithms and most of the rotation criteria implemented in the rotate and the rotatemat commands. Another implementation of Oblimin has been written by Herve Caci, and may be downloaded from Herve's homepage.

Gauss
Li Cai has translated the algorithms into Gauss. The implementation may be downloaded from Li's homepage.

3. Derivative free algorithms for factor analysis

This implementation does not use an explicit derivative but uses numerical derivates. This algorithm is discussed in:
Jennrich, R.I. (2004). Derivative Free Gradient Projection Algorithms for Rotation. Psychometrika, 69, 475-480.

Matlab
A Matlab implementation, with a [PDF] detailed example. It has the main algorithms, and quartimin and quartimax.

Splus and R
An implementation for Splus/R with most of the methods listed above.

4. General GP algorithms

While by far their most common application, the original gradient projection algorithms of Jennrich (2000, 2001) were not restricted to rotation. They could, for example, be used to optimize the DEDICOM criterion, to simultaneously diagonalize several symmetric matrices (See, e.g., Jennrich, 2000), or be used for other non-factor analysis applications. What follows is a description of some specific Matlab code for the general orthogonal and oblique algorithms together with examples of their use. Also given is editable code that may be used directly. The advantage of the factor analysis versions is that they are easier to use for factor analysis applications.

[PDF] Discussion of the Matlab code and examples.

Editable code (text-file).

5. How to produce a random starting matrix?

For criteria used here, it may be useful to consider more than one starting value for the rotation. The identity matrix is used by default. Here we provide functions to obtain a random orthogonal k by k matrix. That is a random orthogonal matrix uniformily distributed over the group of k by k orthogonal matricies. The codes are very brief.

Matlab
The Matlab source code consists of one file with the random start algorithm. Here is the Matlab implementation.

Splus and R
The Splus code can be sourced, and will provide the function Random.Start. Here is the Splus/R implementation.

SAS PROC IML
The SAS code results in a random matrix called rm. Here is the SAS PROC IML implementation.

SPSS Matrix language
The SPSS code results in a random matrix called Tmat. Here is the SPSS matrix language implementation. Disclaimer: this procedure is based on uniform random variable draws, not on normally distributed random variables. Therefore, the resulting random matrices may not be uniformly distributed over all orthogonal random matrices (but they are random).

Stata
Stata 9 has a random start option built in, see the optimization options included in the rotatemat command.

Investigators: Coen A. Bernaards and Robert I. Jennrich.

UCLA Department of Statistics
Last updated: 03-Nov-2005
Access count is: 237125, since 01-Oct-2003
Maintained by: Coen A. Bernaards and Robert I. Jennrich [webstaff@stat.ucla.edu]