1. Construct the sample variance-covariance matrix
2. Find the eigenvectors
3. Projection : use each eigenvector to form a linear combination of original variables
4. The larger, the better :             the k-th principal component is the projection with the k-th largest eigenvalue

Alternative view

   Data matrix  :

1-D data matrix : rank 1
 2-D data matrix :rank 2
K-D data matrix : rank k

Eigenvectors for 1-D sample covariance matrix: rank 1
Eigenvectors for 2-D sample covariance matrix: rank 2
Eigenvectors for k-D sample matrix

Conclusion :

Principal components can be  found by applying eigenvalue decomposition to the
sample covariance matrix of X .

Further discussion :

Adding i.i.d. noise.  This does not change the eigenvectors.  Why ?
      (because the new covariance matrix is equal to the old one plus  a multiple of the identity matrix  I )
 

Connection with automatic basis curve finding (to be discussed later)