Basic facts(cont.)
Covariance matrix is degenerated (I.e, some eigenvalues are zero) if data are confined to a lower dimensional space S
Rank of covariance matrix = number of non-zero eigenvalues = dim. of the space S
This explain why pca works for our first example
Why small errors can be tolerated ?
Large i.i.d. errors are fine too
Heterogeneity is harmful, correlated errors too