Singular value decomposition (SVD)
SVD decomposes any mxn matrix A as
Properties
Σ is a diagonal matrix containing the eigenvalues of ATA
known as “singular values” of A
diagonal entries are sorted from largest to smallest
columns of U are eigenvectors of AAT
columns of V are eigenvectors of ATA
If A is singular (e.g., has rank 3)
only first 3 singular values are nonzero
we can throw away all but first 3 columns of U and V
Choose M’ = U’,  S’ = ΣV’T