Why are principal component scores uncorrelated?
Supose A is a matrix of mean-centred data. The matrix S=cov(A) is m×m, has m distinct eigenvalues, and eigenvectors s1, s2 … sm, which are orthogonal.
The i-th principal component (some people call them “scores”) is the vector zi=Asi. In other words, it’s a linear combination of the columns of A, where the coefficients are the components of the i-th eigenvector of S.
I don’t understand why zi and zj turn out to be uncorrelated for all i≠j. Does it follow from the fact that si and sj are orthogonal? Surely not, because I can easily find a matrix B and a pair of orthogonal vectors x,y such that Bx and By are correlated.