• We are pleased to announce that the winner of our Feedback Prize Draw for the Winter 2024-25 session and winning £150 of gift vouchers is Zhao Liang Tay. Congratulations to Zhao Liang. If you fancy winning £150 worth of gift vouchers (from a major UK store) for the Summer 2025 exam sitting for just a few minutes of your time throughout the session, please see our website at https://www.acted.co.uk/further-info.html?pat=feedback#feedback-prize for more information on how you can make sure your name is included in the draw at the end of the session.
  • Please be advised that the SP1, SP5 and SP7 X1 deadline is the 14th July and not the 17th June as first stated. Please accept out apologies for any confusion caused.

PCA geometric meaning

Y

ykai

Ton up Member
I found a lot of information on the Internet but still can't fully understand the geometric meaning of each step of PCA.
I probably know the application of PCA but I don't know enough about the meaning of this step, but I hope to fully understand the logic and meaning of each step of PCA from beginning to end.

Suppose the centered matrix is a 2*2 matrix X
Step
1.Get the centered matrix=>move the coordinate origin (0.0) to the center of the data group

2. Multiply the transposed matrix of the centered matrix with the centered matrix =>?
X^T*X
I know that the meaning of matrix multiplication is the area between two vectors, but I don't quite understand the meaning of this step. I know diagonal are variance.

3. Take eigenvalues&eigenvectors for the product of centered matrix & its transposed matrix =>?
I only know eigenvalue represent scale & direction of eigenvectors and eigenvector are (x,y).

4. Arrange the unit eigenvectors from left to right according to the value of eigenvalues to get a new matrix W

5.PCA=XW=>Convert the data of X to a new span?Maybe?

6. PCA^T * PCA=>?
I remember that PCA seems to have a sentence of maximizing variance, but I can't understand how it is applied to it.

I need an explanation of the geometric meaning of step 2, 3, 4, 5, 6.
Thank you very much!
 
Last edited:
This information is what I spent a lot of time gathering and guessing.I am really not sure about correctness.
I have no relevant academic background, so I can only guess.
Please let me know if I am wrong, I also really want to confirm that the information I collected is correct.
I'd appreciate it if you could add more that I don't know about and where it's incomplete.

1.Get the centered matrix=>move the coordinate origin (0.0) to the center of the data group

2. Multiply the transposed matrix of the centered matrix with the centered matrix =>
CMP said component in matrix are variance and covariance.
X^T*X=real symmetric matrix
A real symmetric matrix must be obtained = the eigenvectors corresponding to different eigenvalues are orthogonal to each other.
They are vertical to each other.
I guess it could said that eigenvectors are uncorrelated, because cov of orthogonal matrix make component of them be 0,cov=0.
It seems correspond to "PCA identifies a set of uncorrelated linear combinations of the original data", which come from CMP.
linear combinations is eigenvectors ,is a span or coordinate plane.

Determinant=0=>Dimensionality reduction.
I guess it is because PCA doubt that there are too many redundant vectors, so it is necessary to reduce the dimension and take the feature vector.
There is redundant vectors if the eigenvector is 0,so it seems to not correct.

3. Take eigenvalues&eigenvectors for the product of centered matrix & its transposed matrix =>
Get eigenvectors with different explanatory power(eigenvalues).
explanatory power=explained more proportion of variance
eigenvector=>eigenvalue

Find the PC perpendicular to each point in order to get more explanatory power with a small number of PCs.
https://i.stack.imgur.com/Q7HIP.gif
PC which we wanted is black line.

I also read wikipedia for eigenvector and eigenvalue.
I seems to know something about it, but not sure.

4. Arrange the unit eigenvectors from left to right according to the value of eigenvalues to get a new matrix W=>
PCA like more explanatory power.
It come from CMP.

5.PCA=XW=>Convert the data of X to a new span(PC1 or PC2 or PCn)

6. PCA^T * PCA=>Looking proportion explained by PCA for each PCn.Diagonal element
 
Last edited:
I can't find a perfect explanation for Determinant=0,because maybe there are no 0 eigenvalue.
I can't be 100% sure that there is linear dependence that allows us to use Determinant=0.
I don't have enough time to delve into this right now.
 
You must log in or register to reply here.
Back
Top