Here is a thought of the day: in general a principal geodesic component does not pass through the intrinsic mean on the manifold. Which is not the case in the usual Euclidean PCA. The reference is the article by Stephan Huckemann. I think it is worth to carefully go through it.