Geometry of Shared Representations

Advances in the use of neural networks in both cognitive neuroscience and machine learning have generated new challenges: while they have proven powerful at learning complex tasks, what they learn and how they come to perform those tasks often remains a mystery. Here, we examine a novel approach to these challenges, inspired by recent spatial and algebraic analyses of abstraction and generalization in network architectures. We evaluate it, and compare it to other measures, by using it to test theoretical predictions regarding the influence that training has on the development of shared vs. separated representations, and their impact on network performance. We find that the proposed measure outperforms all others in identifying a theoretically predicted, low dimensional set of linear spatial relationships that, in turn, best predict network performance.