Topological Data Analysis (TDA) as a Framework for Understanding Deep Learning Behavior

The rising complexity and opacity of deep learning models over the years has made interpretability, transparency, and trust-worthiness for these models increasingly important. Many common explainability techniques are based off gradient-based, or perturbation based approach that could fail to understand global aspects of neural network behavior. In this paper, we introduce Topological Data Analysis (TDA) with special emphasis on persistent homology, as one of the most useful and powerful frameworks for understanding, and visualizing deep learning model dynamics, through regards to the structured, or topological features in high-dimensional activation spaces in which TDA can expose important patterns (clusters, loops, voids, etc.) as the behavioral traits that neural networks exhibit. We examine the evolution of topological signatures during training in order to understand characteristics such as feature learning, overfitting and generalization. We present experiments using standard architectures (such as Convolutional Neural Networks (CNNs) and Multi-Layer Perceptrons (MLPs)) across datasets such as MNIST and CIFAR-10 which show that persistent diagrams and barcodes are useful for distinguishing structural changes in learned representations. Additionally, TDA-based summaries also offered model agnostic analysis which allowed us to compare representations across different architectures and optimization strategies. By bringing together topological perspective with traditional performance metrics, this study expands our understanding of model behavior, and sets the stage for topology aware model design and evaluation, ultimately establishing TDA as an important approach in the pursuit of explainable and interpretable deep learning.