Backward Feature Correction: How Deep Learning Performs Deep Learning

This paper formally analyzes how multi-layer neural networks can perform hierarchical learning efficiently and automatically by applying SGD and establishes a principle called "backward feature correction", where training higher layers in the network can improve the features of lower level ones.