Understanding Representations and Reducing their Redundancy in Deep Networks

A new regularizer called DeCov is proposed, which leads to significantly reduced overfitting (difference between train and val performance) and greater generalization and takes advantage of the intuition that different features should try to represent different things, an intuition others have explored with similar losses.