Algorithmic Regularization in Over-parameterized Matrix Sensing and Neural Networks with Quadratic Activations

The gradient descent algorithm provides an implicit regularization effect in the learning of over-parameterized matrix factorization models and one-hidden-layer neural networks with quadratic activations and the results solve the conjecture of Gunasekar et al.