Centering Neural Network Gradient Factors

This work proposes centering the slope of hidden unit activation functions as well, which removes the linear component of backpropagated error and improves credit assignment in networks with shortcut connections.