On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks

This work investigates how PINNs are biased towards learning functions along the dominant eigen-directions of their limiting NTK, and constructs novel architectures that employ spatio-temporal and multi-scale random Fourier features, and justifies how such coordinate embedding layers can lead to robust and accurate PINN models.