Why do tree-based models still outperform deep learning on typical tabular data?

An empirical investigation into the differing inductive biases of tree-based models and Neural Networks leads to a series of challenges which should guide researchers aiming to build tabular-specific NNs.