A survey on design choices for self-supervised learning in computer vision

The results demonstrate the advantages of in-domain pretraining and the importance of aligning all design choices in order to ensure optimal results, as well as identifying the characteristic differences between predictive, contrastive and generative self-supervised learning.