Researchers have discovered that very large models can be trained in a self-supervised manner on unfiltered internet images, which is made possible by the use of a data augmentation method designed for self-supervised learning in vision that is overfitted to ImageNet.