Self-supervised learning involves building common sense knowledge in neural networks without human supervision by providing them with a large database of unlabeled image examples to learn from. Unlike supervised learning, it allows the machine to learn generalizable ideas at the core of language and vision.