The discussion covers important problems that can be solved using self-supervised approaches, such as self-play for autonomous vehicles, robotics applications, and simulation. Also, it questions the limits of self-play and neural networks within AGI and highlights recent breakthroughs achieved in natural language processing.