The conversation explores the idea that AI can encompass more than just language and knowledge, by mapping different modalities of knowledge to different tasks, beyond the generalizability of AI systems like GPT-3. However, machine learning systems like GATO are exploring prediction capabilities which can extend beyond mere language models.