The speaker discusses how GPT-3 has limitations in logical reasoning despite its large network, training time, and training corpus, as it struggles to predict what comes next and lacks the ability to know when to switch to logical reasoning.