The speaker discusses the challenges of using the vast data available on the internet to teach AI about human civilization, including the need to understand context and the limitations of structured data.