The speaker discusses the power of deep learning and how neural nets with non-convex objective functions and numerous parameters can learn anything from a small amount of data using stochastic gradient descent, contrary to what is taught in traditional textbooks.