The speaker discusses the benefits of having a general purpose computer that can be trained on arbitrary problems and is both expressive in the forward pass and optimizable via backpropagation and gradient descent, allowing for efficient high parallelism compute graphs.