The speaker shares their experience with distributed image recognition on ImageNet and why they think multi-GPU and multiple machine training is largely a waste of time.