Clip
The Future of Data Efficiency in Neural Nets
As neural nets become more advanced, they will increasingly become more data efficient at learning new tasks, requiring fewer examples to train additional behaviors. Pre-training a massive neural net like GPT will become increasingly useful at training any arbitrary new task.