Chapter
From Imitation Learning to Full Self-Supervised Learning in AlphaFold
51:11 - 56:48 (05:36)
The AlphaGo, AlphaGo Zero, AlphaZero, MuZero, and AlphaFold all evolved from handcrafting some hard-coded constraints around physics and biology with a small end-to-end learning piece or a small learning piece, and growing that learning piece until it consumes the whole system, while using various tricks like self-distillation, to make the training set bigger.
Clips
This podcast discusses the innovations behind AlphaFold, including hard-coded constraints around physics and self-distillation of AlphaFold predictions, which have enabled the deep learning system to learn the 3D structures of proteins more accurately and efficiently than any other method.
51:11 - 54:26 (03:15)
Summary
This podcast discusses the innovations behind AlphaFold, including hard-coded constraints around physics and self-distillation of AlphaFold predictions, which have enabled the deep learning system to learn the 3D structures of proteins more accurately and efficiently than any other method.
ChapterFrom Imitation Learning to Full Self-Supervised Learning in AlphaFold
Episode#299 – Demis Hassabis: DeepMind
PodcastLex Fridman Podcast
The evolution of AI starts with imitation learning and then gradually moves towards self-supervised learning.
54:26 - 56:48 (02:21)
Summary
The evolution of AI starts with imitation learning and then gradually moves towards self-supervised learning. It would have been very difficult to start with full self-supervised learning without going through the previous stages.