Chapter
Sparse Representations are Key to Deep Learning
1:17:29 - 1:22:03 (04:33)
Sparse representations are important for deep learning networks to generalize well and avoid overfitting. The brain also relies on sparse representations with only a small percentage of neurons being active at any given time.
Clips
The brain has 10,000 neurons with only 2% active at one time.
1:17:29 - 1:18:54 (01:24)
Summary
The brain has 10,000 neurons with only 2% active at one time. These active neurons typically represent a location, input from the senses, or a concept.
ChapterSparse Representations are Key to Deep Learning
EpisodeJeff Hawkins: Thousand Brains Theory of Intelligence
PodcastLex Fridman Podcast
By enforcing sparseness in both the activation of neurons and connections between them in Convolutional Neural Networks (CNNs), improvements can be made.
1:18:54 - 1:20:28 (01:34)
Summary
By enforcing sparseness in both the activation of neurons and connections between them in Convolutional Neural Networks (CNNs), improvements can be made. This concept is being explored in a DARPA contest for artificial intelligence.
ChapterSparse Representations are Key to Deep Learning
EpisodeJeff Hawkins: Thousand Brains Theory of Intelligence
PodcastLex Fridman Podcast
The challenge in creating artificial intelligence with sparse representations is figuring out how to divide a high-dimensional space into different pieces.
1:20:28 - 1:22:03 (01:34)
Summary
The challenge in creating artificial intelligence with sparse representations is figuring out how to divide a high-dimensional space into different pieces. This can involve moving multiple cells in order to differentiate between representations, and requires the use of algorithms that can understand the differences between different vectors.