Chapter
The Ideal Test of Human and Machine Intelligence
1:25:21 - 1:37:22 (12:00)
According to the speaker, an ideal test of human and machine intelligence is one that is actionable, highlights the need for progress, and the direction in which to make such progress. The speaker also suggests that a difficult test can be created by requiring a lot of extrapolation over training examples, instead of relying on ambiguity in test questions.
Clips
The difficulty of a test question can come from requiring a lot of extrapolation over the training examples, or from the input prompt being very surprising given the training examples.
1:25:21 - 1:33:33 (08:12)
Summary
The difficulty of a test question can come from requiring a lot of extrapolation over the training examples, or from the input prompt being very surprising given the training examples. The level of difficulty depends on the test taker's existing knowledge and understanding of the task at hand.
ChapterThe Ideal Test of Human and Machine Intelligence
Episode#120 – François Chollet: Measures of Intelligence
PodcastLex Fridman Podcast
In designing an intelligence test for machines, it is crucial to create a test that is actionable and emphasizes the need for progress in a specific direction.
1:33:33 - 1:37:22 (03:48)
Summary
In designing an intelligence test for machines, it is crucial to create a test that is actionable and emphasizes the need for progress in a specific direction. Additionally, the test should measure skill acquisition efficiency rather than just skill itself, and every task should be novel to prevent anticipation.