Chapter
Philosophy of AI: Understanding Time Horizons
49:16 - 53:40 (04:24)
The podcast hosts discuss pushing the horizon back and extending experience in AI. They use chess as an example to explain the importance of time steps in training AI to become better at decision-making.
Clips
The decision to extend the time horizon is crucial in reinforcement learning, as there are problems that cannot be solved with limited horizons.
49:16 - 51:54 (02:38)
Summary
The decision to extend the time horizon is crucial in reinforcement learning, as there are problems that cannot be solved with limited horizons. It is equally good to have a reward of 1 in each time step as there being infinite time steps, but it's important to push the horizon back as more is experienced in the world.
ChapterPhilosophy of AI: Understanding Time Horizons
Episode#75 – Marcus Hutter: Universal Artificial Intelligence, AIXI, and AGI
PodcastLex Fridman Podcast
Asymptotic results in reinforcement learning mean that in the long run, if the agent acts for a long enough time, it will perform optimally or result in some desirable outcome, without a clear indication of how fast this will be achieved.
51:54 - 53:40 (01:46)
Summary
Asymptotic results in reinforcement learning mean that in the long run, if the agent acts for a long enough time, it will perform optimally or result in some desirable outcome, without a clear indication of how fast this will be achieved. If the agent has a fixed horizon, asymptotic results cannot be proven.