Episode
#144 – Michael Littman: Reinforcement Learning and the Future of AI
2:01:55
Published: Sun Dec 13 2020
Description
Michael Littman is a computer scientist at Brown University. Please support this podcast by checking out our sponsors: - SimpliSafe: https://simplisafe.com/lex and use code LEX to get a free security camera - ExpressVPN: https://expressvpn.com/lexpod and use code LexPod to get 3 months free - MasterClass: https://masterclass.com/lex to get 2 for price of 1 - BetterHelp: https://betterhelp.com/lex to get 10% off EPISODE LINKS: Michael's Twitter: https://twitter.com/mlittmancs Michael's Website: https://www.littmania.com/ Michael's YouTube: https://www.youtube.com/user/mlittman PODCAST INFO: Podcast website: https://lexfridman.com/podcast Apple Podcasts: https://apple.co/2lwqZIr Spotify: https://spoti.fi/2nEwCF8 RSS: https://lexfridman.com/feed/podcast/ YouTube Full Episodes: https://youtube.com/lexfridman YouTube Clips: https://youtube.com/lexclips SUPPORT & CONNECT: - Check out the sponsors above, it's the best way to support this podcast - Support on Patreon: https://www.patreon.com/lexfridman - Twitter: https://twitter.com/lexfridman - Instagram: https://www.instagram.com/lexfridman - LinkedIn: https://www.linkedin.com/in/lexfridman - Facebook: https://www.facebook.com/LexFridmanPage - Medium: https://medium.com/@lexfridman OUTLINE: Here's the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time. (00:00) - Introduction (07:43) - Robot and Frank (10:02) - Music (13:13) - Starring in a TurboTax commercial (23:26) - Existential risks of AI (41:48) - Reinforcement learning (1:07:36) - AlphaGo and David Silver (1:17:15) - Will neural networks achieve AGI? (1:29:42) - Bitter Lesson (1:42:32) - Does driving require a theory of mind? (1:51:58) - Book Recommendations (1:57:20) - Meaning of life
Chapters
The speaker offers three potential ideas for future podcast topics, including discussing a historical moment, analyzing a movie, or exploring a book.
00:00 - 02:41 (02:41)
Summary
The speaker offers three potential ideas for future podcast topics, including discussing a historical moment, analyzing a movie, or exploring a book. An advertisement for ExpressVPN, MasterClass, and Better Help follows the discussion.
Episode#144 – Michael Littman: Reinforcement Learning and the Future of AI
PodcastLex Fridman Podcast
The movie Robot and Frank shows the life of a jewel thief with a home robot, which gives us a glimpse of how robots might be deployed as helpers at homes in the near term future.
02:41 - 09:13 (06:31)
Summary
The movie Robot and Frank shows the life of a jewel thief with a home robot, which gives us a glimpse of how robots might be deployed as helpers at homes in the near term future.
Episode#144 – Michael Littman: Reinforcement Learning and the Future of AI
PodcastLex Fridman Podcast
The host discusses how he became interested in music production and got to experience listening to different instruments and layers while playing in a jazz and fusion band in college.
09:13 - 12:55 (03:42)
Summary
The host discusses how he became interested in music production and got to experience listening to different instruments and layers while playing in a jazz and fusion band in college. Later imagining himself applying the same analysis skills to pop hits such as Justin Bieber and Cardi B.
Episode#144 – Michael Littman: Reinforcement Learning and the Future of AI
PodcastLex Fridman Podcast
A glimpse into what goes on behind the scenes of a film set, including the various jobs and roles of crew members, and how guests are managed during filming.
12:55 - 17:58 (05:02)
Summary
A glimpse into what goes on behind the scenes of a film set, including the various jobs and roles of crew members, and how guests are managed during filming.
Episode#144 – Michael Littman: Reinforcement Learning and the Future of AI
PodcastLex Fridman Podcast
Calvin is taught to see things from a different viewpoint, causing a breakthrough, but an unexpected interruption causes a shift in topic to Michael Jackson's stalker.
17:58 - 25:15 (07:16)
Summary
Calvin is taught to see things from a different viewpoint, causing a breakthrough, but an unexpected interruption causes a shift in topic to Michael Jackson's stalker.
Episode#144 – Michael Littman: Reinforcement Learning and the Future of AI
PodcastLex Fridman Podcast
The process of developing technology that can truly interact with the world will provide knowledge on the meaning of doing so, which is currently unknown.
25:15 - 34:03 (08:48)
Summary
The process of developing technology that can truly interact with the world will provide knowledge on the meaning of doing so, which is currently unknown. Although there are pressing issues today, such as nuclear weapons, new knowledge gained from technology development can lead to better solutions.
Episode#144 – Michael Littman: Reinforcement Learning and the Future of AI
PodcastLex Fridman Podcast
The simple algorithms that come with social media, along with interactive AI, can have controlling effects on young people, while also exposing them to online bullying.
34:04 - 40:40 (06:36)
Summary
The simple algorithms that come with social media, along with interactive AI, can have controlling effects on young people, while also exposing them to online bullying. Despite the damaging effects, social media may be pushing society towards progress.
Episode#144 – Michael Littman: Reinforcement Learning and the Future of AI
PodcastLex Fridman Podcast
The speaker discusses how they spent their vacations in college hacking on their home computer and trying to teach it how to play tic-tac-toe, which led them to pursue a degree in computer science.
40:40 - 45:52 (05:11)
Summary
The speaker discusses how they spent their vacations in college hacking on their home computer and trying to teach it how to play tic-tac-toe, which led them to pursue a degree in computer science.
Episode#144 – Michael Littman: Reinforcement Learning and the Future of AI
PodcastLex Fridman Podcast
The neuroscience and cognitive science community are starting to learn how to program and use artificial neural networks, leading to more intersections with deep learning.
45:52 - 50:10 (04:17)
Summary
The neuroscience and cognitive science community are starting to learn how to program and use artificial neural networks, leading to more intersections with deep learning. While some cognitive scientists are complaining about deep networks, there is a growing overlap between the two fields.
Episode#144 – Michael Littman: Reinforcement Learning and the Future of AI
PodcastLex Fridman Podcast
The ability to implement intelligence in a single line of code and the importance of worst-case analysis on sorting algorithms and reinforcement learning algorithms.
50:10 - 55:34 (05:24)
Summary
The ability to implement intelligence in a single line of code and the importance of worst-case analysis on sorting algorithms and reinforcement learning algorithms.
Episode#144 – Michael Littman: Reinforcement Learning and the Future of AI
PodcastLex Fridman Podcast
This podcast episode features a discussion on reinforcement learning and temporal difference learning with a focus on the impact of Jerry Tesaro in developing these fields.
55:34 - 1:02:43 (07:08)
Summary
This podcast episode features a discussion on reinforcement learning and temporal difference learning with a focus on the impact of Jerry Tesaro in developing these fields. The episode also touches upon the role of game playing and the use of neural nets in the learning process.
Episode#144 – Michael Littman: Reinforcement Learning and the Future of AI
PodcastLex Fridman Podcast
The podcast explores the concept of self play and its significance in the learning process of systems, similar to TD Gammon, while delving into discussions with experienced physicists.
1:02:43 - 1:05:19 (02:35)
Summary
The podcast explores the concept of self play and its significance in the learning process of systems, similar to TD Gammon, while delving into discussions with experienced physicists.
Episode#144 – Michael Littman: Reinforcement Learning and the Future of AI
PodcastLex Fridman Podcast
David Silver, a leading researcher in AI and one of the creators of AlphaGo, discusses how traditional AI approaches hit their limit with complex games like Go and how AlphaGo Zero and AlphaZero surpassed these limitations through self-play and reinforcement learning.
1:05:19 - 1:10:28 (05:09)
Summary
David Silver, a leading researcher in AI and one of the creators of AlphaGo, discusses how traditional AI approaches hit their limit with complex games like Go and how AlphaGo Zero and AlphaZero surpassed these limitations through self-play and reinforcement learning.
Episode#144 – Michael Littman: Reinforcement Learning and the Future of AI
PodcastLex Fridman Podcast
The application of reinforcement learning and neural net can help modify the heuristics elevator banks use for decision making in maximizing throughput and stoppage on floors in the most optimal manner.
1:10:28 - 1:16:18 (05:50)
Summary
The application of reinforcement learning and neural net can help modify the heuristics elevator banks use for decision making in maximizing throughput and stoppage on floors in the most optimal manner.
Episode#144 – Michael Littman: Reinforcement Learning and the Future of AI
PodcastLex Fridman Podcast
Ben Johnson explains the capabilities and limitations of computer chess and whether or not we will reach a point where even the best human chess players can no longer win against computer opponents.
1:16:18 - 1:22:50 (06:31)
Summary
Ben Johnson explains the capabilities and limitations of computer chess and whether or not we will reach a point where even the best human chess players can no longer win against computer opponents.
Episode#144 – Michael Littman: Reinforcement Learning and the Future of AI
PodcastLex Fridman Podcast
This podcast discusses the possibility of applying self-supervised learning to human-significant problems such as autonomous vehicles and robotics applications.
1:22:50 - 1:27:25 (04:35)
Summary
This podcast discusses the possibility of applying self-supervised learning to human-significant problems such as autonomous vehicles and robotics applications. It raises questions about the limits of self-play and neural networks in language models in the context of AGI.
Episode#144 – Michael Littman: Reinforcement Learning and the Future of AI
PodcastLex Fridman Podcast
The podcast discusses how advanced AI language models can potentially be used to manipulate human behavior for advertisers, and the importance of remaining cautious when developing and implementing these models.
1:27:25 - 1:39:45 (12:19)
Summary
The podcast discusses how advanced AI language models can potentially be used to manipulate human behavior for advertisers, and the importance of remaining cautious when developing and implementing these models.
Episode#144 – Michael Littman: Reinforcement Learning and the Future of AI
PodcastLex Fridman Podcast
The guest speaker and computer scientist, was not an avid reader, in contrast to many in his field of academia.
1:39:45 - 1:43:51 (04:06)
Summary
The guest speaker and computer scientist, was not an avid reader, in contrast to many in his field of academia. He instead focused on cognitive science while loving to associate with cognitive scientists.
Episode#144 – Michael Littman: Reinforcement Learning and the Future of AI
PodcastLex Fridman Podcast
Self-driving cars are difficult because driving is a social interaction activity that relies upon predicting the actions of others.
1:43:51 - 1:47:31 (03:40)
Summary
Self-driving cars are difficult because driving is a social interaction activity that relies upon predicting the actions of others. This is particularly challenging when focusing on pedestrians, for example.
Episode#144 – Michael Littman: Reinforcement Learning and the Future of AI
PodcastLex Fridman Podcast
The gradual adoption of self-driving cars within closed communities is proposed until AI technology is advanced enough to handle real-world problems.
1:47:31 - 1:52:06 (04:34)
Summary
The gradual adoption of self-driving cars within closed communities is proposed until AI technology is advanced enough to handle real-world problems. There are concerns regarding the potential negative impact of AI, but its progress is largely driven by entrepreneurs like Elon Musk.
Episode#144 – Michael Littman: Reinforcement Learning and the Future of AI
PodcastLex Fridman Podcast
The idea of the alignment problem is useful when thinking about the ethics of creating AGI, similar to how the trolley problem is useful for autonomous vehicles.
1:52:06 - 1:57:19 (05:12)
Summary
The idea of the alignment problem is useful when thinking about the ethics of creating AGI, similar to how the trolley problem is useful for autonomous vehicles. Reading books like Program or Be Programmed by Douglas Ross can help individuals understand the importance of becoming programmers in some form.
Episode#144 – Michael Littman: Reinforcement Learning and the Future of AI
PodcastLex Fridman Podcast
The concept of the meaning of life is explored in the context of reinforcement learning agents and researchers, with insights from a personal experience shared by the speaker.
1:57:19 - 2:01:31 (04:11)
Summary
The concept of the meaning of life is explored in the context of reinforcement learning agents and researchers, with insights from a personal experience shared by the speaker.