Episode
#108 – Sergey Levine: Robotics and Machine Learning
1:37:59
Published: Tue Jul 14 2020
Description
Sergey Levine is a professor at Berkeley and a world-class researcher in deep learning, reinforcement learning, robotics, and computer vision, including the development of algorithms for end-to-end training of neural network policies that combine perception and control, scalable algorithms for inverse reinforcement learning, and deep RL algorithms. Support this podcast by supporting these sponsors: - ExpressVPN: https://www.expressvpn.com/lexpod - Cash App – use code "LexPodcast" and download: - Cash App (App Store): https://apple.co/2sPrUHe - Cash App (Google Play): https://bit.ly/2MlvP5w If you would like to get more information about this podcast go to https://lexfridman.com/ai or connect with @lexfridman on Twitter, LinkedIn, Facebook, Medium, or YouTube where you can watch the video versions of these conversations. If you enjoy the podcast, please rate it 5 stars on Apple Podcasts, follow on Spotify, or support it on Patreon. Here's the outline of the episode. On some podcast players you should be able to click the timestamp to jump to that time. OUTLINE: 00:00 - Introduction 03:05 - State-of-the-art robots vs humans 16:13 - Robotics may help us understand intelligence 22:49 - End-to-end learning in robotics 27:01 - Canonical problem in robotics 31:44 - Commonsense reasoning in robotics 34:41 - Can we solve robotics through learning? 44:55 - What is reinforcement learning? 1:06:36 - Tesla Autopilot 1:08:15 - Simulation in reinforcement learning 1:13:46 - Can we learn gravity from data? 1:16:03 - Self-play 1:17:39 - Reward functions 1:27:01 - Bitter lesson by Rich Sutton 1:32:13 - Advice for students interesting in AI 1:33:55 - Meaning of life
Chapters
Host Lex Fridman talks to Sergey Levine, a professor at Berkeley and expert in deep learning, reinforcement learning, robotics, and computer vision.
00:00 - 03:00 (03:00)
Summary
Host Lex Fridman talks to Sergey Levine, a professor at Berkeley and expert in deep learning, reinforcement learning, robotics, and computer vision. Topics include end-to-end training of neural network policies, scalable algorithms for inverse reinforcement learning, and deep RL algorithms.
Episode#108 – Sergey Levine: Robotics and Machine Learning
PodcastLex Fridman Podcast
The process of distilling a common sense understanding of the world through machine learning is a big challenge, as it requires the capability to take an unstructured mass of experience and transform it into knowledge.
03:00 - 14:51 (11:50)
Summary
The process of distilling a common sense understanding of the world through machine learning is a big challenge, as it requires the capability to take an unstructured mass of experience and transform it into knowledge. However, with interactive learning and continuous experience, acquiring a common sense understanding is much more likely.
Episode#108 – Sergey Levine: Robotics and Machine Learning
PodcastLex Fridman Podcast
Robotics can help us understand and learn about artificial intelligence, beyond just being a pragmatic and useful field in itself.
14:51 - 21:43 (06:51)
Summary
Robotics can help us understand and learn about artificial intelligence, beyond just being a pragmatic and useful field in itself. The potential for robots to help us understand intelligence and ourselves is a source of inspiration for many people in this field.
Episode#108 – Sergey Levine: Robotics and Machine Learning
PodcastLex Fridman Podcast
Pieter Abbeel discusses the challenges of combining perception and control in robotics and how he believes treating them together produces better results than trying to separate them.
21:43 - 27:46 (06:03)
Summary
Pieter Abbeel discusses the challenges of combining perception and control in robotics and how he believes treating them together produces better results than trying to separate them.
Episode#108 – Sergey Levine: Robotics and Machine Learning
PodcastLex Fridman Podcast
The speaker debates whether or not robotics is the best way to study and develop artificial intelligence, as some believe the lack of common sense reasoning in current systems is due to the inability to interact with specific worlds to gain knowledge, while others see robotics as a fundamental space to explore fundamental learning mechanisms for general intelligence.
27:46 - 35:23 (07:36)
Summary
The speaker debates whether or not robotics is the best way to study and develop artificial intelligence, as some believe the lack of common sense reasoning in current systems is due to the inability to interact with specific worlds to gain knowledge, while others see robotics as a fundamental space to explore fundamental learning mechanisms for general intelligence.
Episode#108 – Sergey Levine: Robotics and Machine Learning
PodcastLex Fridman Podcast
The difference between learning-based systems and classic optimal control systems is that the former's performance gets better with time but the question of explainability is closely tied to its performance.
35:23 - 44:12 (08:49)
Summary
The difference between learning-based systems and classic optimal control systems is that the former's performance gets better with time but the question of explainability is closely tied to its performance.
Episode#108 – Sergey Levine: Robotics and Machine Learning
PodcastLex Fridman Podcast
This podcast episode discusses the concepts of reinforcement learning, including policy based, model-based, on policy and off policy.
44:12 - 53:49 (09:37)
Summary
This podcast episode discusses the concepts of reinforcement learning, including policy based, model-based, on policy and off policy. The speakers consider the potential for reinforcement learning to solve a broad range of AI problems in the future.
Episode#108 – Sergey Levine: Robotics and Machine Learning
PodcastLex Fridman Podcast
A discussion on the combination of reinforcement learning algorithms with high capacity neural net representations and how it leads to near optimal control without the need for a complete model of the world.
53:49 - 1:05:25 (11:35)
Summary
A discussion on the combination of reinforcement learning algorithms with high capacity neural net representations and how it leads to near optimal control without the need for a complete model of the world.
Episode#108 – Sergey Levine: Robotics and Machine Learning
PodcastLex Fridman Podcast
Simulation is a useful tool in machine learning, but in the long run, machines that learn from real data will improve perpetually, as relying solely on simulated data can create a bottleneck.
1:05:25 - 1:13:04 (07:39)
Summary
Simulation is a useful tool in machine learning, but in the long run, machines that learn from real data will improve perpetually, as relying solely on simulated data can create a bottleneck. Challenges in reinforcement learning, such as the need to create realistic simulations, become more apparent when running programs in the real world.
Episode#108 – Sergey Levine: Robotics and Machine Learning
PodcastLex Fridman Podcast
This podcast discusses how discovering new things can be an emergent property of some other objective that quantifies capability, and how multi-agent interaction can lead to understanding that other beings in the world have their own goals, intentions, and thoughts.
1:13:04 - 1:23:17 (10:12)
Summary
This podcast discusses how discovering new things can be an emergent property of some other objective that quantifies capability, and how multi-agent interaction can lead to understanding that other beings in the world have their own goals, intentions, and thoughts.
Episode#108 – Sergey Levine: Robotics and Machine Learning
PodcastLex Fridman Podcast
The speaker discusses the importance of optimizing objectives for AI without creating unintended consequences that may arise in safety-critical systems.
1:23:17 - 1:34:13 (10:56)
Summary
The speaker discusses the importance of optimizing objectives for AI without creating unintended consequences that may arise in safety-critical systems. He refers to Isaac Haslamoff's work as 'inspiring'.
Episode#108 – Sergey Levine: Robotics and Machine Learning
PodcastLex Fridman Podcast
The limitations of AI simulators and relying on label datasets can hinder progress in making real-world changes, leading to a bigger philosophical question about the meaning of life.
1:34:13 - 1:37:36 (03:22)
Summary
The limitations of AI simulators and relying on label datasets can hinder progress in making real-world changes, leading to a bigger philosophical question about the meaning of life.