Jim Keller is a legendary microprocessor engineer, previously at AMD, Apple, Tesla, Intel, and now Tenstorrent. Please support this podcast by checking out our sponsors: - Athletic Greens: https://athleticgreens.com/lex and use code LEX to get 1 month of fish oil - Brooklinen: https://brooklinen.com and use code LEX to get $25 off + free shipping - ExpressVPN: https://expressvpn.com/lexpod and use code LexPod to get 3 months free - Belcampo: https://belcampo.com/lex and use code LEX to get 20% off first order EPISODE LINKS: Jim's Twitter: https://twitter.com/jimkxa Jim's Wiki: https://en.wikipedia.org/wiki/Jim_Keller_(engineer) Tenstorrent: https://www.tenstorrent.com/ PODCAST INFO: Podcast website: https://lexfridman.com/podcast Apple Podcasts: https://apple.co/2lwqZIr Spotify: https://spoti.fi/2nEwCF8 RSS: https://lexfridman.com/feed/podcast/ YouTube Full Episodes: https://youtube.com/lexfridman YouTube Clips: https://youtube.com/lexclips SUPPORT & CONNECT: - Check out the sponsors above, it's the best way to support this podcast - Support on Patreon: https://www.patreon.com/lexfridman - Twitter: https://twitter.com/lexfridman - Instagram: https://www.instagram.com/lexfridman - LinkedIn: https://www.linkedin.com/in/lexfridman - Facebook: https://www.facebook.com/LexFridmanPage - Medium: https://medium.com/@lexfridman OUTLINE: Here's the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time. (00:00) - Introduction (07:02) - Good design is both science and engineering (13:03) - Javascript (17:09) - RISC vs CISC (21:09) - What makes a great processor? (22:38) - Intel vs ARM (24:27) - Steve Jobs and Apple (27:05) - Elon Musk and Steve Jobs (32:50) - Father (36:33) - Perfection (42:48) - Modular design (48:22) - Moore's law (55:20) - Hardware for deep learning (1:02:14) - Making neural networks fast at scale (1:09:51) - Andrej Karpathy and Chris Lattner (1:14:05) - How GPUs work (1:18:12) - Tesla Autopilot, NVIDIA, and Mobileye (1:22:52) - Andrej Karpathy and Software 2.0 (1:29:13) - Tesla Dojo (1:31:49) - Neural networks will understand physics better than humans (1:34:02) - Re-engineering the human brain (1:38:56) - Infinite fun and the Culture Series by Iain Banks (1:40:50) - Neuralink (1:46:13) - Dreams (1:50:06) - Ideas (2:00:19) - Aliens (2:05:16) - Jordan Peterson (2:10:13) - Viruses (2:13:22) - WallStreetBets and Robinhood (2:21:25) - Advice for young people (2:23:15) - Human condition (2:25:43) - Fear is a cage (2:30:34) - Love (2:36:57) - Regrets
The podcast host is advertising Athletic Greens and Bell Campo sustainably raised meats, offering a free one-month supply of fish oil and 20% off the first purchase respectively.
This podcast explores the significance of engineering when it comes to building software and hardware systems, and the role it plays in the effectiveness of theory and invention.
The speaker discusses the history of JavaScript and how it differed from other managed code frameworks at the time, which were all in a separate external container. He emphasizes the importance of releasing a simple tool at the right time and iterating based on user feedback.
An overview of ARM's processors for different design points and the dominance of PCs in the technology industry, as well as Intel's market share and Microsoft's approach to winning the browser game.
The host discusses the contrasting personality traits of Elon Musk and Steve Jobs, highlighting Jobs' tendency to be emotional, angry, and passionate while making decisions, while Musk has a love and passion for engineering and troubleshooting.
The debate of whether to prioritize being tough or being kind as a leader can be challenging, and often people who have experienced childhood trauma believe that it is important to be kind first. However, some argue that once a direction is set towards order, the force vector to move forward becomes unstoppable.
A story of a man who wanted to explain how a floating point worked, but first he took time to explain how the entire system worked in order to fully grasp the concept.
The challenge in a big organization is balancing the people who like to generate perfect ideas with those who want to get things done right away. Architects may come up with great ideas but often put them aside and move on to the next project.
The speaker explores the possibilities of building beautiful, complex systems with modular design through computing hardware and software design, referencing the internet and social networking sites like Twitter as examples.
As computers continue to get better at a rapid pace, the ability to scale from one to thousands of computers poses challenges in terms of selecting the right network for optimal performance. The advancements may come from a mix of innovations in hardware and software rather than just reducing the size of transistors.
The speaker explains how running a GPU program on each pixel allows for complex graphics processing, with each group of pixels running a simple program. As technology has advanced, more transistors have allowed for greater complexity.
Chip Huyen, a machine learning engineer and teacher, talks about hardware computation and her work on MLIR, which is a project on the mid-level intermediate representation of computation and coordination of computers.
The process of sending packets to processors and executing them in a data flow graph, while using an Ethernet header in a network, and the length of time it takes to get a network running with a new network card.
The hardware-software boundary is sometimes hard to define, but there are cases where hardware is better suited to handle certain tasks than software. For example, when executing a graph or rasterizing a triangle, it makes more sense for the hardware to handle it.
The importance of neural networks in autonomous driving development and how different companies approach it, including Tesla's ground-up approach versus Mobileye's less prioritized implementation.
To improve the neural network, constant improvement of the data landscape is necessary. Companies should weigh the benefits of building their own neural network training hardware from scratch.
The number of hierarchy steps in the human mind is not obvious, and dedicating more hardware to holding information may not enhance our ability to form a singular identity. While animals have basic instincts, predators have the ability to solve differential equations to catch prey.
A discussion on the subconscious mind and the potential for virtual reality technology to manipulate and explore it.
The speaker shares his strategy of preparing for finals by avoiding new material the day before and discusses the experience of dreaming.
In this podcast, the speaker discusses the benefits of curiosity and creativity when it comes to problem solving. They explain how these skills can help you approach problems from new angles and increase your ability to find solutions even in the face of uncertainty.
The possibility of consciousness in AI raises questions about what inputs are needed for consciousness to arise. While creating smart beings is fascinating, the process raises ethical and philosophical questions.
There are risks associated with expanding the human mind, including the potential for infinite suffering, as opposed to the infinite possibilities for fun. One doctor realized he had been harming his patients with a 10-year prescription of certain medications.
The recent GameStop stock frenzy has left many wondering how a group of retail investors on Reddit were able to take on a hedge fund. While the full story may never be known, it's clear that a distributed group of people were able to challenge centralized powers in a unique way.
The GameStop stock market frenzy was a battle between amateurs and experts, resulting in a financial risk that was outside the capacity of both parties. It also exposed weaknesses in the understanding of the financial system by some individuals involved.
The last few financial crises were set off by derivative events that tried to lower volatility in the short run but created tail events that triggered financial crashes. The same goes for personal events like upbringing that can have long-lasting effects on people's lives.
The key to building a successful startup team is hiring people who understand people and can create a culture of collaboration, engagement and a healthy work-life balance. It's important to prioritize strong relationships with family and friends as well as finding team members who are passionate about solving big problems.
The podcast discusses the different types of love, including physical, emotional, intellectual, and spiritual, as well as exploring Bukowski's idea of love being a fog that disappears with the light of the morning.
The value in understanding and analyzing people lies in the potential for personal growth and becoming a better version of oneself, rather than solely seeking short-term infatuation.
The effectiveness of coordinating thousands of people towards a common goal can overcome seemingly impossible tasks; however, it demands a solution that is political in nature and requires a certain level of kindness and direction. In this transcript excerpt, Bob Jimmiller discusses how he hopes to be remembered in a single sentence legacy entry.
In this podcast, Jim Keller talks about his upcoming project Tenestorrent and how it will revolutionize the computing space. He also shares his thoughts on the future of computing and the importance of imagination in creating the seemingly impossible.