Episode
#344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation
2:33:57
Published: Tue Dec 06 2022
Description
Noam Brown is a research scientist at FAIR, Meta AI, co-creator of AI that achieved superhuman level performance in games of No-Limit Texas Hold'em and Diplomacy. Please support this podcast by checking out our sponsors: - True Classic Tees: https://trueclassictees.com/lex and use code LEX to get 25% off - Audible: https://audible.com/lex to get 30-day free trial - InsideTracker: https://insidetracker.com/lex to get 20% off - ExpressVPN: https://expressvpn.com/lexpod to get 3 months free EPISODE LINKS: Noam's Twitter: https://twitter.com/polynoamial Noam's LinkedIn: https://www.linkedin.com/in/noam-brown-8b785b62/ webDiplomacy: https://webdiplomacy.net/ Noam's papers: Superhuman AI for multiplayer poker: https://par.nsf.gov/servlets/purl/10119653 Superhuman AI for heads-up no-limit poker: https://par.nsf.gov/servlets/purl/10077416 Human-level play in the game of Diplomacy: https://www.science.org/doi/10.1126/science.ade9097 PODCAST INFO: Podcast website: https://lexfridman.com/podcast Apple Podcasts: https://apple.co/2lwqZIr Spotify: https://spoti.fi/2nEwCF8 RSS: https://lexfridman.com/feed/podcast/ YouTube Full Episodes: https://youtube.com/lexfridman YouTube Clips: https://youtube.com/lexclips SUPPORT & CONNECT: - Check out the sponsors above, it's the best way to support this podcast - Support on Patreon: https://www.patreon.com/lexfridman - Twitter: https://twitter.com/lexfridman - Instagram: https://www.instagram.com/lexfridman - LinkedIn: https://www.linkedin.com/in/lexfridman - Facebook: https://www.facebook.com/lexfridman - Medium: https://medium.com/@lexfridman OUTLINE: Here's the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time. (00:00) - Introduction (05:37) - No Limit Texas Hold 'em (09:30) - Solving poker (22:40) - Poker vs Chess (29:18) - AI playing poker (1:02:46) - Heads-up vs Multi-way poker (1:13:37) - Greatest poker player of all time (1:17:10) - Diplomacy game (1:27:01) - AI negotiating with humans (2:09:26) - AI in geopolitics (2:14:11) - Human-like AI for games (2:20:12) - Ethics of AI (2:24:26) - AGI (2:28:25) - Advice to beginners
Chapters
This podcast episode discusses the power of simple stories in conveying profound messages, as well as an AI system that out-negotiates humans in the Diplomacy board game using natural language.
00:00 - 03:01 (03:01)
Summary
This podcast episode discusses the power of simple stories in conveying profound messages, as well as an AI system that out-negotiates humans in the Diplomacy board game using natural language.
Episode#344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation
PodcastLex Fridman Podcast
The development of AI has reached new levels of success, as many systems have solved or achieved human-level performance on classic strategy games such as Diplomacy and No Limit Texas Hold'em Poker.
03:01 - 07:33 (04:31)
Summary
The development of AI has reached new levels of success, as many systems have solved or achieved human-level performance on classic strategy games such as Diplomacy and No Limit Texas Hold'em Poker.
Episode#344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation
PodcastLex Fridman Podcast
There exists an optimal strategy in any finite two-player zero-sum game, where players are guaranteed not to lose an expectation regardless of their opponent's moves.
07:33 - 12:14 (04:40)
Summary
There exists an optimal strategy in any finite two-player zero-sum game, where players are guaranteed not to lose an expectation regardless of their opponent's moves. This applies to games like chess, poker, and even rock-paper-scissors if players randomly choose between throwing rock-paper-scissors with equal probability.
Episode#344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation
PodcastLex Fridman Podcast
In two-player zero sum games, there is no Nash equilibrium if players seek to name the larger number.
12:14 - 17:21 (05:06)
Summary
In two-player zero sum games, there is no Nash equilibrium if players seek to name the larger number. However, for games like Risk or poker, Nash equilibria exist but may not be as relevant in practice due to sponsorship and personality factors.
Episode#344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation
PodcastLex Fridman Podcast
Self-play allows for perfect information game optimization through generalization of past experiences, while algorithms can start from random play and learn through continuous self-play.
17:21 - 22:46 (05:25)
Summary
Self-play allows for perfect information game optimization through generalization of past experiences, while algorithms can start from random play and learn through continuous self-play.
Episode#344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation
PodcastLex Fridman Podcast
Reputation plays an important role in decision making because it affects the perception of our actions.
22:46 - 27:42 (04:56)
Summary
Reputation plays an important role in decision making because it affects the perception of our actions. In poker for example, having a reputation for bluffing may make it less likely for others to believe a bluff, while not having a reputation for bluffing may make a bluff more successful.
Episode#344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation
PodcastLex Fridman Podcast
The key to winning in poker is not necessarily being unpredictable, but rather understanding your opponent's predictability and how to take advantage of it.
27:42 - 32:17 (04:34)
Summary
The key to winning in poker is not necessarily being unpredictable, but rather understanding your opponent's predictability and how to take advantage of it. This involves playing a balanced strategy and anticipating how your opponent will react to your moves.
Episode#344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation
PodcastLex Fridman Podcast
In 2015, poker playing AI bots focused on pre-computing a strategy while today they use machine learning algorithms to adjust their strategy based on real-time gameplay.
32:17 - 39:07 (06:50)
Summary
In 2015, poker playing AI bots focused on pre-computing a strategy while today they use machine learning algorithms to adjust their strategy based on real-time gameplay. An AI poker bot lost to professional players in a 2015 competition, demonstrating the limitations of pre-computed strategies.
Episode#344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation
PodcastLex Fridman Podcast
Discover how maximizing the value of your hand in poker involves understanding what you would be doing with other hands as well, and how a little search can go a long way towards creating a successful pre-computed strategy.
39:07 - 45:22 (06:14)
Summary
Discover how maximizing the value of your hand in poker involves understanding what you would be doing with other hands as well, and how a little search can go a long way towards creating a successful pre-computed strategy.
Episode#344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation
PodcastLex Fridman Podcast
Monte Carlo tree search, which involves thinking several moves ahead in a game, can be successful in perfect information board games like chess and Go, but is not as effective in games like poker.
45:22 - 54:10 (08:48)
Summary
Monte Carlo tree search, which involves thinking several moves ahead in a game, can be successful in perfect information board games like chess and Go, but is not as effective in games like poker.
Episode#344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation
PodcastLex Fridman Podcast
The team at Covariant.ai tells us how it's teaching robots to manipulate objects more flexibility and some of the challenges along the way.
54:10 - 59:59 (05:48)
Summary
The team at Covariant.ai tells us how it's teaching robots to manipulate objects more flexibility and some of the challenges along the way.
Episode#344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation
PodcastLex Fridman Podcast
The techniques used in two-player poker to approximate an equilibrium still work in practice for six-player poker despite not offering guarantees outside of two-player zero-sum games, leading to debate among academics and the poker community.
59:59 - 1:08:22 (08:23)
Summary
The techniques used in two-player poker to approximate an equilibrium still work in practice for six-player poker despite not offering guarantees outside of two-player zero-sum games, leading to debate among academics and the poker community.
Episode#344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation
PodcastLex Fridman Podcast
In the latest and greatest poker AIs, neural nets are used heavily for the value function as they are powerful tools in situations where finding features is difficult.
1:08:22 - 1:17:08 (08:45)
Summary
In the latest and greatest poker AIs, neural nets are used heavily for the value function as they are powerful tools in situations where finding features is difficult. Due to the need to reason about beliefs in poker, the way neural nets are used in the game is different from how they are used in chess or Go.
Episode#344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation
PodcastLex Fridman Podcast
Diplomacy is a strategy game with a role playing component which allows players to act out as leaders of different countries in history.
1:17:08 - 1:21:19 (04:10)
Summary
Diplomacy is a strategy game with a role playing component which allows players to act out as leaders of different countries in history. The social aspect of the game makes it equally important as the game theory component for winning, making it a potential artificial intelligence problem.
Episode#344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation
PodcastLex Fridman Podcast
Compared to other games such as chess or poker, diplomacy offers incredibly complex conversations and strategies that players have to master.
1:21:19 - 1:30:10 (08:51)
Summary
Compared to other games such as chess or poker, diplomacy offers incredibly complex conversations and strategies that players have to master. In the ideal version of the game, no one actually wins, which adds an extra level of intrigue to the gameplay.
Episode#344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation
PodcastLex Fridman Podcast
The Turing test and game-playing AI show that to compete with humans, machines need to understand human behavior and adapt to it.
1:30:10 - 1:39:35 (09:24)
Summary
The Turing test and game-playing AI show that to compete with humans, machines need to understand human behavior and adapt to it. This understanding is what allowed AI to achieve superhuman performance in games like chess and Go.
Episode#344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation
PodcastLex Fridman Podcast
Top diplomacy players say that diplomacy is a game about trust, where players must build trust in an environment designed to discourage trust.
1:39:35 - 1:49:06 (09:31)
Summary
Top diplomacy players say that diplomacy is a game about trust, where players must build trust in an environment designed to discourage trust. Researchers have created a language model that is controllable on a set of "intense" actions, which allows players to plan which action to play and persuade the other player to execute the desired action, making it easier to build trust in the game.
Episode#344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation
PodcastLex Fridman Podcast
A research study showed that an AI bot was able to surpass human players in a zero-sum version of the game of diplomacy, leading experts to reconsider traditional approaches to the game.
1:49:06 - 1:59:12 (10:06)
Summary
A research study showed that an AI bot was able to surpass human players in a zero-sum version of the game of diplomacy, leading experts to reconsider traditional approaches to the game. The bot's unconventional strategies proved to be more effective than those employed by humans.
Episode#344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation
PodcastLex Fridman Podcast
The self-play process in diplomacy involves conditioning the language model on good intents and deviating from human anchor policy if there is an action with high expected value.
1:59:12 - 2:09:20 (10:08)
Summary
The self-play process in diplomacy involves conditioning the language model on good intents and deviating from human anchor policy if there is an action with high expected value. Additionally, pre-training the language model on internet data aids in approximating human play.
Episode#344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation
PodcastLex Fridman Podcast
The development of human-like AI systems with different styles of humans requires stronger cheat detection systems in order to ensure that human versus human games are played in a deeply fair way.
2:09:20 - 2:21:47 (12:27)
Summary
The development of human-like AI systems with different styles of humans requires stronger cheat detection systems in order to ensure that human versus human games are played in a deeply fair way.
Episode#344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation
PodcastLex Fridman Podcast
The development of AI systems that can generate language, images, and play games like diplomacy is transforming human society.
2:21:47 - 2:28:25 (06:37)
Summary
The development of AI systems that can generate language, images, and play games like diplomacy is transforming human society. The potential of applying RL methods towards the real world is fascinating, as it can change the way we live our lives.
Episode#344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation
PodcastLex Fridman Podcast
Math skills, particularly in linear algebra and statistics, are crucial to effectively learn and understand machine learning.
2:28:25 - 2:34:06 (05:41)
Summary
Math skills, particularly in linear algebra and statistics, are crucial to effectively learn and understand machine learning. Understanding the reward function is more important than defining the actual policy to achieve it in order to minimize unintended consequences.