Championship Poker Bot: 'A Nuclear Weapon For Poker'Professor Tuomas Sandholm Discusses His Heads-Up No-Limit Hold'em Bot 'Tartanian7' And What It Means For The Future Of Poker |
|
Tuomas Sandholm is Professor at Carnegie Mellon University in the Computer Science Department with more than 450 published papers. Over the past decade, he has applied his computational game theory knowledge and optimization algorithms to develop a computer program, or “bot.”
Sandholm’s objective was to design the greatest heads-up no-limit Texas hold’em player in the world and, possibly, the ultimate training tool for that particular form of poker.
When asked if he thought, in the future, if learning from what his program has discovered would be mandatory for all those wanting to compete at the top of the game, Sandholm didn’t mince words.
“I think so. It’s a bit of a nuclear weapon for poker. You don’t want to be bringing a knife to a gun fight.”
Certainly a strong statement, but Sandholm’s success with his program lends weight to the words. This year his bot won both categories of competition for heads-up no-limit at the Association for the Advancement of Artificial Intelligence (AAAI) Annual Computer Poker Competition, beating each opponent along the way with statistical significance.
The idea of a ‘poker bot’ has captivated the collective consciousness of the poker world since the inception of online poker. For the dubious, the idea that they could be playing poker against a machine was disconcerting. For futurists the possibility of applying computer science to the game was exciting. The truth is that people have been creating bots and running them in online games for at least a decade, with the first commercially available program, called “WinHoldEm,” even being featured prominently in tech magazine Wired.
The New York Times even wrote a feature on bots in 2011, with the conclusion of the article being that, for now, most human players should not fear playing against bots. One reason is because the top online poker sites have sophisticated methods of identifying robotic players and actively hunt for them.
But Prof. Sandholm also notes that while the bots out there might be sophisticated enough to be winning players, they aren’t nearly as strong as the top bots that he has created and competed against. “They aren’t very good. The current ones that are out there now, some of them have participated in the AAAI competition and turned out to be not very good.“
So, bots are currently out in the marketplace, but for now researchers at university computer science programs mostly develop the very good ones. Card Player recently spoke to Sandholm at length about the competition, the development of his bot, and what’s in store for this particular program in the future.
Designing ‘Tartanian7’ To Dominate
The ninth annual Computer Poker Competition consisted of six categories, two of which featuring no-limit hold’em. Approximately 50 million hands were played. The categories all used duplicate matches, which in the case of two-player games meant that they played an ordinary heads-up match of a set number of hands between two bots, with the specific cards dealt being recorded. At the end of the specified number of hands the bot’s memories are reset, they switch positions and play the set of hands over with the other player’s cards.
“The competition has two categories in heads-up no-limit Texas hold’em,” elaborated Prof. Sandholm. “One is called ‘total bankroll,’ in which the bots play against each other and whichever bot wins the most virtual money wins. The other category is called ‘instant runoff,’ in which you again have all of the bots play each other and throw out the one that loses the most and then play another round and so on until there is one bot remaining.”
Sandholm said these two categories measure different strengths. “The total bankroll measures mostly how well you do against the weak opponents and the elimination measures mostly how well you do against the strong opponents,” he explained.
Sandholm’s program, officially entered by him and his Ph.D students Noam Brown and Sam Ganzfried, was named “Tartanian7”. Sometimes teams make variations of their program for the two separate hold’em categories, but the same version of Tartanian7 won both categories and, impressively, was the best against every single one of the opponents with statistical significance.
When asked if the success of his program had anything to do with his own experience playing strategy games and poker Sandholm quickly noted that, “we did not program in any human poker playing knowledge. Our approach was purely algorithmic and our algorithms created the program.”
“They key was really our superior knowledge of computational game theory and optimization algorithms,” said Sandholm. “More specifically, all of the top bots were created with the following kind of framework that my student Andrew Gilpin and I put together in 2005. You first run an abstraction algorithm that generates a smaller, but strategically similar, game so that we can then use an equilibrium-finding algorithm that finds a Nash equilibrium or an approximate Nash equilibrium for the smaller game. The original game (in this case, heads-up no-limit hold’em) is far too large to solve for Nash equilibrium. So we solve the smaller, abstract game, and then we use a reverse-mapping algorithm to map the answer from the abstract game back to the original game.”
Proposed by John Forbes Nash, the Nobel Prize winning mathematician who was the subject of the Hollywood movie A Beautiful Mind, the concept of a Nash equilibrium essentially is a solution in a non-cooperative game in which the player has no reason to diverge from their strategy given an opponent’s approach. In poker, it would be a game-theoretic strategy that would work against any approach your opponent presents.
Having found what he considers to at least be an approximation of the Nash equilibrium for heads-up no-limit hold’em, Sandholm entered the same bot into both categories.
“We did not do any opponent exploitation this year. We have done research on the subject and have used it in some previous years, specifically in the total bankroll competition, because there you can do well using opponent exploitation by taking advantage of the weakest opponents. But this year we didn’t do any of that. We just had an approximately game-theoretic solution and we used that to win both categories, playing with a static strategy throughout. That you could win both competitions with the same program, with the same exact program that plays superbly against both weak and strong opponents, is something I find interesting.”
Make sure to check back on CardPlayer.com for the second part of this article, in which Sandholm discusses the possibility of man vs. machine match with Tartanian7, how the bot could be used as a teaching tool and more.
Copyright © 2001-2024
CardPlayer.com. All rights reserved.
