Poker Bot: 'A Nuclear Weapon For Poker' Part Two

Tuomas Sandholm is Professor at Carnegie Mellon University in the Computer Science Department with more than 450 published papers. Over the past decade, he has applied his computational game theory knowledge and optimization algorithms to develop a computer program, or “bot.” In 2014 the bot he developed, Tartanian7, won both categories it entered in the Association for the Advancement of Artificial Intelligence (AAAI) Annual Computer Poker Competition, beating each opponent along the way with statistical significance.

In this, the second part of Card Player’s interview with Prof. Sandholm, he discusses the possibility of man vs. machine match with Tartanian7, how the bot could be used as a teaching tool and more. The first part of the story can be found here.

Man vs. Bot

Since starting work on the project in 2005 that eventually produced Tartanian7 in 2014, four Ph.D. students have assisted Sandholm. In the last year they’ve worked full time, with super computing time spent on the program being somewhere between 1 and 2 million core hours.

Sandholm expects that the approach his team used to build this bot will make it dominant at heads-up no-limit hold’em versus any possible opponent.

It has already proved true against other top bots, but how would it fare against human professionals who specialize in the heads-up format and make a living playing the game?

“It’s currently unknown whether the best program, which is ours, is better than the top human professionals. I’d conjecture that it is, but there has not been a controlled man vs. machine match yet.”

Over the past few decades computer scientists have tested the capacity of their game playing against top human players, with famous matches like Garry Kasparov vs. IBM’s super computer Deep Blue in chess and poker pros Phil Laak and Ali Eslami taking on the University of Alberta’s limit-hold’em heads-up bot Polaris. At first humans were able to compete, but eventually the programs gained the advantage.

Sandholm is hoping to set up a similar test for Tartanian7 in the future. “The tricky thing for a match is that you need to play a lot of hands. I would say at minimum 10,000 hands before you can tell who is better,” said Sandholm. “Nash equilibrium is unbeatable. We have an approximation of the Nash equilibrium, so our bot is still open to being beaten, but I think it will be very hard to find it’s leaks.”

Teaching Machine

The approach to learning poker since the start of the new millennium has changed in many ways. While most people used to just play low-stakes games to learn or read poker strategy books, modern day players have access to far more sophisticated tools. These included card-up video instruction from top pros and tools that analyze online poker play and spit out hard, statistical data, such as preflop raise percentage (PFR%) and the percentage of time you voluntarily put money in the pot (VPIP%).

While these tools, along with analyzing human professional play, is important, Sandholm thinks all of the best players will learn the game from playing against bots like Tartanian7 in the future.

“This bot has so much to tell people about how to play poker that it’s ridiculous,” said Sandholm with near breathless excitement. “It plays poker very differently from how humans play poker. Humans learn from each other how humans play the game, not how it is optimally played. This bot, in contrast, has never seen a human play poker. Instead, it has reasoned from first principles how poker should be played and the conclusions are different from what humans have reached.”

Sandholm explained that by reasoning from first principals he simply means that the bot determined its strategy purely from the rules of the game, using the game-theoretic solution concept of Nash equilibrium. It doesn’t base its play on any historical experience against humans or other bots. It’s really figuring out how to best play the game from just the rules.

With this different approach the bot, Sandholm believes, has found a different way to play that could be very instructive for humans to learn from.

“People could play against the bot and learn from observation and practice, or even play against it and be able to ask it for advice as far as what it would do if it where in their shoes, so we will be able to make a very cool training tool out of this bot.”

When asked for an example of how the bot approaches heads-up no-limit hold’em differently then most human players, Sandholm first noted that it has a far more varied approach to bet sizing, one of the most important and skillful aspects of the game.

“Humans typically use a small number of bet sizes in any given situation. Our bot does not do that, it uses a large range of bet sizes in a given situation with the bets sometimes being smaller or much larger than humans would conventionally use,” said Sandholm.

“I think by using one or two bet sizes humans can avoid signaling too much about the strength of their hand,“ Sandholm continued. “But the computer can use a larger number of bet sizes because it knows it isn’t giving away too much because it balances its bets. Another example is limping, by the way. Limping in on the button heads-up no-limit is often considered by humans as a novice thing to do, but the bot will do that.”

Sandholm is convinced that the way they went about building the bot’s strategy makes it the best heads-up no-limit hold’em player in the world, and thus the best to learn from for aspiring players.

“Now that being said, a player trained by our bot in terms of strategy might still give off tells, but setting that aside,” Sandholm said with slight laugh.

Looking Forward

With demonstrative wins in two categories at the AAAI Annual Computer Poker Competition Sandholm and his bot have laid the groundwork for many exciting developments in the future. With hopes for a match between Tartanian7 and a top human player sometime in future and also the development of a training tool using the bot and it’s strategy, there is plenty to look forward to in the small slice of the world of computer science that intersects with poker. Who knows how that intersection might change the face of the game in the coming years.

Related Articles

Venetian Rings In New Year With $1.1M DeepStack

Caesars Unveils New Virginia Casino With Poker Room

CoinPoker Partners With Raise Your Edge Training

Most Viewed Articles