Bot makes poker pros fold: What's next for artificial intelligence?

Carnegie Mellon's win at Texas Hold 'em Poker heralds a big leap for artificial intelligence.

Andrew Rush/Pittsburg Post-Gazette/AP
Computer science Ph.D. student Noam Brown (center) helps professional poker player Daniel McAulay get started during the 'Brains Vs. Artificial Intelligence: Upping the Ante' rematch at Rivers Casino in Pittsburgh.

Not even four top poker players could out-bluff Libratus.

Carnegie Mellon’s No-Limit Texas Hold 'em software made short work of four of the world’s best professional poker players in Pittsburgh at the grueling "Brains vs. Artificial Intelligence" poker tournament.

Poker now joins chess, Jeopardy, go, and many other games at which programs outplay people. But poker is different from all the others in one big way: players have to guess based on partial, or "imperfect" information.

“Chess and Go are games of perfect information," explains Libratus co-creator Noam Brown, a Ph.D. candidate at Carnegie Mellon. "All the information in the game is available for both sides to see. Poker is a game of imperfect information, since neither player can see their opponent’s cards,” he writes in an email to The Christian Science Monitor.

In other words, no matter how impressive your card skills, your power of prediction is limited by Texas Hold 'em because only certain cards are visible. An ace in your hand and an ace on the table might look like good news, unless your opponent is holding a pair of aces, or worse. No amount of computational power can answer that question.

“The typical approach to addressing perfect-information games [like chess] was to search through a game tree to find an optimal path,” he continues. But that’s no good in poker, because without knowing what cards are where, you can’t even figure out where on the tree you are.

Libratus is a whole new kind of machine.

Rather than try to sort through an unknowable tree, it focuses on finding a favorable move that represents a so-called “Nash equilibrium” solution, named for Nobel Laureate John Nash of “Beautiful Mind” fame.

The Nash equilibrium is often explained with the Prisoner’s Dilemma, which describes a situation pitting two criminal suspects against each other, incentivized to betray and punished for loyalty. Most people instinctively hit on the mathematically rational solution of betraying the co-conspirator and accepting a medium penalty, even though it isn’t the ideal solution.

The key is that even without knowing whether the co-conspirator will betray you or not, a rational strategy exists. You won’t always end up in the best possible situation, but you may be able to avoid the worst outcome.

Libratus plays poker in a similar way, never guaranteed to win any particular hand but likely to stay in the black over the long run.

The result was a poker tournament unlike any other. People tend to bet and bluff in certain increments, and noticing those patterns helps professionals find an edge, but the computer was hard to read.

The “biggest difference is that most human poker players do what Libratus does, betting with multiple bet sizes, but humans only have one or two usually. Libratus mixes in a bunch, maybe 10 or 15, and even mixes in situations that I thought didn't make sense before this competition started. It would be way too complicated for a human to do this sort of thing correctly,” Dong Kim, the highest performing of the four human players said in a Reddit interview.

As Libratus was analyzing the day’s games each night, the pros did the same. While they weren’t able to find a consistent winning strategy, they suggest the experience made them better poker players. “Once you face Libratus, there's nothing worse any human could ever do to you. Every human is going to seem like a walk in the park,” said Jason Les, another one of the players.

At the end of 120,000 poker hands played over almost three weeks, Liberatus was up 1.77 million poker chips, and all human players were in the red, according to host Rivers Casino. Fortunately for the participants, they won’t have to hand over any money. Instead, they’ll split a $200,000 performance-based reward designed to elicit their best play.

It was an absolute beatdown. The human team lost at a rate of approximately 2.5 times what the program lost at before,” at the previous tournament, said professional player Doug Polk, reported Forbes.

The size of the win was surprising, even to the program’s creators, Tuomas Sandholm and Mr. Brown. “I think everyone expected computers to eventually outdo humans in poker, but the speed at which it happened was definitely surprising. Just 20 months earlier our precursor bot Claudico lost by a fairly wide margin. This time around, we won by a far wider margin. Nobody expected this, not even us. The international betting sites had us as 4:1 underdogs going into the competition,” Brown says.

The imperfect information nature of poker makes the win a huge achievement for the AI community, with far-reaching real world applications that Brown says include negotiations, auctions, and security interactions, to name a few. “In truth, most real-world scenarios involve hidden information. In the real world, not all the information is laid out neatly for all sides to see like pieces on a chessboard. There is uncertainty and deception,” he explains.

One remaining enclave of human superiority is multiplayer situations, however. Liberatus prefers bilateral dealing to group negotiation. It took on each poker player one-at-a-time, and couldn't have won in an ensemble game. Still, two-sided negotiations are common in the real world, and Libratus lays the foundations for computer programs that help negotiators elevate the art of the deal to a science. 

Brown continues, “None of the algorithms in Libratus are specific to poker. We did not program it to play poker. We programmed it to learn any imperfect-information game, and fed it the rules of No-Limit Texas Hold'em as a way to evaluate its performance.”

With AI quickly learning to navigate both go’s practically infinite possibility tree, and poker’s fog of war, even the experts are running out of milestones to topple. Writing to the Monitor from a workshop called “What’s next for AI in games,” Brown had this to say about the future of artificial intelligence: “I think many people in AI are asking what comes next, and the answer isn't clear. But either way it's clear that AI will continue to make major advances in the coming years.”

You've read  of  free articles. Subscribe to continue.
Real news can be honest, hopeful, credible, constructive.
What is the Monitor difference? Tackling the tough headlines – with humanity. Listening to sources – with respect. Seeing the story that others are missing by reporting what so often gets overlooked: the values that connect us. That’s Monitor reporting – news that changes how you see the world.

Dear Reader,

About a year ago, I happened upon this statement about the Monitor in the Harvard Business Review – under the charming heading of “do things that don’t interest you”:

“Many things that end up” being meaningful, writes social scientist Joseph Grenny, “have come from conference workshops, articles, or online videos that began as a chore and ended with an insight. My work in Kenya, for example, was heavily influenced by a Christian Science Monitor article I had forced myself to read 10 years earlier. Sometimes, we call things ‘boring’ simply because they lie outside the box we are currently in.”

If you were to come up with a punchline to a joke about the Monitor, that would probably be it. We’re seen as being global, fair, insightful, and perhaps a bit too earnest. We’re the bran muffin of journalism.

But you know what? We change lives. And I’m going to argue that we change lives precisely because we force open that too-small box that most human beings think they live in.

The Monitor is a peculiar little publication that’s hard for the world to figure out. We’re run by a church, but we’re not only for church members and we’re not about converting people. We’re known as being fair even as the world becomes as polarized as at any time since the newspaper’s founding in 1908.

We have a mission beyond circulation, we want to bridge divides. We’re about kicking down the door of thought everywhere and saying, “You are bigger and more capable than you realize. And we can prove it.”

If you’re looking for bran muffin journalism, you can subscribe to the Monitor for $15. You’ll get the Monitor Weekly magazine, the Monitor Daily email, and unlimited access to

QR Code to Bot makes poker pros fold: What's next for artificial intelligence?
Read this article in
QR Code to Subscription page
Start your subscription today