A new artificial intelligence framework known as Recursive Belief-based Learning (ReBeL) has been created by Facebook developers which proves itself by mastering at a game that has deemed to be difficult for AI programs: Texas Hold’em poker. ReBeL implements new concepts that allow it to better handle the partial information aspects of poker, and even outperform a previous superhuman poker AI, Libratus.
AI systems in recent years have improved in cracking several complex games. DeepMind’s AlphaZero program would teach itself chess and such, use self-play to reach new heights in similar games in a matter of hours. Libratus, as well, through self-play showed an amazing ability to learn Heads-Up NLH. ReBeL does the same but infuses a new notion altogether constituting a ‘game state’, allowing the AI to better understand hidden information during self-play.
ReBeL considers information about the visible game state, like the known cards, bet sizing, and even the range of hands the opponent might have. Additionally, it considers each players’ belief about the state they’re in, similar to how anyone might consider whether an opponent thinks they’re ahead of behind in a hand.
As per what researchers say, ReBeL has aced the imperfect-information games, thanks to the very approach. The Facebook team did experiments in which the framework played 2-player versions of Hold’em, Turn Endgame Hold’em – a simplified version of the game with no raises on the first two betting rounds – and Liar’s Dice. The result is an AI you wouldn’t wish to face ever across the online felts. ReBeL defeated heads-up specialist Dong Kim by 165 thousandths of a big blind per hand over a 7,500-hand match. That’s higher than the 147 thousandths of a big blind by which Libratus defeated four human players in 2017! And if you’re worried you might run into an opponent running ReBeL online, researchers have taken precautions against that happening.
“The most immediate risk posed by this work is its potential for cheating in recreational games such as poker,” the team wrote in its paper. “Partly for this reason, we have decided not to release the code for poker.”
The developers believe that the ReBeL framework can help develop better general equilibrium-finding algorithms with applications in auctions, negotiations, cybersecurity, and self-driving vehicles, among other areas.