Heads Up No Limit

broken image


Expert-Level Artificial Intelligence in Heads-Up No-Limit Poker

  1. Heads Up No Limit Hold'em Book

Get free heads up no limit holdem preflop charts for 100bb stacks. Learn a fundamentally sound strategy for which hands to raise, 3bet and more. At No Limit Holdem, he won around $5.38 million, mostly at the heads-up tables. Currently, poker is off-limits in the US, Cates is playing online poker from Romania - his girlfriend is from there - under his PS nickname ‘w00ki3z.' He can be seen at the Heads-Up SNG tables as well as the cash tables (mostly mixed games).

Links

Heads-up no-limit texas hold’em

Twitch | YouTube | Twitter
Downloads & Videos | Media Contact

DeepStack bridges the gap between AI techniques for games of perfect information—like checkers, chess and Go—with ones for imperfect information games–like poker–to reason while it plays using 'intuition' honed through deep learning to reassess its strategy with each decision.

With a study completed in December 2016 and published in Science in March 2017, DeepStack became the first AI capable of beating professional poker players at heads-up no-limit Texas hold'em poker.

DeepStack computes a strategy based on the current state of the game for only the remainder of the hand, not maintaining one for the full game, which leads to lower overall exploitability.

DeepStack avoids reasoning about the full remaining game by substituting computation beyond a certain depth with a fast-approximate estimate. Automatically trained with deep learning, DeepStack's 'intuition' gives a gut feeling of the value of holding any cards in any situation.

DeepStack considers a reduced number of actions, allowing it to play at conventional human speeds. The system re-solves games in under five seconds using a simple gaming laptop with an Nvidia GPU.

The first computer program to outplay human professionals at heads-up no-limit Hold'em poker

In a study completed December 2016 and involving 44,000 hands of poker, DeepStack defeated 11 professional poker players with only one outside the margin of statistical significance. Over all games played, DeepStack won 49 big blinds/100 (always folding would only lose 75 bb/100), over four standard deviations from zero, making it the first computer program to beat professional poker players in heads-up no-limit Texas hold'em poker.

Games are serious business

Don't let the name fool you, 'games' of imperfect information provide a general mathematical model that describes how decision-makers interact. AI research has a long history of using parlour games to study these models, but attention has been focused primarily on perfect information games, like checkers, chess or go. Poker is the quintessential game of imperfect information, where you and your opponent hold information that each other doesn't have (your cards).

Until now, competitive AI approaches in imperfect information games have typically reasoned about the entire game, producing a complete strategy prior to play. However, to make this approach feasible in heads-up no-limit Texas hold'em—a game with vastly more unique situations than there are atoms in the universe—a simplified abstraction of the game is often needed.

A fundamentally different approach

Heads

DeepStack is the first theoretically sound application of heuristic search methods—which have been famously successful in games like checkers, chess, and Go—to imperfect information games.

At the heart of DeepStack is continual re-solving, a sound local strategy computation that only considers situations as they arise during play. This lets DeepStack avoid computing a complete strategy in advance, skirting the need for explicit abstraction.

During re-solving, DeepStack doesn't need to reason about the entire remainder of the game because it substitutes computation beyond a certain depth with a fast approximate estimate, DeepStack's 'intuition' – a gut feeling of the value of holding any possible private cards in any possible poker situation.

Finally, DeepStack's intuition, much like human intuition, needs to be trained. We train it with deep learning using examples generated from random poker situations.

Heads
Heads up no limit strategy

Twitch | YouTube | Twitter
Downloads & Videos | Media Contact

DeepStack bridges the gap between AI techniques for games of perfect information—like checkers, chess and Go—with ones for imperfect information games–like poker–to reason while it plays using 'intuition' honed through deep learning to reassess its strategy with each decision.

With a study completed in December 2016 and published in Science in March 2017, DeepStack became the first AI capable of beating professional poker players at heads-up no-limit Texas hold'em poker.

DeepStack computes a strategy based on the current state of the game for only the remainder of the hand, not maintaining one for the full game, which leads to lower overall exploitability.

DeepStack avoids reasoning about the full remaining game by substituting computation beyond a certain depth with a fast-approximate estimate. Automatically trained with deep learning, DeepStack's 'intuition' gives a gut feeling of the value of holding any cards in any situation.

DeepStack considers a reduced number of actions, allowing it to play at conventional human speeds. The system re-solves games in under five seconds using a simple gaming laptop with an Nvidia GPU.

The first computer program to outplay human professionals at heads-up no-limit Hold'em poker

In a study completed December 2016 and involving 44,000 hands of poker, DeepStack defeated 11 professional poker players with only one outside the margin of statistical significance. Over all games played, DeepStack won 49 big blinds/100 (always folding would only lose 75 bb/100), over four standard deviations from zero, making it the first computer program to beat professional poker players in heads-up no-limit Texas hold'em poker.

Games are serious business

Don't let the name fool you, 'games' of imperfect information provide a general mathematical model that describes how decision-makers interact. AI research has a long history of using parlour games to study these models, but attention has been focused primarily on perfect information games, like checkers, chess or go. Poker is the quintessential game of imperfect information, where you and your opponent hold information that each other doesn't have (your cards).

Until now, competitive AI approaches in imperfect information games have typically reasoned about the entire game, producing a complete strategy prior to play. However, to make this approach feasible in heads-up no-limit Texas hold'em—a game with vastly more unique situations than there are atoms in the universe—a simplified abstraction of the game is often needed.

A fundamentally different approach

DeepStack is the first theoretically sound application of heuristic search methods—which have been famously successful in games like checkers, chess, and Go—to imperfect information games.

At the heart of DeepStack is continual re-solving, a sound local strategy computation that only considers situations as they arise during play. This lets DeepStack avoid computing a complete strategy in advance, skirting the need for explicit abstraction.

During re-solving, DeepStack doesn't need to reason about the entire remainder of the game because it substitutes computation beyond a certain depth with a fast approximate estimate, DeepStack's 'intuition' – a gut feeling of the value of holding any possible private cards in any possible poker situation.

Finally, DeepStack's intuition, much like human intuition, needs to be trained. We train it with deep learning using examples generated from random poker situations.

DeepStack is theoretically sound, produces strategies substantially more difficult to exploit than abstraction-based techniques and defeats professional poker players at heads-up no-limit poker with statistical significance.

Download

Paper & Supplements

Hand Histories

Members (Front-back)

Michael Bowling, Dustin Morrill, Nolan Bard, Trevor Davis, Kevin Waugh, Michael Johanson, Viliam Lisý, Martin Schmid, Matej Moravčík, Neil Burch

low-variance Evaluation

The performance of DeepStack and its opponents was evaluated using AIVAT, a provably unbiased low-variance technique based on carefully constructed control variates. Thanks to this technique, which gives an unbiased performance estimate with 85% reduction in standard deviation, we can show statistical significance in matches with as few as 3,000 games.

Abstraction-based Approaches

Despite using ideas from abstraction, DeepStack is fundamentally different from abstraction-based approaches, which compute and store a strategy prior to play. While DeepStack restricts the number of actions in its lookahead trees, it has no need for explicit abstraction as each re-solve starts from the actual public state, meaning DeepStack always perfectly understands the current situation.

Professional Matches

We evaluated DeepStack by playing it against a pool of professional poker players recruited by the International Federation of Poker. 44,852 games were played by 33 players from 17 countries. Eleven players completed the requested 3,000 games with DeepStack beating all but one by a statistically-significant margin. Over all games played, DeepStack outperformed players by over four standard deviations from zero.


Heuristic Search

At a conceptual level, DeepStack's continual re-solving, 'intuitive' local search and sparse lookahead trees describe heuristic search, which is responsible for many AI successes in perfect information games. Until DeepStack, no theoretically sound application of heuristic search was known in imperfect information games.

','resolveObject':','resolvedBy':'manual','resolved':true}'>
','resolvedBy':'manual','resolved':true}'>
','resolveObject':','resolvedBy':'manual','resolved':true}'>

On This Page

Introduction

Heads Up Hold 'Em is an Ultimate Texas Hold 'Em variant by Galaxy Gaming, based on Texas Hold 'Em. The player may raise his bet one time, and has three opportunities to do so. The earlier he raises, the more he can bet. The main differences between Heads Up Hold 'Em and Ultimate Texas Hold 'Em are in the former the player may raise only 3x his Ante bet before the flop, but the game includes bad beat bonuses for losing with a straight or higher.

Rules

Following are the rules for Heads Up Hold 'Em. For those used to the terminology in Ultimate Texas Hold 'Em, what is called the Blind there called the Odds bet here.

Enjoy Live Casino action with a casino classic, Craps. Play Live Craps right here in real time with real croupiers, in full high definition. Two dice are rolled, and you predict the outcome, with a variety of different betting options – it's as simple as that. You can play Live Craps 24 hours a day here at Grosvenor Casino – and you can also check out all our other terrific Live Games. Operated by two dealers, we have successfully brought to life one of the most enjoyable casino table games! Other Craps bets that you will encounter at the live and online Craps tables include field bets, hardways bets, proposition bets, big 8 bets, big 6 bets, and dozens of others. The easiest bets for beginners include the passline and don't passline bets. Live craps roll.

  1. The game is played with a single ordinary 52-card deck.
  2. The player must make an equal bet on both the Ante and Odds.
  3. Two cards are dealt face down to the player and dealer. The player may look at his own cards.
  4. The player can check or make a Play bet equal to three times the Ante.
  5. The dealer turns over three community cards.
  6. If the player previously checked, then he may make a Play bet equal to two times his Ante or check again. If the player already made a Play bet, then he may not bet further.
  7. Two final community cards are turned over.
  8. If the player previously checked twice, then he must either make a Play bet equal to exactly his Ante, or fold, losing both his Ante and Odds bets. If the player already raised he may not bet further.
  9. The player and dealer will both make the best possible hand using any combination of their own two cards and the five community cards.
  10. The dealer will need at least a pair to open.
  11. The following table shows how the Ante, Odds, and Play bets are scored, according to who wins, and whether the dealer opens.

    Scoring Rules

    WinnerDealer OpensAntePlayOdds
    PlayerYesWinWinSee rule 12
    PlayerNoPushWinSee rule 12
    DealerYesLoseLoseSee rule 12
    DealerNoPushLoseLose
    TieYes or NoPushPushPush
  12. Winning Ante and Play bets pay 1 to 1. Winning Odds bets pay according to value of the hand and whether it wins or loses. The following pay table shows what winning Odds bets pay.

    Winning Odds Bet Pay Table

    HandPays
    Royal Flush500
    Straight Flush50
    Quads10
    Full House3
    Flush1.5
    Straight1
    All OtherPush

    Galaxy Gaming, the game owner, has four different pay tables for losing blind bets, as follows. Sahara hotel las vegas.

    Losing Odds Bet Pay Table

    HandPay Table
    1234
    Straight Flush500500500500
    Quads50505025
    Full House1010106
    Flush8655
    Straight5544
    All OtherLossLossLossLoss
  13. In addition, there are two side bets, that pay based on the player's cards only, the Trips Plus and Pocket Bonus. These side bets are explained after the analysis of the base game.

Strategy

The player should make the 3X raise with any pair except deuces. Otherwise, use the following table shows when to make the large 3X raise.

The strategy for the medium and small raise are the same as in Ultimate Texas Hold 'Em, as follows:

Medium Raise: Make the 2X raise with any of the following:

  • Two pair or better.
  • Hidden pair*, except pocket deuces.
  • Four to a flush including a hidden 10 or better to that flush

* Hidden pair = Any pair with at least one card in your hole cards (thus the pair is hidden to the dealer).

Small Raise: Make the 1X raise with any of the following, otherwise fold:

  • Hidden pair or better.
  • Less than 21 dealer outs beat you.

For a more powerful small and medium raise strategy, I recommend the James Grossjean strategy card for Ultimate Texas Hold 'Em.

Analysis

The following table shows all possible outcomes of each hand, what it pays, the probability, and contribution to the return under the losing Odds bet pay table number 1 (the one that goes 500-50-10-8-5). The lower right cell shows a house edge of 2.36%.

Return TableExpand

PlayerRaiseDealer
Qualifies
WinnerPaysCombinationsProbabilityReturn
Fold-25,498,078,560,9200.197674-0.395349
Less than pair1NoDealer-260,518,663,4240.002176-0.004352
Straight flush1YesDealer49820,279,1000.0000010.000363
Four of a kind1YesDealer481,726,735,9800.0000620.002980
Full house1YesDealer810,082,720,2200.0003630.002900
Flush1YesDealer649,072,032,2160.0017640.010586
Straight1YesDealer347,952,010,7200.0017240.005172
Less than straight1YesDealer-33,006,630,550,1640.108098-0.324295
Anything1Y/NPush0455,081,939,8240.0163620.000000
Royal flush1NoPlayer5016,914,8800.0000000.000125
Straight flush1NoPlayer51279,004,3200.0000100.000512
Four of a kind1NoPlayer11-0.0000000.000000
Full house1NoPlayer4-0.0000000.000000
Flush1NoPlayer2.543,096,215,6000.0015490.003874
Straight1NoPlayer2145,034,240,5800.0052140.010429
Less than straight1NoPlayer1816,981,676,8240.0293730.029373
Royal flush1YesPlayer50246,580,7600.0000020.000841
Straight flush1YesPlayer522,023,968,5880.0000730.003784
Four of a kind1YesPlayer12964,337,3280.0000350.000416
Full house1YesPlayer546,108,374,1920.0016580.008289
Flush1YesPlayer3.5202,016,746,2360.0072630.025421
Straight1YesPlayer3421,268,280,0800.0151460.045438
Less than straight1YesPlayer21,600,861,520,2040.0575560.115113
Less than pair2NoDealer-311,023,268,7840.000396-0.001189
Straight flush2YesDealer49716,724,4600.0000010.000299
Four of a kind2YesDealer47311,575,4600.0000110.000527
Full house2YesDealer718,705,932,5800.0006730.004708
Flush2YesDealer527,615,003,6640.0009930.004964
Straight2YesDealer221,031,977,4400.0007560.001512
Less than straight2YesDealer-41,979,644,169,3840.071175-0.284699
Anything2Y/NPush0214,144,135,7200.0076990.000000
Royal flush2NoPlayer50211,938,6800.0000000.000215
Straight flush2NoPlayer52397,598,4000.0000140.000743
Four of a kind2NoPlayer12-0.0000000.000000
Full house2NoPlayer5-0.0000000.000000
Flush2NoPlayer3.545,718,738,9200.0016440.005753
Straight2NoPlayer353,068,201,3800.0019080.005724
Less than straight2NoPlayer21,207,385,216,7120.0434100.086819
Royal flush2YesPlayer503147,692,8800.0000050.002671
Straight flush2YesPlayer533,016,851,6120.0001080.005749
Four of a kind2YesPlayer1320,440,911,3120.0007350.009554
Full house2YesPlayer6320,575,227,4080.0115260.069155
Flush2YesPlayer4.5183,447,763,4040.0065960.029680
Straight2YesPlayer4158,035,798,3600.0056820.022728
Less than straight2YesDealer32,415,318,761,2800.0868390.260516
Less than pair3NoDealer-466,873,993,6000.002404-0.009617
Straight flush3YesDealer49614,499,4000.0000010.000259
Four of a kind3YesDealer46316,891,1200.0000110.000524
Full house3YesDealer613,387,474,0800.0004810.002888
Flush3YesDealer420,484,007,0800.0007360.002946
Straight3YesDealer122,371,396,7200.0008040.000804
Less than straight3YesDealer-53,136,124,565,4000.112754-0.563771
Anything3Y/NPush0223,641,379,5200.0080410.000000
Royal flush3NoPlayer50386,472,3600.0000030.001564
Straight flush3NoPlayer53180,911,8800.0000070.000345
Four of a kind3NoPlayer13-0.0000000.000000
Full house3NoPlayer6-0.0000000.000000
Flush3NoPlayer4.541,791,833,3600.0015030.006762
Straight3NoPlayer464,386,219,8400.0023150.009260
Less than straight3NoPlayer31,317,173,128,5600.0473570.142070
Royal flush3YesPlayer504556,552,4400.0000200.010085
Straight flush3YesPlayer541,444,036,6400.0000520.002804
Four of a kind3YesPlayer1421,003,399,3600.0007550.010572
Full house3YesPlayer7261,421,403,0400.0093990.065793
Flush3YesPlayer5.5199,160,655,3600.0071600.039383
Straight3YesPlayer5195,058,020,4800.0070130.035065
Less than straight3YesPlayer43,140,424,343,7600.1129090.451635
Total27,813,810,024,0001.000000-0.023584

As shown in the lower right cell, the house edge is 2.36%. This is the expected loss to the Ante wager only. For example, if the player bets $5 on both the Ante and Odds, then his expected loss would be $5 × 0.023584 = 11.79¢.

The average final wager per hand is 3.67 units. That makes the element of risk 2.36%/3.67 = 0.64%. This means for every dollar you wager in the game, on anything, other than the side bets, you can expect to lose 0.64¢.

By comparison, the element of risk in Ultimate Texas Hold 'Em is 0.53%.

The standard deviation, relative to to the Ante bet, is 4.56.

The next table shows the house edge and element of risk under all four losing Odds bet pay tables according to the pay table for a losing Odds bet.

House Edge Summary

HandLosing Odds Bet Pay Table
1234
Straight Flush500500500500
Quads50505025
Full House1010106
Flush8655
Straight5544
All OtherLossLossLossLoss
House edge2.36%3.06%3.73%4.55%
Element of Risk0.64%0.83%1.02%1.24%


This is full table cloth.

Trips Plus

The Trips Plus bet will pay according to the poker value of the player's hand regardless of the value of the dealer's hand. Following is an analysis of the most common pay table.

Trips Plus Return Table

HandPaysCombinationsProbabilityReturn
Royal flush1004,3240.0000320.003232
Straight flush4037,2600.0002790.011140
Four of a kind30224,8480.0016810.050420
Full house83,473,1840.0259610.207688
Flush74,047,6440.0302550.211785
Straight46,180,0200.0461940.184775
Three of a kind36,461,6200.0482990.144896
All other-1113,355,6600.847300-0.847300
Total133,784,5601.000000-0.033363

The next table shows four known pay tables for the Trips Plus.

Trips Plus Pay Tables

HandPay Table
1234
Royal flush100100100100
Straight flush40404040
Four of a kind30303030
Full house9887
Flush7676
Straight4545
Three of a kind3333
All other-1-1-1-1
Total-0.74%-1.74%-3.34%-4.34%

Pocket Bonus

The Pocket Bonus bet will pay according to the value of the player's two hole cards. The follow tables show what each two cards pays, the probability, and contribution to the total return for each known pay table for the Pocket Bonus.

Pocket Bonus — Pay Table 1

HandPaysCombinationsProbabilityReturn
Pair of aces3060.0045250.135747
Ace & face suited20120.0090500.180995
Ace & face unsuited10360.0271490.271493
Pair 2s - Ks5720.0542990.271493
Loser-11,2000.904977-0.904977
Total1,3261.000000-0.045249

Pocket Bonus — Pay Table 2

HandPaysCombinationsProbabilityReturn
Pair of aces2560.0045250.113122
Ace & face suited20120.0090500.180995
Ace & face unsuited10360.0271490.271493
Pair 2s - Ks5720.0542990.271493
Loser-11,2000.904977-0.904977
Total1,3261.000000-0.067873

Pocket Bonus — Pay Table 3

HandPaysCombinationsProbabilityReturn
Pair of aces3060.0045250.135747
Ace & face suited20120.0090500.180995
Ace & face unsuited10360.0271490.271493
Pair 2s - Ks4720.0542990.217195
Loser-11,2000.904977-0.904977
Total1,3261.000000-0.099548

Acknowledgments

I would like to thank Charles Mousseau and Stephen How for their assistance, which confirmed my analysis. Thanks to Charles also for his advice on the medium and small raise strategy.

Internal Links

Detailed calculations on some starting hands.

Extneral Links

Play Heads Up Hold 'Em at the Galaxy Gaming web site. One click and you're playing.


Heads Up No Limit Hold'em Book

Written by:Michael Shackleford



broken image