卷积神经网络计数流程图_用神经网络计数卡

最新推荐文章于 2023-08-20 22:27:22 发布

weixin_26717189

最新推荐文章于 2023-08-20 22:27:22 发布

阅读量468

点赞数

文章标签：神经网络深度学习卷积神经网络 tensorflow https

原文链接：https://towardsdatascience.com/counting-cards-with-a-neural-net-ce426c2b6e57

版权

卷积神经网络计数流程图

You can find my code on my Github here.

您可以在我的Github上找到我的代码。

Here are the links to my previous posts on blackjack. I used a modified version of my old blackjack simulator (discussed in detail in the linked posts). If you want to know more about how I coded it up or just need a refresher on basic blackjack strategy, you can read these first:

这是我以前在二十一点上的帖子的链接。 我使用了旧的二十一点模拟器的修改版(在链接文章中详细讨论了)。 如果您想了解更多有关我如何编码的信息，或者只需要对基本的二十一点策略进行复习，则可以先阅读以下内容：

One thing that perplexed me last year when I trained a neural net to play blackjack was why my neural net didn’t do better when I provided card counting information to it. Doesn’t card counting confer a significant advantage to the player?

困惑着我，去年当我训练神经网络玩二十一点ØNE件事就是为什么当我提供的算牌信息，它在我的神经网络没有做的更好。算牌不会给玩家带来重大优势吗？

For those unfamiliar with the concept, counting cards is a way of tracking the number of high cards (10, Jack, Queen, King, Ace) left in the deck. The idea is that when there are a lot of face cards left in the deck, it’s advantageous to players (because we are more likely to be dealt high cards and the dealer is also more likely to bust). So if our card counting tips us off that the remaining cards are more likely to be high, we should take advantage by making larger than normal bets.

对于不熟悉此概念的人，计数卡是一种跟踪剩余在甲板上的高级卡(10张，杰克，皇后，国王，王牌)数量的方法。这个想法是，当甲板上有很多面子牌时，这对玩家是有利的(因为我们更有可能获得高额牌，而发牌者也更有可能破产)。因此，如果我们的纸牌计数使我们发现剩余纸牌的可能性更高，那么我们应该通过比正常情况下更大的赌注加以利用。

When I tried to integrate card counting into my blackjack code last year, it failed to add anything because I did it incorrectly. The model that I built previously took in information like the cards in the player’s hand and the card that the dealer is showing and decided whether to keep hitting or stay.

去年，当我尝试将卡计数集成到我的二十一点代码中时，由于我做错了，它未能添加任何内容。我之前建立的模型吸收了诸如玩家手中的卡和发牌者所显示的卡之类的信息，并决定是继续击打还是留下。

I attempted to augment my model by providing it with a detailed count of all the cards that had been seen thus far as part of its training data (the count is refreshed every time that the deck is reshuffled). I figured that the count would allow the neural net to predict when the dealer was likely to bust — and this additional information would improve its hit/stay decision making.

我试图通过向模型提供到目前为止已被视为其训练数据一部分的所有卡片的详细计数来扩充我的模型(每次重新排列牌组时都会刷新该计数)。我认为，该计数将使神经网络能够预测经销商何时可能破产—并且这些附加信息将改善其命中/停留决策。

Surprisingly, this was not what happened. The performance actually got worse! The win probability dropped from 42% without card counting data to 38% with it. The tie probability also drops from 9% to 5% hinting at a more aggressive playing style.

令人惊讶的是，这不是发生的事情。性能实际上变差了！ 获胜几率从无纸牌计数数据的42％下降到有数据的38％。 平局概率也从9％下降到5％，暗示了更具侵略性的比赛风格。

有时机器学错了东西 (Sometimes Machines Learn The Wrong Thing)

As you can see below, when given card count data, the neural net is able to do ever so slightly better in terms of probability of a win or tie when the dealer is showing a low card but significantly underperforms its simpler (no card count data) predecessor when the dealer is showing a 10 or an ace (11 denotes ace). The probabilities in the following plots were generated by simulating approximately 250,000 blackjack games with trained neural nets deciding whether to hit or stay.

如下所示，如果给定卡数数据，则神经网络在发牌者显示低卡数但显着落后于简单数据(无卡数数据)的情况下，在获胜或平局的可能性方面要稍好一些。 )的前身，即庄家展示的是10或ace(11表示ace)。以下情节中的概率是通过使用训练有素的神经网络模拟大约25万个二十一点游戏来决定击中还是停留而生成的。

Image for post — Win or tie probability with and without card counting data fed to neural net (Image by author)

If we isolate out just the games where the dealer is showing a 10, face card, or an ace we can see the cause of the underperformance. Our supposedly smarter neural net (the one with the card count data) is doing some pretty dumb things. Looking at the blue bars in the plot below, we can see that it chooses to hit frequently even when it already holds a high hand total of 17 or more (it even hits occasionally when it’s already at 20). Contrast that to our simpler model — it correctly knows to stay for hands totaling 17 or more.

如果我们仅将庄家出示10张，面子卡或ace的游戏隔离开，我们就可以看出性能不佳的原因。 我们所谓的更智能的神经网络(带有卡计数数据的神经网络)正在做一些非常愚蠢的事情。 查看下面图表中的蓝色条，我们可以看到它选择了频繁击球，即使它已经拥有高手牌总数达到或超过17(甚至已经达到20时也偶尔击中)。与我们更简单的模型相比，它正确地知道要停留总计17张或更多的牌。

It’s not all bad. Earlier we saw that adding card count data improves the winning probability when the dealer is showing a low card (6 or less). Intuitively, the reason for this is that when the dealer is showing a low card, the dealer will definitely have to hit until his hand totals at least 17 (or he or she busts). So having knowledge of when the dealer is more likely to bust should prove somewhat helpful in these situations. And as we can see, the behavior of our card counting neural net is different. It is much more likely to stay for hands totaling 13, 14, and 15 — and based on the higher win probabilities, it seems to be on average making the right decision in these cases.

并不全是坏事。较早前我们看到，当发牌者显示一张低牌(6张或更少)时，增加卡数数据可以提高中奖几率。直观地讲，这样做的原因是，当发牌者出示一张低牌时，发牌者肯定要打直到他的手牌总数至少为17(否则他或她破产)。因此，了解经销商何时更可能破产的事实在这些情况下会有所帮助。正如我们所看到的，我们的卡片计数神经网络的行为是不同的。在总计13、14和15的牌局中，它更有可能留下来，并且基于更高的获胜概率，在这些情况下，似乎平均可以做出正确的决定。

Still, the overall decline in win probability is disappointing. Choosing to hit when our hand is already at 19 or 20 is a big facepalm. Based on our examination of how the neural net’s behavior changed once we added in card count data, it looks like while the additional data embeds some signal, it also caused our model to become overfit and make some foolish decisions. So let’s attempt to fix this.

尽管如此，获胜几率的整体下降还是令人失望的。当我们的手已经在19或20时选择击打是个大难题。根据我们对神经网络行为的观察，一旦我们添加了卡计数数据，神经网络的行为就会发生变化，看起来当附加数据中嵌入了一些信号时，这也导致我们的模型变得过拟合并做出了一些愚蠢的决策。因此，让我们尝试解决此问题。

试图修正我们的模型 (Trying To Fix Our Model)

When I went back and reviewed how counting cards actually worked, I realized I was thinking about it wrong. The additional features offer limited assistance in terms of helping our model know whether to hit or stay. In fact, as we saw, the added features were more likely to confuse the model than help it. But where the card count data might be able to help us is in deciding how much to bet. If we know that there are a lot of high cards left in the deck, we should bet more because the probability of blackjack (two card hand consisting of an ace and a 10 or a face card) is higher. Conversely, if we know that the remaining cards consist primarily of low cards, we should just make the minimum bet.

当我回过头来回顾计数卡的实际工作原理时，我意识到自己在想错。附加功能在帮助我们的模型知道撞击还是停留方面提供的帮助有限。实际上，正如我们所看到的那样，添加的功能比帮助模型更容易混淆模型。 但是，卡数数据可能在哪些方面可以帮助我们确定赌多少。 如果我们知道甲板上还剩下很多高牌，我们应该下更多的赌注，因为二十一点(两张牌由一张ace和一张10或一张面牌组成)的可能性更高。相反，如果我们知道其余的牌主要由低牌组成，那么我们应该进行最低下注。

In order to do this, instead of shoving everything into a single model, we can split up the responsibilities. We could have our old model that already worked pretty well handle the hit or stay decision and build a new model which uses the card count data to decide how much to bet. It would look something like what’s pictured below. A note on the card count features — I keep a count of how many of each card type I’ve seen thus far and when the dealer reshuffles the deck (or stack) I reset all the counts to 0.

为了做到这一点，我们不必将所有事情都推到一个单一的模型中，而是可以分担责任。我们可以让已经运行良好的旧模型处理命中或停留决定，并建立一个新模型，该模型使用卡计数数据来决定下注多少。它看起来像下面的图片。关于纸牌数量功能的注释-我记下了到目前为止我所见过的每种纸牌类型的数量，并且当发牌人重新洗牌(或堆叠)时，我将所有计数重置为0。

Of course all this depends on whether the card count data truly does help us predict blackjacks. Let’s check. We can use a ROC curve to check the efficacy of neural net 2 (if you need a refresher on what ROC curves are, check out the following link).

当然，这取决于卡计数数据是否确实有助于我们预测二十一点。让我们检查。我们可以使用ROC曲线来检查神经网络2的功效(如果需要对ROC曲线进行复习，请查看以下链接)。

Judging by the higher area under its ROC curve (blue line), neural net 2 does seem to add value (relative to deciding randomly):

从其ROC曲线下的较高区域(蓝线)来看，神经网络2似乎确实增加了价值(相对于随机决定)：

So let’s use it to size our bets and see if our performance improves. I simulated 100 blackjack decks (like a long night at the casino) 100 times and compared the average results and distributions between the following two betting strategies:

因此，让我们用它来衡量我们的赌注，看看我们的表现是否有所改善。我模拟了100次二十一点套牌(就像在赌场里漫长的一夜)，并比较了以下两种投注策略的平均结果和分布：

Making the minimum bet of $10 every time.
每次最少下注$ 10。
When the count is advantageous, bet more based on how confident we are (formula below). Otherwise bet $10.
如果计数是有利的，则根据我们的信心(下式)进行更多下注。否则下注$ 10。

Here’s how I decided to size the bets (full disclosure — I didn’t think super hard about this decision rule):

这是我决定下注的方式(全面披露-对于此决定规则，我并不觉得辛苦)：

Using the training data, calculate the mean and standard deviation of the probabilities (of a blackjack) generated by neural net 2.Z-score neural net 2’s output prediction:
Z_prob = (prediction - mean)/std_devif Z_prob > 0:
    bet = 10*(1 + Z_prob)
else:
    bet = 10

Basically if the predicted probability of getting a blackjack is higher than average, I bet more — and the excess amount that I bet is determined by how much higher than average the predicted probability is.

基本上，如果获得二十一点的预测概率高于平均水平，我会下更多的赌注，而我所下的超额赌注则取决于预测概率比平均水平高多少。

Cool, let’s see if we can actually improve our gambling results. The following plot compares gambling performance with and without neural net 2. While it looks like we still aren’t able to consistently make money, the dynamic bet sizing does improve our performance. The mean ending bankroll with dynamic bet sizing is 12% higher than without it.

很酷，让我们看看我们是否可以真正提高赌博效果。下图比较了使用和不使用神经网络2时的赌博性能。虽然看起来我们仍然无法持续赚钱，但动态投注规模的确提高了我们的表现。动态下注大小的平均结束资金比没有下注的情况高12％。

Finally, let’s compare the distributions of our ending bankrolls to make sure our dynamic (and larger) bets is not causing excessive volatility in our betting results. The shape of the distributions seem reasonably similar with the dynamic bet sizing one (blue) slightly shifted to the right, which is good.

最后，让我们比较期末存款的分布情况，以确保动态(和较大)的下注不会导致我们的下注结果过度波动。分布的形状似乎与动态下注一个(蓝色)的大小稍微向右移动相似，这很好。

结论 (Conclusion)

I was hoping for a bigger boost, but oh well. It looks like more tinkering is necessary, but at least it looks like we are onto something. It also goes to show that as cool and versatile as deep learning models can be, it still pays to be thoughtful about and to sanity check (as much as we can) the way our model works. Finally, winning when the odds are stacked against you is hard! Cheers!

我原本希望得到更大的推动，但是哦。似乎有必要进行更多的修补，但至少看起来像我们在做些事情。它还表明，尽管深度学习模型可以酷炫且用途广泛，但仍然值得深思和对模型的工作方式进行健全性检查(尽我们所能)。最后，在赔率对您不利的情况下赢球很难！干杯!