An AI that learns to play poker, on modest hardware

What it is

DeepPlay is a long-term project: an AI that learns to play poker on its own, playing millions of hands against itself, rather than applying a chart somebody else wrote.

The direction is clear, and it is what keeps this from being a daydream. Pluribus, the bot that beat professional players at six-handed tables in 2019, was not trained in a datacenter: it was trained on an ordinary server, in a matter of days, for a two-figure compute bill in dollars. If that fit there, this fits in what I have at home. That is the bet the project is making, and the reason every decision is taken while looking at what an hour of machine time costs.

And I will be plain about where it stands: not far along. There is a trainer running, a platform around it to import hands and walk them street by street, and a test bench to decide whether one version is better than the last. There is no milestone worth boasting about yet, so I am not boasting about one; when there is, it will be written here with its measurement next to it.

Technical stack

The agent is a Deep CFR trainer in PyTorch on top of the OpenSpiel game engine, playing against itself with the work spread across several processes, Redis Streams queues between them and a broker acting as the model registry. Around it sits what makes it usable by a person: a FastAPI backend with a hand-history parser of my own, an equity and ICM engine and board-texture analysis, and in front of it a Next.js interface that replays a hand street by street.

All of it runs in containers on my own server, which is also what puts in the compute hours: no rented GPU and no cloud bill. That is not a shortcoming of the setup, it is the constraint of the project — proving that it is enough is precisely the point.