How To Teach An AI To Play Hex: Shards Of Fate

Hex is getting smarter. It’s learning. While it might not quite be approaching the colloquial indifference (or destructive potential) of Wargames’ WOPR, it is watching you play - it’s watching everyone play - and it’s adapting.

After their tremendously successful Kickstarter (warning: contains clown death) and while settling a protracted lawsuit with the creators of Magic: The Gathering, developer Cryptozoic have been diligently ploughing through the alpha and beta stages of Hex [official site], regularly bolting on new features and laboriously constructing the vision that CEO Corey Jones first prophesied nearly two and half years ago.

In that same time, Hex’s players have already duked it out in thousands of games, but what Cryptozoic are now excited about is the opportunity for them to stop fighting one another and instead turn their attentions to a common foe: the game itself. The game is destined to be an "MMOTCG", and now their focus is turning to creating MMO standards like quests and raids. I visited Cryptozoic to play the new mode and to find out: how do you teach an AI to play a card game that's changing all the time?

It’s been possible to play Hex in its alpha and beta stages for months, but Jones insists that the game is still nowhere close to its full potential in terms of either execution or presentation. “Showing the alpha to people was the hardest thing,” he says, during an interview at Cryptozoic’s Los Angeles office, “because it’s like you’re showing your baby to people, but... it’s a really ugly baby. It’s going to be beautiful when it gets older! Now, I feel like we have the best-looking full TCG out there.”

It’s certainly come along. Back in July, Ben built some decks and drew some cards, noting that Hex had already achieved one of Jones’ main aims of offering a more complex alternative to Hearthstone. With its socketed cards, player champions and reactive play, Hex has grown into a game that can be a little intimidating to the new player, but Jones says that part of the point of the new PvE campaigns will be to offer a gradual introduction. “One of the things the new PvE campaign is going to do is help the player understand the complexities of Hex,” he says. “We jammed in a bunch at the start with all the equipment and socketed cards and characters you can level up. Now, we have to be careful that we don’t overwhelm people.”

Watching a demonstration of the journey through one of these new PvE campaigns, “overwhelmed” is definitely not how I would describe my response. An overhead view of a very traditional fantasy map sees us following predetermined paths through a forest, node by node, fighting initially very straightforward opponents before coming to a fork in the road and making a choice about where to head next. Between these moments of card combat, short dialogues flesh out the story or sometimes present new choices and challenges, with the player’s character class and race sometimes a factor in these.

In short, it’s much of what you’d expect from a traditional RPG experience and your first quests in a fantasy land. It looks a little like one of the early Heroes of Might & Magic games and feels very familiar, certainly a lot less intimidating than simply stumbling into Hex’s first deckbuilding options.

It’s not lacking in the presentation department, either, and reflects how a considerable amount of Cyrptozoic’s energy is going into trying to make Hex an ever flashier, ever fancier thing to play with. Part of this is for the sake of functionality, because a clear and helpful interface, with its many sorting options, stat breakdowns and indices, makes everything easier to get a grip on. But it’s also because Jones and his team are taking pride in building a game that is far prettier than it ever need be. Elaborate orchestral soundtracks and gorgeous art are evidence of a great effort to create not just a game, but a world. Embellishments are everywhere.

Most impressive, however, is their tackling of the most important aspect of the single-player experience. Designing the fundamentals of a card game is one challenge, devising an ever-growing roster of cards to play with is yet another, but what Jones’ team now have to do is create an artificial intelligence that presents a credible and capable challenge to players. It has to know how to play, understand every card in the game and, like any of Hex’s human opponents, be able to anticipate, to adapt and even to surprise. “And,” insists Jones, “it doesn’t cheat.”

It’s such a specific claim to make that I’ve no idea how credible Jones is when he insists that engineer Chris Woods “is the most experienced person building AIs for trading card games alive.” Still, he certainly seems to have a plan. Woods explains that he has turned the “language of the game’s cards” into a programming language itself. Hex’s AI reads cards, much as we might, and understands what they do in the context of play.

“We have a compiler that takes the English written by the designers and turns it into code executed by the [game] engine to make the card work,” he says. “What the AI does is take that code, investigate that program and say ‘This is what the card does.’ It can quickly classify and *learn* new cards. You don’t have to explicitly go in and tell it how each card works.”

“It’s not as difficult as people make it out to be because you can simplify the problem in a lot of interesting ways,” he continues. “Of, say, two thousand cards, how many of those are doing something you really haven’t seen before, or that you can’t build from pieces you haven’t seen before? In those two thousand cards, you’re going to have, say, fifty cards that all deal damage or destroy troops.”

Getting the AI to understand what all those cards do, breaking them down into their basic functions, is the simpler side of things. Getting them to apply that knowledge, that’s where the engineering gets rather cool. Hex’s ever-expanding pool of players have fought a tremendous volume of duels over the years and quietly, discreetly, the game has been watching. It’s collected data on card usage, on deck composition and on who is winning and losing, all of which goes into a relational database and acts as a body of general knowledge. The AI, says Woods, has access to data that tells it not simply what sort of cards tend to win games, but what sort of cards tend to be played together, what cards follow one another and so what sort of tactics a particular player might be trying. It looks, as any of us might, at trends.

This also means the AI should be far better at doing two things that AIs are rarely good for: guessing and learning, both of which are vital for playing Hex. Hex is a game of incomplete information, a game where it’s not possible to know everything about what your opponent’s next moves could be and where bluffing or misdirection can be very important. This is something AIs often struggle with and they do best in situations when their relentlessly logical brains can see and calculate all future outcomes. But, says Woods, “there’s a reason why chess is both the best strategy game in the world and none of us play it. It is a great game, but it’s also super dry, because it’s all about perfect information and perfect moves.” There’s little room for the unexpected, whereas a well-shuffled deck of cards is nothing but a series of unexpected outcomes.

Hex’s AI can anticipate you pulling a quick one-two by looking at your first move, analysing how many previous players have made a move like that, considering what other cards those players tend to include in their decks, and then estimating what you’re likely to follow it up with and even what your grander strategy might be. At the same time, it’s constantly learning its own new card combos and strategies by analysing everyone else’s successes and failures, baiting and switching you with the newest, smartest ploys.

Of course, these are best guesses and it’s still possible to outwit the AI by trying something novel or unexpected, but this should work much the same as it would with any human player. The idea is to create something that responds in an organic, human fashion. To further mix things up a little, Woods says a series of quirks and biases have been coded into the systems.

“There’s a list of forty or fifty different traits,” he says. “Whenever the game is asked [to generate] a new AI, it rolls on that list and picks up six to eight new traits. Each of those is technically a weakness of some sort.” These traits cause the AI do things like to value particular cards higher than others, or to respond more aggressively to certain threats, perceived or actual. Again, the goal is to present an opponent who plays realistically, with preferences and particulars.

I’m intrigued to see how all this comes together and perhaps even if I’ll be able to tell the difference between one of these AIs, resplendent with metadata, from a real opponent. There is still much work to be done and, though Jones is hoping to to have things “finished” by the end of the year, nobody is expecting a smooth ride along a road that has already been bumpy and which has seen many ideas attempted and discarded. Jones says the team went through four different models for level progression alone and he has no shame in acknowledging that mistakes have been made, with more likely to come.

“I think a lot of developers mistakenly believe that making a game like Hearthstone was easy,” he says. “I think what they’re discovering, as we know after spending four years working on Hex, making a fully-fledged TCG online is insanely difficult.” While making a digital card game allows for both more complexity and less player error (nobody can make an illegal move or forget a rule, countless stats can be tracked), the temptation to add features is “a double-edged sword," says Jones. "You have to be careful. We have all of this canvas available, we look at this embarrassment of riches in terms of design space and think, ‘Wow, all the things we can do!’ But we have to slowly parcel this stuff out.”

A TCG is never finished, either. The game’s fourth set of cards is currently being tested, the fifth is already in development and a sixth is being designed, with the team typically starting off with paper prototypes of these between eight and twelve months before they see release. Jones concedes that the team have recently had to begin banning occasional cards because they became game-breakingly powerful, but one or two bans aren’t uncommon in TCGs and he feels the team aren’t “going to have too many surprises” in that respect.

“TCGs have been around long enough that we know where the gotchas are, but the other side of things, the parts where we’re trying to innovate and inject digital design in, that’s the part where we’re going to get caught,” he says. “What I’ve had to tell our fans and our community is, trust me, you’d rather have us pushing boundaries and taking risks if, in the end, it means a card has to be banned or something has to be changed. It’s better than playing it safe. That risk-taking leads directly to all the innovation that we’re going to get.”

Jones is willing to take those risks and make those mistakes as long as his team continue trying new things, seeing what works and learning from the experience. With a growing player base, following a tremendous Kickstarter, and after finally settling with Wizards of the Coast, I would imagine Cryptozoic feel increasingly confident and don’t mind taking one or two tumbles as they reach toward new things. As long as they keep learning, just like their AI.

Hex: Shards of Fate is currently in open beta.