How To Teach An AI To Play Hex: Shards Of Fate

Hex is getting smarter. It’s learning. While it might not quite be approaching the colloquial indifference (or destructive potential) of Wargames’ WOPR, it is watching you play – it’s watching everyone play – and it’s adapting.

After their tremendously successful Kickstarter (warning: contains clown death) and while settling a protracted lawsuit with the creators of Magic: The Gathering, developer Cryptozoic have been diligently ploughing through the alpha and beta stages of Hex [official site], regularly bolting on new features and laboriously constructing the vision that CEO Corey Jones first prophesied nearly two and half years ago.

In that same time, Hex’s players have already duked it out in thousands of games, but what Cryptozoic are now excited about is the opportunity for them to stop fighting one another and instead turn their attentions to a common foe: the game itself. The game is destined to be an “MMOTCG”, and now their focus is turning to creating MMO standards like quests and raids. I visited Cryptozoic to play the new mode and to find out: how do you teach an AI to play a card game that’s changing all the time?

It’s been possible to play Hex in its alpha and beta stages for months, but Jones insists that the game is still nowhere close to its full potential in terms of either execution or presentation. “Showing the alpha to people was the hardest thing,” he says, during an interview at Cryptozoic’s Los Angeles office, “because it’s like you’re showing your baby to people, but… it’s a really ugly baby. It’s going to be beautiful when it gets older! Now, I feel like we have the best-looking full TCG out there.”

It’s certainly come along. Back in July, Ben built some decks and drew some cards, noting that Hex had already achieved one of Jones’ main aims of offering a more complex alternative to Hearthstone. With its socketed cards, player champions and reactive play, Hex has grown into a game that can be a little intimidating to the new player, but Jones says that part of the point of the new PvE campaigns will be to offer a gradual introduction. “One of the things the new PvE campaign is going to do is help the player understand the complexities of Hex,” he says. “We jammed in a bunch at the start with all the equipment and socketed cards and characters you can level up. Now, we have to be careful that we don’t overwhelm people.”

Watching a demonstration of the journey through one of these new PvE campaigns, “overwhelmed” is definitely not how I would describe my response. An overhead view of a very traditional fantasy map sees us following predetermined paths through a forest, node by node, fighting initially very straightforward opponents before coming to a fork in the road and making a choice about where to head next. Between these moments of card combat, short dialogues flesh out the story or sometimes present new choices and challenges, with the player’s character class and race sometimes a factor in these.

In short, it’s much of what you’d expect from a traditional RPG experience and your first quests in a fantasy land. It looks a little like one of the early Heroes of Might & Magic games and feels very familiar, certainly a lot less intimidating than simply stumbling into Hex’s first deckbuilding options.

It’s not lacking in the presentation department, either, and reflects how a considerable amount of Cyrptozoic’s energy is going into trying to make Hex an ever flashier, ever fancier thing to play with. Part of this is for the sake of functionality, because a clear and helpful interface, with its many sorting options, stat breakdowns and indices, makes everything easier to get a grip on. But it’s also because Jones and his team are taking pride in building a game that is far prettier than it ever need be. Elaborate orchestral soundtracks and gorgeous art are evidence of a great effort to create not just a game, but a world. Embellishments are everywhere.

Most impressive, however, is their tackling of the most important aspect of the single-player experience. Designing the fundamentals of a card game is one challenge, devising an ever-growing roster of cards to play with is yet another, but what Jones’ team now have to do is create an artificial intelligence that presents a credible and capable challenge to players. It has to know how to play, understand every card in the game and, like any of Hex’s human opponents, be able to anticipate, to adapt and even to surprise. “And,” insists Jones, “it doesn’t cheat.”

It’s such a specific claim to make that I’ve no idea how credible Jones is when he insists that engineer Chris Woods “is the most experienced person building AIs for trading card games alive.” Still, he certainly seems to have a plan. Woods explains that he has turned the “language of the game’s cards” into a programming language itself. Hex’s AI reads cards, much as we might, and understands what they do in the context of play.

“We have a compiler that takes the English written by the designers and turns it into code executed by the [game] engine to make the card work,” he says. “What the AI does is take that code, investigate that program and say ‘This is what the card does.’ It can quickly classify and *learn* new cards. You don’t have to explicitly go in and tell it how each card works.”

“It’s not as difficult as people make it out to be because you can simplify the problem in a lot of interesting ways,” he continues. “Of, say, two thousand cards, how many of those are doing something you really haven’t seen before, or that you can’t build from pieces you haven’t seen before? In those two thousand cards, you’re going to have, say, fifty cards that all deal damage or destroy troops.”

Getting the AI to understand what all those cards do, breaking them down into their basic functions, is the simpler side of things. Getting them to apply that knowledge, that’s where the engineering gets rather cool. Hex’s ever-expanding pool of players have fought a tremendous volume of duels over the years and quietly, discreetly, the game has been watching. It’s collected data on card usage, on deck composition and on who is winning and losing, all of which goes into a relational database and acts as a body of general knowledge. The AI, says Woods, has access to data that tells it not simply what sort of cards tend to win games, but what sort of cards tend to be played together, what cards follow one another and so what sort of tactics a particular player might be trying. It looks, as any of us might, at trends.

This also means the AI should be far better at doing two things that AIs are rarely good for: guessing and learning, both of which are vital for playing Hex. Hex is a game of incomplete information, a game where it’s not possible to know everything about what your opponent’s next moves could be and where bluffing or misdirection can be very important. This is something AIs often struggle with and they do best in situations when their relentlessly logical brains can see and calculate all future outcomes. But, says Woods, “there’s a reason why chess is both the best strategy game in the world and none of us play it. It is a great game, but it’s also super dry, because it’s all about perfect information and perfect moves.” There’s little room for the unexpected, whereas a well-shuffled deck of cards is nothing but a series of unexpected outcomes.

Hex’s AI can anticipate you pulling a quick one-two by looking at your first move, analysing how many previous players have made a move like that, considering what other cards those players tend to include in their decks, and then estimating what you’re likely to follow it up with and even what your grander strategy might be. At the same time, it’s constantly learning its own new card combos and strategies by analysing everyone else’s successes and failures, baiting and switching you with the newest, smartest ploys.

Of course, these are best guesses and it’s still possible to outwit the AI by trying something novel or unexpected, but this should work much the same as it would with any human player. The idea is to create something that responds in an organic, human fashion. To further mix things up a little, Woods says a series of quirks and biases have been coded into the systems.

“There’s a list of forty or fifty different traits,” he says. “Whenever the game is asked [to generate] a new AI, it rolls on that list and picks up six to eight new traits. Each of those is technically a weakness of some sort.” These traits cause the AI do things like to value particular cards higher than others, or to respond more aggressively to certain threats, perceived or actual. Again, the goal is to present an opponent who plays realistically, with preferences and particulars.

I’m intrigued to see how all this comes together and perhaps even if I’ll be able to tell the difference between one of these AIs, resplendent with metadata, from a real opponent. There is still much work to be done and, though Jones is hoping to to have things “finished” by the end of the year, nobody is expecting a smooth ride along a road that has already been bumpy and which has seen many ideas attempted and discarded. Jones says the team went through four different models for level progression alone and he has no shame in acknowledging that mistakes have been made, with more likely to come.

“I think a lot of developers mistakenly believe that making a game like Hearthstone was easy,” he says. “I think what they’re discovering, as we know after spending four years working on Hex, making a fully-fledged TCG online is insanely difficult.” While making a digital card game allows for both more complexity and less player error (nobody can make an illegal move or forget a rule, countless stats can be tracked), the temptation to add features is “a double-edged sword,” says Jones. “You have to be careful. We have all of this canvas available, we look at this embarrassment of riches in terms of design space and think, ‘Wow, all the things we can do!’ But we have to slowly parcel this stuff out.”

A TCG is never finished, either. The game’s fourth set of cards is currently being tested, the fifth is already in development and a sixth is being designed, with the team typically starting off with paper prototypes of these between eight and twelve months before they see release. Jones concedes that the team have recently had to begin banning occasional cards because they became game-breakingly powerful, but one or two bans aren’t uncommon in TCGs and he feels the team aren’t “going to have too many surprises” in that respect.

“TCGs have been around long enough that we know where the gotchas are, but the other side of things, the parts where we’re trying to innovate and inject digital design in, that’s the part where we’re going to get caught,” he says. “What I’ve had to tell our fans and our community is, trust me, you’d rather have us pushing boundaries and taking risks if, in the end, it means a card has to be banned or something has to be changed. It’s better than playing it safe. That risk-taking leads directly to all the innovation that we’re going to get.”

Jones is willing to take those risks and make those mistakes as long as his team continue trying new things, seeing what works and learning from the experience. With a growing player base, following a tremendous Kickstarter, and after finally settling with Wizards of the Coast, I would imagine Cryptozoic feel increasingly confident and don’t mind taking one or two tumbles as they reach toward new things. As long as they keep learning, just like their AI.

Hex: Shards of Fate is currently in open beta.

17 Comments

  1. WiggumEsquilax says:

    I won’t be happy until the A.I.:
    A)Learns to troll,
    B)Observes a players’ triggers in chat and flame contextually,
    C)Griefs it’s own A.I. teammate in 2 v 2 battles.

  2. Arexis says:

    I’d like this game more if I could play it. It seems to be really annoying about security settings on the PC, and the devs were pretty useless when I reached out for help suggesting fixes or setting changes that didn’t apply or exist in my version of Windows (Still on Vista, I know, I’m poor). A shame, because this is the kind of thing I really enjoy.

    • Tywin says:

      I don’t think you can really fault Hex for not supporting an operating system that Microsoft doesn’t even support anymore.

      • Arexis says:

        I do agree with the point that it’s hard to fault them for not supporting the OS. That’s why I check system requirements for every game I play. If their site didn’t list Vista, I wouldn’t have tried it in the first place.

        • Tywin says:

          Oops, sure enough, they list Vista and even XP. Never mind then, carry on.

  3. SquirrelCrusader says:

    I would take Cory Jones’ claims with a grain of salt. He gets very excited when talking about Hex, and it’s not like he’s very knowledgeable when it comes to computing in general.

    I have a very hard times reconciling all the claims above with the simple fact that the current version of the game has many bugs, and an AI that’s… not so smart.

    • Senthir says:

      One of the things with their AI is that their AI and our AI are vastly different things. Historically, we’d complain on the forums about how bad the AI is and how drastically it needs to improve. Then a dev will respond with “Oh, you’re playing on the old, stupid AI. We’re playing on a much better one, it’ll be out soon.” and then it releases, and it’s much better. The cycle seems to continue forever, we complain, they tell us they already have a better one, then we get the better one.

      I’m not saying the current AI is great, because it ain’t, but how much it has improved since the start of the game is pretty substantial. It still has some hilarious quirks which this article shed some light on, like for example the fact that it will occasionally cast a card with no target to cast on, fizzling it away which I assume means it looks at past data and says “Players who play this card when their opponent is at 5 or less health win the game 95% of the time, I’ll play it.”. But all in all, it’s a lot better, Arena AI is pretty solid for what it is, and I’m glad they’re still working to make it better because I REALLY want the PVE aspect of the game to be fun.

      • SquirrelCrusader says:

        Deciding whether to cast a card based solely on historical data is completely stupid. That doesn’t mean you can’t use historical information to make your AI “better”, just that you have to be very careful when doing so.

        I don’t think the quirk you mention has anything to do with this anyway: The AI routinely targets its own troops with spells intended for the opponent, and I doubt it “learned” that by watching users play. It looks to me like the current AI has nothing to do with the one being described here, and I have a hard time figuring out why they’re not releasing it if it’s ready.

        I understand that this interview is just a PR exercise, but CZE should stop overstating their own importance at every opportunity they get. Take the following tidbit for instance:

        he has turned the language of the game’s cards into a programming language itself. Hex’s AI reads cards, much as we might, and understands what they do in the context of play.

        It’s nothing out of the ordinary really. All the card game projects I know do this in one form or another. Even MTGO. Even hobby ones. That technique has nothing to do with AI, it is purely a productivity tool. Without it you would have to code the same thing over and over.

        If you handle this translation properly, it can help you build a robust AI but you still have to code that AI. That’s where the difficulty lies.

        • malkav11 says:

          It probably isn’t ready. That seems like the most logical explanation to me.

  4. NephilimNexus says:

    Wouldn’t you prefer a nice game of chess?

  5. kalzekdor says:

    To the engineers behind this AI, well done. If someone asked me to develop an AI for a TCG, this is exactly what I would have done: transform the text effects of a card into a specific set of known functionality (two-fold benefit of having cards that can be computer-parsed, and making it easy for new cards to be added), analyze datasets gathered from real players, and I would even have implemented something like their Trait/Personality system. I’m gonna have to keep an eye on this Chris Woods fellow; he knows his stuff.

    • Valkyr says:

      So you’re basically saying someone is brilliant because he had the same thought process as you?…

      • kalzekdor says:

        Usually, yes. Self-aggrandizing, I know, but I’ve studied AI for a long time, and I am exceptionally intelligent. (Though, of course, there’s no reason that one would take my word on that, so it’s mostly a non-sequitur. You brought it up, though.)

        • mukuste says:

          The Dunning-Kruger is strong with this one.

        • elden says:

          Good lord…please stop…forever. It’s how any sophomore computer science student would approach the problem – you are not a special flower.

  6. Rumpelstilskin says:

    I found the article unnecessarily long.

  7. Premium User Badge

    garfieldsam says:

    Aghhhh can you go back and ask him for a follow-up interview for data nerds? I’d LOVE to hear any kind of details about what machine learning and text analytics algorithms they’re using and how they had to be tailored (or even tailor-built) for the purpose of a TCG AI.