OpenAI’s Dota 2 bots sets their sights on the big leagues

Dota 2 OpenAI

It’s easy enough to make a bot that can trounce a human player in a first-person shooter – just react faster and shoot straighter – but anything strategic presents a fresh set of exceptionally complex problems. To this day, few Starcraft or Dota bots could rival a decently skilled player or group, at least until now. AI research group OpenAI reckon that their Dota 2 bot team (dubbed OpenAI Five) is nearly good enough to give the pros a run for their money, and will be testing that theory this August at The International 2018.

While the videos below gives you the abridged version of this story, OpenAI’s full explanation on how the bot AI works and their expectations is eye-opening stuff. Back in April, the OpenAI Five team was beginning to show promise. By May 15th, it was evenly matched with OpenAI’s own staff. By June 6th, it was consistently beating both OpenAI and Valve’s players, and was nearly breaking even against amateur (but organised) and semi-pro teams. With every month, the AI learns more about the game. Suddenly, their hopes for an August victory don’t seem so far-fetched.

One of the most interesting details is that OpenAI Five’s individual agents, while operating as a team, don’t communicate with each other. While an internal variable (dubbed ‘team spirit’) dictates just how much of a lone wolf each bot is, they are all technically playing solo and just acting based on what they can observe about their teammates’ current status. Of course, being AI they can observe the status of their team far more actively than any human, but it’s still surprising that no communication or planning of active combination attacks is even required.

The OpenAI bots are admittedly limited in their knowledge, and are playing a rather strict house-ruled game of Dota. All matches are played with the same teams (mirror-matches of Necrophos, Sniper, Viper, Crystal Maiden and Lich), and warding, Roshan, invisibility, summons, illusions and scans are against the rules. Given a year or two of training on a much wider scale, the AI could likely learn the ins and outs of every mechanic, but for the time being this limited rule-set is the bots’ playground. Perhaps next year OpenAI will be ready to play through the entire tournament.

While I don’t normally follow Dota 2’s competitive scene, I know what team I’m rooting for in┬áThe International now. I, for one, welcome our new high-APM robot overlords. Especially if they’re helpful, supportive and not swearing at each other in Russian. I want to know when I can roll with some of these AI’s on my team.

Dota 2 is free to play and you can get it here on Steam.

10 Comments

  1. TillEulenspiegel says:

    StarCraft is tricky because there are so many different strategies, and a large part of the game is figuring out what your opponent is doing and responding appropriately in each phase of the game.

    DOTA, for all its dozens of heroes and items and crap, seems like a much more constrained scenario. There’s a progression of goals that everyone works towards, there are certain things you have to be good at. You know what your opponent has picked, so you can’t get surprised like with a bunker rush or whatever which would suddenly force you to respond in a completely different way.

    • beleester says:

      StarCraft does have a fairly well-described progression outside of certain specific cheeses. Opening play, take an expansion, midgame army, take more expansions, finish your army, crush your enemies. A reward function of “make a big army and kill the enemy’s army” would probably take you pretty far.

      And I’d bet that AIs are actually really good at finding cheeses – they start with no preconceptions about how the game “should” go, and a quick win means they don’t need to do as much long-range planning and get rewarded faster. If you built a StarCraft AI on a similar model to this DOTA one, I would expect that it would first invent a bunch of cheesy plays, then discover counters to those cheeses, then eventually settle on a “normal” build when it can’t win by cheesing any more.

  2. Artiforg says:

    I have to take issue with this:

    “To this day, few Starcraft or Dota bots could rival a decently skilled player or group, at least until now.”

    This is quite misleading. The bots are only a match for a team IF the human players are heavily restricted in the items/mechanics/heroes that they can use.

    It’s like tying the players hands behind their back then claiming that the bots are a match for them.

    • Ewn says:

      Give it an year. AI are like progeny kids.

      • Dominic Tarason says:

        I did say as much. In the space of just a couple months the AI has gone from losing soundly to casual players to holding its own against 99th percentile semi-pros.

        Give it a shitload more processing power and the chance to learn increasingly complex match-ups and it will improve. Self-teaching AI just needs a little room to stretch its legs, so to speak.

        • Artiforg says:

          Oh I agree that in time they’ll improve. But at this very moment in time the only way they can defeat semi-pro’s is to hobble the humans. In a proper match where the human players could select their own heroes and use the normal mechanics then the OpenAI would, most likely, get defeated.

          I don’t understand the no wards restriction, even the built in bots use wards, not as strategically as a human would but they cover the obvious spots.

          What I would like to see with OpenAI is for the separate heroes to be controlled by separate instances of the AI rather than one instance so their would have to be some communication between the instances – but not the telepathic type we see now with the built in bots where they chain stuns perfectly. Maybe have them communicate with the chat wheel – or even chat, so that there’s some delay (no matter how small) so that it doesn’t become unfairly stacked in the AI’s favour.

          • left1000 says:

            Yeah, it’s a sort of misleading claim. The bots are not demonstrating that they’ve learned strategy. Just that they’ve learned tactics. A starcraft bot could probably out micro a starcraft pro, and if you told the bot what strategy the human had to use, then maybe the bot could win. That’s basically all this bot is training to do. Out tactic better and better humans when the AI has already been told what strategy the humans will use. I imagine it’s nearly trivial to come up with strategies that give the AI no chance at all. Especially given the fact that not only is the strategy locked, but all subterfuge related mechanics are also removed from the game. It’s no surprise to me a computer can click faster and more accurately than a human. It’s also not very impressive, and if anything, this is all just PR for openai, to what end? I’m not sure. Probably trying to drum up more investment? not sure what other reason they have to try to overstate their achievements?

          • beleester says:

            The article says the bots learned long-term strategy as well as tactics. They learned to ignore the “safe” lane in order to push the fight onto the lane that’s harder for their opponent defend. Another video shows them making a sacrifice play to destroy the enemy barracks – trading a short term loss for long term gain. The article also says they prioritize leveling their support hero in the early game, something that’s not a traditional playstyle but the bots learned that an early advantage helps them snowball.

            Yes, the full game is still more complicated, but the bots are definitely capable of long-term planning. They aren’t just winning by superior micro.

        • Shinard says:

          Well, that is impressive, but is it winning against semi-pros with these restrictions? Because if so… not that impressed, yet. You’re playing one team who specifically trains, day in day out, for a no Rosh no wards no invis no summons no illusions no scans Sniper/Necro/Lich/Viper/CM mirror match, against another team that has probably never trained for those conditions. I’ve gotta say, it seems like building a chess AI that only knows the Fool’s Mate, then pitting it against a professional player who’s only allowed to play the Fool’s Mate. Dota’s about varied strategies, original plans and maximising the potential of every last mechanic in the game – so I’d say the AI hasn’t scratched the surface, yet. It’s impressive what they’ve done, but the AI really isn’t a world beater yet. In time, maybe. Now, not so much.

  3. Shadow says:

    If only they’d devote this kind of attention to Civ AI. It’s been sorely needed ever since Civ5 came out.

Comment on this story

HTML: Allowed code: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>