Smarter Than I: can an AI coach you on your Overwatch play?

I dream of a future where robots teach me how to get gud. A future where algorithms can pinpoint my every mistake, and tell me exactly how to avoid them. Software that learns as I do, offering insights beyond the reach of mortal minds.

That doesn't exist yet. But its predecessor just might.

Meet Omnicoach, an AI-based coaching tool for Overwatch that feeds on videos. You record yourself playing a match, upload the vid to Omnicoach's website - for a fee - then watch as an algorithm unpicks where you went wrong. It lays out a timeline of the game, with tips tied to orange highlights that let you jump straight to each nugget of advice. "Pay attention where you engage", they might say, or "make sure your teammates are within range when attempting to Earthshatter".

An AI didn't actually write those tips. For now, that's the domain of humans. Humans such as Daniel Kotai and Levente Endre Nádai, who I spoke to over Skype. Daniel is an esport community specialist with a semi-professional background in Overwatch, he tells me, while Levente introduces himself as Omnicoach's chief technology officer and lead coach.

"Omnicoach was designed for player who want to improve up to Diamond or lower Master ranks", says Levente. "The idea was to scale up the coaching I've been doing in the last two and a half years. We target any player who wants to improve and get help with their improvement, because it's really not that easy." As the testimonials on Omnicoach's website point out, paying for one on one coaching is beyond many people's reach. Omnicoach could really be on to something: an affordable, genuinely helpful coaching tool powered by an AI that knows where it's most needed.

But first lets burst some bubbles, lest you float away. I've had Omnicoach pass judgement on two matches, and both were returned to me with tips that mainly concerned specific hero match-ups, tips that I would have found just as useful outside the context of that particular game. When I brought that up with Daniel and Levente, they were understandably defensive. And a tad insulting.

"Currently we have about 2500 individual tips, around 35% of which are general tips", Daniel informed me. "When you work with really low-level players, they still have a hard time understanding what they should be doing against a particular hero in the enemy team, and we figured that these kind of match-up tips are going to help give them a foundational understanding about what their role is in the game."

Omnicoach does ask what your rank is when you upload each video, in theory letting it tailor tips to an individual's skill level. I rank at mid-platinum, though, and still felt inundated with general advice. That applied both when I was playing as Hammond, my main boy, and Reinhardt (who I'm admittedly fairly useless at).

That might not have been the only reason I saw so many matchup tips, mind. "It also depends on what kind of game you played," Daniel added, "and it can be different for certain heroes. Some heroes have more abilities, or different statuses that make them harder to mine."

Sure enough, Reinhardt came up as someone with limited scope for specific tips. "There are tips", Levente insisted, "but not as many because you don't have a lot of interactions when it comes to mechanics. Rather you have interactions with your team, and in your game-sense, and it's harder to coach that using our system. It is doable on some level, but it has its limitations, because understanding the 3D whereabouts of your hero and the team is not something we can do. It's a really hard thing to achieve."

That's a fairly dramatic limitation, and it's to Omnicoach's credit that they were happy to discuss it. They made it clear that their software wouldn't be much use to anyone above diamond rank. For my part, I should stress that my videos did contain genuinely helpful advice - specific mistakes that the AI could point to, and offer simple tips that either emphasised the importance of avoiding a certain situation or explained how to get into a better one.

Omnicoach works based on "more traditional image recognition and computer vision techniques", rather than what Levente calls "trendy deep learning and machine learning type things". "When you work with deep learning," he continued, "you need hundreds of thousands of images to be able to understand what's going on on the screen."

Relying on image recognition allows Omnicoach to provide tips based on a video alone - and also opens up the possibility of knocking together coaching tools for other games. "Because how we structure our data can be unified, and because the games are basically just events on a timescale - if you want to simplify it - we are able to process these events and tell what's going on." A prototype for Apex Legends is already in the works, Levente told me, which he expects to have ready "in a really reasonable amount of time."

Could Omnicoach be about to corner the coaching market? My research for this article didn't turn up anything that worked in quite the same way, though I did find a tool that claimed to do better. Visor isn't available anymore, but for a time it provided users with an overlay that offered Overwatch advice in real-time. When I asked my two Omnicoachers if they could ever see their software doing the same, they reminded me that there's a reason Visor isn't available anymore.

"We think that's cheating," Daniel said. "We're all about coaching players, and helping them understand what they did wrong, helping them improve, but doing this real-time while the game's going on is not really fair, in our opinion." He pointed to the way Visor could tell you when an enemy's ult was charged, which does indeed seem dodgy. Even if it weren't, Levente was adamant that post-match coaching is the best way to learn anyway.

"I play squash and I go to a coach, and if I focus on training I can actually train, make my technique better. But when I would do that in a game, having my coach shout at me from the back, I wouldn't be able to perform and I wouldn't be able to actually learn anything. So just from that perspective I don't think it's beneficial."

That's a good reason to declare video analysis the future of AI-coaching - but how long will Omnicoach's human-centric approach reign supreme? Daniel and Levente use AI to serve up human-written tips where they're appropriate, but beyond that lies a world where AI itself is capable of giving advice that a human couldn't. Other projects have already started dipping their toes into that world, notably SAP's HANA analytics platform, which supposedly helped Dota 2 pros Team Liquid train for the International.

They didn't win.

HANA and its ilk rely on the sort of data analysis that Omnicoach doesn't particularly concern itself with, even though that's precisely where AI can shine. Not that Levente was blind to that: "This is a really interesting topic from a technological view. If you see some team composition happening again and again, and you're able to use a lot of data to tell that a certain composition on a certain map is going to have a win-rate of 60%, we could recommend that to the players -- this is actually something that we can do today."

"I see a chance of doing this, of course we're going to need a lot more data, but there are already tools in machine-learning that can help you understand big data. An ultimate coach, though? An AI that can just sit down at a new game and coach it? I don't think that's something we're going to see in the near future."

It seems I'll have to put my more far-fetched robo-dreams to bed -- at least for now.