I’ve written a lot of about toxicity in MOBAs in many guises over the years, but a recent Re:work case study (that’s Google’s HR insights project) has some really interesting info about League of Legends developer, Riot, tracking employees’ toxic behaviour in its game. The company has been using that behaviour to help with staff disciplinary matters. Specifically, they took some of the worst chat logs and showed them to the offending staff members.
God, those must have been some seriously uncomfortable meetings.
Here’s what happened according to the case study:
“Riot’s Talent team wondered whether there was a correlation between in-game toxicity and real-world behavior. All Riot employees (called “Rioters”) play League of Legends and every player has a behavioral profile. The Talent team partnered with their game designers to see if in-game toxicity might predict workplace troubles.”
The blog post is keen to point out that this isn’t about tackling a big toxicity problem or anything, just about using their resources to keep it from becoming a problem as the company grows. Looking at some of the other research linked to in the blog post it seems like focusing on improving the behaviour of toxic employees might actually be more impactful or valuable than replacing average workers with star performers, although as with anything it depends on the particular situation (the paper I just linked to event says as much: “Although we certainly cannot answer this question for all possible settings, wecan assess this trade-off for our own setting”.
“Riot looked at the preceding 12 months of gameplay of every employee and discovered there was a correlation between in-game and in-Riot toxicity. They determined that 25% of employees who had been let go in the previous year were players with unusually high in-game toxicity. The most common bad behaviors they found were passive aggression (snarky comments) and the use of authoritative language, sometimes using their authority as a Riot employee to intimidate or threaten others.”
The company took the thirty most toxic employees (all more junior-level staff) and divided them up into people who needed a “stern warning” and people who “should leave Riot, because their in-game chat was unusually toxic”. Each employee then had a meeting scheduled to discuss the in-game behaviour and, for the worst offenders, where they would be confronted with their in-game chatlogs.
The study notes that some employees were let go and that the chatlogs factored into that, although it states that there were also existing serious problems in those cases. But generally the response was a more positive one.
“Pretty much everyone we spoke with was appalled at their own behaviour. We actually received some essays from employees vowing to change their ways and become not just more considerate gamers but better people,” says Jay Moldenhauer-Salazar, Riot’s head of Talent.
For me, this is that thing where you think you’re anonymous online but you’re really not and sometimes that can come back to bite you. It also chimes with a lot of the discussion around the disconnect between online interactions and what we are happy to do or say face-to-face. The New Yorker has an interesting and nuanced look at the subject of online commenting and toxicity which seems relevant to this discussion.
The study goes on to say:
“The team is now trying to figure out how to use this in-game information as a signal during the hiring process. Riot asks applicants for their in-game handle during the application process so they can review their gameplay and identify any toxic chats and behaviors.
“Riot is experimenting with displaying this information in their applicant tracking system using a simple stoplight code – red, yellow, green. For any applicant flagged “red” (the most toxic), there are sample chat logs so a recruiter or hiring manager can see how an applicant conducts themselves in the very product they’re hoping to work on.”
My first thought is that I’d really like to see a follow-up study to see how that contrition and vowing to improve actually plays out. I think it’s entirely possible to say that you want to change when called out on something and to mean it, but actually implementing that change is a different matter.
Another thought is about how chatlogs could affect hiring. I’m actually not sure how I feel about that. On one level you want the people who will be the public face of your company to represent it well and embody your values so if they don’t and if they’re behaving badly that would impact whether you want them as an employee. On another, I guess I imagine so many “what if” scenarios. What if someone had changed a lot but had been really toxic in the past? I assume the system might try to recognise shifts over time but would it work in practice?
I think there’s also just another part of me that hates conflating the things I do for pleasure and the things I do for work. I make a real effort to keep the gaming I do for me and the gaming I do for RPS, if not distinct, then at least separate enough that some of what I play feels like actual downtime and not work. So even though the only really terrible things in my chatlogs are bad jokes and puns I’d still feel really uncomfortable giving up my game handle.
I don’t quite know how to explain the discomfort adequately. The closest I can get is that when you are acting in the name of your avatar or profile you can be a slightly different version of you than you are when your real name is attached or when you’re in a work situation or something. I’d feel incredibly exposed giving that up to an employer even though there’s no negative information contained in that.
A final thought is that there’s a curious number mentioned in the study: “If a new player encounters toxic behavior in their very first game, they are 320% less likely to come back again.” I am so curious to know what that 320% means and how they worked it out. I got in touch with Riot to ask but so far haven’t found someone able to give a response. I mean, I assume it just expresses a relationship with a baseline figure they have for average likelihood to return or something, but when you’re skim reading it comes off as more “NOT ONLY WILL I UNINSTALL THIS GAME I WILL ALSO SALT MY HARDDRIVE SO NOTHING MAY EVER GROW HERE AGAIN.”
I’ll post an update if I get an explanation on that front!