Steam blocks another weird trick for estimating sales

Valve have stifled another statistical technique people used to estimate Steam sales figures, one which temporarily revived the popular estimatron SteamSpy. This latest weird trick, based on Steam achievement numbers, has been cut off less than a week after game dev Tyler Glaiel–of Closure and The End Is Night fame–brought it to public attention. While such numbers are mostly useful to Jo Public for funsies and conspiracy theories, many developers had previously said they used SteamSpy estimates in their market research. Valve do claim they’re working on better tools for developers to get data out of Steam, but for now they’re back in the lurch.

The technique Glaiel popularised used Steam achievements to estimate sales, building on a trick he picked up from a key trading site. He explained in a blog post:

“They were looking at achievement data and extrapolating how many users would be necessary to get the percents shown. For example, if an achievement had ‘50%’ of players achieving it, that would imply at least 2 players. ‘33%’ implies at least 3. If a game has both of those, it implies at least 6 players (3/6 for the 50%, 2/6 for the 33%).”

With sufficiently precise percentages, which Valve’s Steam API offered, and some serious Numberwang (I won’t get into it but Glaiel’s post does), he could calculate how many people had any game with Steam achievements in their libraries. This technique, Glaiel said, gave numbers far more precise than from SteamSpy’s method based on sampling random public user profiles. He shared his technique with SteamSpy operator Sergey Galyonkin, who used it to get the site back up and running.

After revealing this technique, Glaiel tweeted on June 29th that they had “emailed Steam about it a week ago and got no response, so I’m assuming they just don’t care.” Ha ha. Come July 4th, he noted the technique no longer worked. Glaiel said the API had started giving out rounded achievement numbers, which are far too imprecise for this mathematical trickery to work. Once again, everyone’s in the dark.

SteamSpy died in April after Valve changed Steam’s default privacy settings, hiding the game list on user profiles that SteamSpy relied upon. Valve said that the European Union’s new General Data Protection Regulation demanded the change, though the company have made no secret of their reservations about the site.

“We had two problems with SteamSpy,” Valve’s Jan-Peter Ewert said on June 28th at White Nights, a games industry conference in Russia, when asked by dev Michael Kuzmin about blocking the tool many had found useful. Oleg Chumakov filmed the exchange.

“One is GDPR came along. The other thing is, SteamSpy had a lot of variance in how accurate it was. It was very accurate for some games, it was very inaccurate for a few others, and so a couple of developers-” at which point Kuzmin interrupted saying he knew SteamSpy only had predictions but still found those useful.

“So, to be clear, we don’t have a business selling iPhones,” Ewert resumed. “The only way that we can make money is if you make good decisions in bringing the right games to the platform and finding your audience. And so yes, we are very much working on new tools and new ways of getting data out of Steam, and we hope that data will be more accurate and more useful than what SteamSpy previously offered.”

That’s a vague plan at best. Promising, but vague – and with no timeline.

Some have taken Valve blocking Glaiel’s one weird trick as proof that the Steamers lied about the GDPR’s requirements and used it as a smokescreen to kill SteamSpy, seeing as Glaiel didn’t go anywhere near any user’s profile. But why not both? It could be true that Valve needed to hide certain information for GDPR reasons, then realised they were glad to have that ‘hole’ filled and decided to fill this hole too while working on installing a proper window themselves. It’s not like people have a right to this data, even if it can be helpful.

SteamSpy’s data could be less helpful in the hands of some players and fans. Sure, it can be fun to coo over big numbers (“oooh lookit that one with all those 0s like a digital ghost’s wail”), but a little knowledge can be a dangerous thing – and the Internet is already plenty dangerous. Often when I saw players refer to SteamSpy numbers, they were filtered through shoddy analysis and a poor understanding of how games are made then wielded as weapons to chastise “greedy” developers. I have not missed that at all during SteamSpy’s downtime.

For the curious, SteamSpy overlord Galyonkin shared a snapshot of figures calculated by Glaiel’s technique with Ars Technica. Team Fortress 2 is the most-‘owned’ game on Steam, according to that, though of course it is free-to-play now. Counter-Strike: Global Offensive is top of the paid games, then Playerunknown’s Battlegrounds isn’t far behind. And I’m sure our Alec will be relieved to hear that an estimated 3,759,377 other lost souls have clicked a portion of their life away in AdVenture Capitalist. You’re not alone there, pal.

39 Comments

  1. KDR_11k says:

    The extreme secrecy over sales numbers in the business looks quite weird, other industries are much more open about that.

    • podbaydoors says:

      But if Valve just gives this data away now, how can they charge publishers for access to it later?

      • Cinek says:

        Yes. Eg. in car manufacturing it’s a standard policy to release sales numbers, in some markets it’s done monthly.

        • TillEulenspiegel says:

          That’s weird. For the most part, only public companies release sales numbers, and only to the extent they’re legally required to.

    • skittles says:

      The silly thing is that if they hadn’t gone public with the method Valve would likely not picked up how they were doing it and we would have had a great source of sales numbers. Instead, they got everything shut down. To anyone with neurons firing in their head it should have been blatantly obvious Valve would shut it down. So I really don’t understand why they released the info publicly like they did.

  2. Vilos Cohaagen says:

    Anecdotally I feel like the tide of opinion is turning against Valve these days. They no longer feel like “saviors”, mud no longer just slides off them. It was inevitable I guess. I’m glad GOG seem to be doing better. GOG Connect has really weakened Steam’s stranglehold on me.

    • Cinek says:

      Good. The less monopoly the better. Fact that Valve turns increasingly anti-consumer hopefully will only accelerate diversification.

      • flyingscorpions says:

        Yeah, as annoying as all these publisher owned storefronts are)(Origin, Blizzard/Activision, Microsoft), at least this creates some room for competition in terms of pricing.

  3. Someoldguy says:

    +5 weekly geek points for fitting Numberwang into an article.

  4. joekiller says:

    TF2. 50 million. Wow.

  5. airmikee99 says:

    Does anyone ever really look forward to a game that was made solely because other similar games sold well in the past? Shouldn’t we be encouraging developers to make games that they truly want to make, not because the genre is the latest fad? Can’t they look at Steam’s numbers for what games are actually being played, or is that a secondary concern to making games that are trending on sales lists?

    • DoubleG says:

      “Shouldn’t we be encouraging developers to make games that they truly want to make”

      Making a game means going without income for years before you see a cent of revenue — unless you’re already flush with cash, you NEED to know approximately how much money you’ll see on the other side. You can go broke making a fantastic game for a genre that historically sells poorly.

      Independent developers, which lack first party connections and accumulated market research, are much more vulnerable to this problem than large publishers. That’s why SteamSpy was so valuable, and why losing it is such a loss.

      • airmikee99 says:

        That’s a really long winded way of saying, “No, I would rather play games made by people looking to make a quick buck rather than making a game they truly want to make.”

        Ah well, brevity ain’t for everyone.

        • Merus says:

          That’s a completely bad faith reading. Here’s what that feels like: why do you want developers to starve?

          Being able to work out how long you have before you won’t be able to pay off the credit card is vital for independent developers to be able make the games they want to make. Weirdly, landlords don’t accept rent in the form of passion, and the amount of progress you can make on a game working part-time is not usually enough to keep momentum going.

        • Massenstein says:

          That’s a weird way to read a comment which simply explains why developers, especially small independent ones, have to care about sales figures.

        • Sargonite says:

          If you live in a country where people don’t need money to not starve, you should let game devs know so they can move there and make the games they want to make without worrying about money. Or if you’re independently wealthy, you should find ones you like and dump hundreds of thousands on them to cover their costs of living.

          Or you could idly wave your rotten misconceptions in everyone’s face and wonder why they aren’t smiling at the stench, I guess.

    • Monggerel says:

      Y’mean every sequel or spiritual successor ever?
      Absolutely, by all means, half my favourite things are sequels.

      The other half is Depeche Mode cover bands.

    • Xocrates says:

      The best game idea in the world is useless if you don’t have the money to make it or if no-one plays it.

      Also, generally speaking a single developer will have multiple ideas floating around at the same time, so often the decision is less “make games that they truly want to make” or make money, and more which of the games we want to make is more likely to turn a profit and allow us to make the next one.

      And steamspy was pretty invaluable there. Especially because the difference in numbers between genres can be massive. A number one seller in one category can not even scrap the top ten in another, and that type of knowledge can be invaluable when deciding how much money can you actually invest in any given title.

    • PlinyTheWelder says:

      That’s an extremely simple view of how creative work…works.

      I played in touring bands for years. At the same time I did several noise core type side projects.

      I loved it. People in other bands loved it and a tiny cross section of other weirdos loved it.

      But no label was going to invest in putting it out and I certainly wasn’t going to stop playing the music that paid the bills even if I had quite a bit less passion for it at 31 then I did at 17.

      Anybody who has ever tried to get paid for creative work has had to balance audience appetite with their own desires.

      Not to mention you seem to think that people can’t do great work while staying within genre conventions.

      A Song Of Ice And Fire found its audience amongst fantasy novel fans while remaining a compelling work of literary fiction.

      The Goddather is a masterpiece of cinema that hews closely to the classic gangster film genre guidelines.

      Doom 2016 is a fantastic game squarely fit into conventional fps design.

      If a developer wants to make a game like “You Must Be Eighteen Or Older To Enter” they are, of course, welcome to do that as a labor of love.

      But if they have employees, and lines of credit as health care packages they need to create great games that are also great PRODUCTS.

      The minute you begin doing your love for your job you start compromising.

    • airmikee99 says:

      I have to wonder how many of you complain about Hollywood releasing nothing but sequels they know will bring in cash. Or how many complain about the lack of original works in any medium. Or how many of you complain about EA just releasing another carbon copy game for cash.

      Probably all of you, but as soon as your stupid indie games are threatened, WAAAAAHHHH!!!!! Let them focus on making cash!!!!

      Hypocrites, all of ya. :)

  6. Chillicothe says:

    It ain’t just the random “Never Pay More Than $20 For A PC Game” crowd; it’s also great to chastise people for not buying hidden classics.

    Besides, Valve Gonna Valve and do their own.

  7. Baines says:

    It is known that larger publishers oppose the idea of all their sales data being publicly freely available. Multiple developers, ranging from big to small, have also vocally opposed their sales data being publicly freely available.

    Is it really any surprise that Valve shut down a fairly accurate way to calculate sales data, once they (and publishers) were made aware of its existence?

    • Hedgeclipper says:

      There’s a bit of angst floating about this comment section but from the way the calculations are described in the article I’m not at all sure they’d give accurate sales figures as achievements rely on the games being installed and played.

      I’m pretty sure I’m not the only one with a big backlog I haven’t even got to downloading and for me at least games played is definitely biased from indie titles (small cheap grab it on sale because why not I might feel like it one day but I’ve got a lot so I’ve only played a small %) toward bigger AAA stuff (its Thursday and I don’t want to think too hard and its big and glossy and cost more so I really should play it sometime).

      • Premium User Badge

        Ingix says:

        I read the original article and your doubts are not applicable. The numbers in question describe (for a given achievement of a given game) the ratio of accounts having achieved a certain achievement vs. all owners. If you haven’t played a game (but own it), you will be counted into “all owners”, which is the number that the method tries to find out.

        Basically, if a game had 5 owners, any achievement could be reached by 0%, 20%, 40%, 60%, 80% or 100% of owners. If you know from Steam that 15,15561569688% of players have an achievement, you can rule out 5 owners. And 6 oweners, a.s.o, assuming the numbers are really accurate to that many decimals.

  8. GSGregory says:

    If it was about protecting user privacy they would of hidden the data altogether instead they made the data less accurate by rounding it and made it useless for doing one specific thing.

  9. Neurotic says:

    …What They Did Next Will Blow Your Mind

    • kinyajuu says:

      That’s just how they get you to watch one last round of commercials. I already know what happens, I’m going to bed.

  10. Dominare says:

    Beware he who would deny you access to information, for in his heart, he dreams himself your master.

    • Don Reba says:

      Meaning anyone who wants privacy deems themselves the masters of the universe? Fair enough.

      • PlinyTheWelder says:

        What privacy? There’s no “privacy” issues at play here.

        This is a cumulative number arrived automatically by looking at valves number of people who got achievements.

        Nowhere did it say YOU got that achievement.

        Unless you think people have a collective right to not be included in anonymous listings of percentages of customers who got achievements..

        This was never about privacy. It’s always been about valve trying to create a product they can SELL to developers and killing off any free option that might end up being better than whatever they offer in the future.

        The privacy thing was always corporate double speak bullshit.

    • kinyajuu says:

      Those who demand your knowledge also think they own you. ;) Two sides to every coin.

  11. jezcentral says:

    I don’t think there’s any denying that the first change, that one that killed off SteamSpy, was due to GDPR. GDPR says you need to default users to the most private privacy setting, and that is all Valve did.

    This new change is almost certainly just about hiding sales figures.

  12. KraiZor says:

    The solution to people making incorrect assumptions based on incomplete data isn’t to give them less data, it’s to give them more complete data. People will just continue to speculate.

  13. Linkblade says:

    “After revealing this technique, Glaiel tweeted on June 29th that they had “emailed Steam about it a week ago and got no response, so I’m assuming they just don’t care.” ”

    They found an ingenious trick to get the numbers they want. Why the hell did they inform Steam about that? What did he expect? Steam saying: “Well done!”???
    They just could have hidden this trick to run the SteamSpy page and Steam wouldn’t have known how they did it for a very long time.

Comment on this story

HTML: Allowed code: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>