How Steam selects the games it shows you (according to Steam)

Outlast 2 recommendations

The people working on Steam (and more generally, the team at Valve) seem to be on a transparency kick at the moment. There are multiple blog entries which try to open up particularly opaque bits of the company so users can understand what’s going on. Presumably there’s an element of using that understanding to defuse criticism when it comes to subjects like the terrible reputation of Steam’s support system and whether it’s warranted if you add in some more data. The most recent entry zeroes in on how the store itself works and aims to share the “thinking” the system has used to come up with game recommendations:

Robin Walker notes “We want to show you more of what it’s doing and why – and we have some features planned to help with this, starting with one we’re launching today: an algorithm section on game pages that states why the Store thinks this game will (or will not) be interesting to you.”

Recommended games already had a snippet which said that I was getting a game in this feed because it shared tags with other games I’d played. But now when you click through to those games it goes into a bit more detail.

I’ve run through the first few in my list to see what its current hit rate is like and the results are as follows:

Prey

Prey

In Prey, you awaken aboard Talos I, a space station orbiting the moon in the year 2032. You are the key subject of an experiment meant to alter humanity forever – but things have gone terribly wrong. The space station has been overrun by hostile aliens and you are now being hunted.

Steam picked this because it’s similar to Bioshock 2 and No Man’s Sky, the user reviews are generally Very Positive, it’s in the top sellers, 21 friends own it and 5 friends want it.

I have 0.1 hours in Bioshock 2 and it was because I was taking a couple of screenshots for an apartment feature in multiplayer mode not in the main game. Conversely I have 67 hours in No Man’s Sky. The system doesn’t seem to take into account those relative weightings or that I have no interest in other games on the BioShock side of the equation.

In terms of the user reviews and top seller stuff it makes sense that a game getting positive coverage and which is proving to be popular across a wider audience might override those concerns even if it was registering them in some way. From a commercial point of view it makes sense that Steam would show me this game because I might buy it to be part of the conversation. It could have shown me a smaller title – a walking sim of some kind instead – and I wonder how it makes the decision not to. I might be an anomaly here but I’d be more likely to buy the more obscure game. Prey chat is at saturation point all around me and I actively seek alternatives.

On the friend front it’s a lot of game journos who own the game, so colleagues as well as my regular gaming buddies. I have no idea whether there is any way to factor in things like press keys – I assume not – so that’s a potentially distorting factor both for me and for anyone who is friends with people working in the industry.

The friends who want Prey are a mixed bag in terms of how their interests align with mine. Mostly I know them through Dota and Counter-Strike so they’re people I encounter in a very specific vein of competitive multiplayer.

911 Operator

911 Operator

Game about the difficult work of people that manage emergency lines and services. Your task is to answer incoming calls and to react properly – give first aid instructions, advise, dispatch correct number of firemen / police / ambulances, or sometimes – just ignore the call. Play on ANY CITY in the world!

I’d never heard of this one but I’m getting it as the second recommendation because it’s similar to Youtubers Life and Planet Coaster, the user reviews come out as Very Positive and 7 friends own it.

Real talk: I played Youtubers Life for work for an hour and a half and it was rubs, and I spent half an hour on Planet Coaster before the lack of a tutorial meant I gave up. Three of the seven friends who own the game dont’ actually seem to have booted it up because when I click for their names it says that they have it in their library but only four have actually played. There’s no-one in that list who jumps out as being a kindred spirit.

It’s actually the game description which appeals but it’s because it puts me in mind of Cook Serve Delicious but with ambulances rather than any of the reasoning shown by the machine.

Outlast 2

Outlast 2 recommendations

Outlast 2 introduces you to Sullivan Knoth and his followers, who left our wicked world behind to give birth to Temple Gate, a town, deep in the wilderness and hidden from civilization. Knoth and his flock are preparing for the tribulations of the end of times and you’re right in the thick of it.

This one is because I accidentally left Five Nights At Freddy’s running after I thought I’d quit and had left the room and because I have a smidge of time with Narcosis – a deep sea horror game that I was playing as a spider crab befriending sim. It is “currently popular” and 4 friends own it. Two are work colleagues and two have never booted it up.

User tags include “survival horror”, “horror”, “psychological horror”, “violent”, “dark” and “parkour”. I actually clicked to see the list of tags and it invited me to submit some of my own starting with the ones I’ve used on other products. Those are “origami”, “papercraft” and “paper” which should give you an inkling of how far off the mark Outlast 2 actually was as a recommendation.

Something worth noting here is that they system requires users to engage in the feedback loop and tell the system where it’s going wrong so that either the team behind the algorithms or the algorithms themselves can adjest accordingly.

I tend to get stuck in uncanny valleys of recommendation engines because I tend to give systems the bare minimum of data needed to use them. I need Steam for work and to play Dota so I have to agree to it tracking some elements of my behaviour and I’m human so sometimes I’ll follow a link from something it’s tweaked and either buy or not buy. But because I don’t like feeding in extra data I don’t tend to use the discovery queue and if I do browse it it’s more that curiosity about how close to the mark it is rather than because I’m looking to buy or play something.

For me the promise of less friction on a shopping/entertainment platform doesn’t outweigh the discomfort I have about sharing more data than I absolutely have to. Obviously I still participate by having a Steam account and using areas of the service so it’s about the degree to which I offer this stuff up willingly rather than anything else. Your personal boundaries will vary.

The other interesting thing here is just about the phrasing. Walker’s blog post positions the Steam store as an entity:

“The Store is constantly trying to balance all the different interested groups of players and developers. It knows that it has a limited number of spaces it can use to show games to a player. It has some knowledge of the player, if the player is logged in and has a purchase / play history. It has some knowledge of the game, based on what the developer has told it and what previous purchasers of the game have said & done. It chews on all that data, and finally, decides which games it should show the player in all the various sections of the Store.”

I assume it’s shorthand for referring to the set of algorithms and types of intelligence Valve are deploying to manage the storefront’s variable behaviour but I enjoy seeing how these things gradually take on a fledgling identity through our communication with them and as we scrabble for ways to quickly explain what they’re doing.

Valve are working on another two big explainy blog posts. Next they’ll gab about “some ways the Store is being exploited”, which sounds like it might dig into changes to gifting and keys and whatnot. The third will get into Steam Direct, the upcoming scheme to replace Greenlight with more-open publishing, and talk about the fee Valve will charge for that.

41 Comments

  1. Chewbacca says:

    Still trying to figure out the link between Prey and NMS. Because it’s in space?

    • dash387 says:

      it’s probably something like that.

      steam recommended tides of numenera to me because i played life is strange. if you hover over the “similar” game it explains further: similar tags include story rich… oh well

    • Xocrates says:

      Steam recommends Styx: Shards of Darkness to me because I played… Torchlight 2.

      Keep in mind, I own plenty of stealth games, including the first Styx, but Torchlight 2 is literally the only game mentioned.

  2. jwoozy says:

    Gee Valve, maybe just don’t let an ocean of unplayable vaporware trash drown your entire storefront?

    • Aetylus says:

      The valve post talks about making a good “store” (those carefully curated shops where you, the customer, go because you know they have stuff you like).

      Unfortunately, valve’s actions repeatedly show that they are trying to create a “market” (of that ‘open’ type, where the free for all of ‘unregulated market forces’ allow a flood of utter crap to gush over you, the consumer).

      They can’t be both.

  3. matoos says:

    Do you think that it may be in any way related to the fact that game journalists consume games in a different way than overall population, and that’s why your recommendations are kinda skewed?

    • Wulfram says:

      Yeah, I don’t think games journalists are a very good test case.

      Though I wasn’t particularly keen on my recommendation. In particular, “because you’ve played games tagged “action”” seemed kinda silly since that covers so many varied games.

    • Philippa Warr says:

      I honestly don’t know because I don’t know if it’s factoring any of that in.

      For example, some games have press access branches you activate on Steam before the game releases which is how you play the review copy of the game. The data Steam are making visible give me no clue about whether they take that into consideration or if the AI is learning anything from that. I also activate a lot of non-Steam games on Steam to make use of the screenshot functionality which could theoretically be another source of data depending on what Steam could scrape from that. It might not give granular data but it might play into whether it chooses to recommend me games of the type which a lot of Steam users are playing or whether it starts plucking things from the experimental fringes.

      Another thing to bear in mind here is that having other game journos on my friends list wouldn’t necessarily skew the system’s effectiveness. It might learn that particular titles are useful for my job if it learns to lump those people together meaningfully. It’s more that here we’re seeing an obviously imperfect system and one which I’m refusing to refine in some ways and not others. It’s interesting to me where it might still get a hit and where the misses creep in given it can only go on certain bits of info.

  4. something says:

    Basing recommendations on tags is as close to random as makes no odds.

    • Wulfram says:

      It doesn’t seem a totally bad concept, but they could maybe do with weighting it somewhat for tag frequency – sharing “action-RPG” should mean more than sharing “action”.

      • something says:

        From my current recommendations:

        Open World games
        Due to your recent playtime in other Open World games
        – Factorio
        – TY the Tasmanian Tiger
        Singleplayer games
        Due to your recent playtime in other Singleplayer games
        – Iron Snout
        – One Finger Death Punch
        – NEKOPARA Vol. 2
        – Rabi-Ribi

        For reference, recently I have been mostly playing Fallout 4 and Dishonored 2. So… I guess it is true that I like single player, open world games but I’m not sure they’re using that information very well.

        Other tags it offers me include Adventure, Atmospheric and First Person. Each of those is even broader than you might imagine.

        • Wulfram says:

          Like I said, they should use weighting to focus on the less common tags.

          Playing Dishonoured 2 should lead the algorithm to games with “Stealth” (293 games), “Assassin” (40 games) and “Magic” (115 games), not Action (11,582) and not so much First person (930).

    • Frank says:

      I disagree. I have blocked “Visual Novel” and always find “Metroidvania” worth a look, for example.

      Crowd-sourcing tags is one way to go, but it’s certainly limited. If recommendations are really impacting their bottom line, they could do like Pandora’s music genome project (or similarly, what AMG or jinni does) and hire people to add richer tags following consistent rules.

  5. automatic says:

    I bet they also use an obscure cookie network, from which RPS is certainly part of. Not a conspiracy theorist here. If was stuffed with tech and money like Valve is I’d use it too. That’s how the system works.

  6. Mr Bismarck says:

    On the plus side, my impulse-purchases on Steam have dropped to practically nil because the landing page has descended into a complete and utter bacchanalian clusterfuck.

  7. mpk says:

    Today I’ve been recommended:

    Viscera Cleanup Detail, because of its similarity with Farming Simulator 13
    Far Cry (the original) because I’ve played Signal From Tolva for about an hour (but not because I own and played the shit out of Far Cry 2)
    Creativerse, because I have played open world multiplayer games, which are tags so broad as to be meaningless.

    I purposely turn off as much of the data gathering on everything I use, be it Steam, Netflix or anything else using this kind of algorithm. I get rather sick of being presented with choices based on movies or games I’ve already watched and played or, in some cases, accidentally clicked on and launched for less than five seconds. If I want more of the same kind of thing… well then that’s easy to find. Give me the things I don’t know I want to see or play, and then I’ll be more interested.

    I’d be quite happy with turning the entire thing off – give me an A-to-Z list and let me browse on my own terms.

  8. Premium User Badge

    sylmarien says:

    Yeah, it is bad, even close to very very bad. (And I checked just after reading the article to see if it was any better than before and… Nope)

    And the truth is I do like the idea of the store understanding my tastes and then presenting me “things I may like”, that’s in fact the main reason I use Google Play Music. But on Steam the results are so bad that I don’t bother anymore, and that’s a shame (especially considering that it has access to how much time I played games, or how I categorized them (I hide a lot of the game that don’t interest me and I have a (huge) “dropped” category in my library). But no, Steam still considers relevant to show me titles “similar” to BallisticNG which I installed, played 5 minutes and uninstalled because I didn’t like it…

    I so yearn for a good system that can curate this overwhelmingly huge store for me, but they aren’t even close to something “not awful” so it will have to wait…

  9. SuperJonty says:

    I’ve just been offered Dragon Age: Origins, because I own Dragon Age: Origins Ultimate Edition. Seems to be working as well as before…

    • Premium User Badge

      Don Reba says:

      This is the Amazon school of thought on recommendations: you recently bought a video card, so you must be into buying video cards; here are some other video cards you could have bought.

      • malkav11 says:

        I just bought a mattress so Facebook is alive with mattress ads. It’s ridiculous.

        • Premium User Badge

          Don Reba says:

          If you are using Facebook in English (or are willing to switch), I highly recommend the F.B. Purity browser plugin. It removes all ads and lets you tweak a lot of things, like turning off the recently introduced HUGE FONT for short posts.

  10. Premium User Badge

    Don Reba says:

    Steam has decided a long time ago that female protagonists are my “thing”, so it recommends games to me based mainly on that feature.

    • Premium User Badge

      Don Reba says:

      The only reason I play Portal and Mirror’s Edge is to indulge a fantasy of wearing a sports bra, obviously.

      • Premium User Badge

        kfix says:

        …well I suppose there MIGHT be other reasons, but none that I want to know about.

  11. Sin Vega says:

    Netflix once recommended I’m Alan Partridge to me, with the line “Because you liked Breaking Bad”. So I’m pretty skeptical of the glorious future that every internet business thinks algorithms will deliver.

    • valuum says:

      I count three capital letters in each. If you include Because. Flawless.

      • Slazia says:

        To be honest, AP should be recommended to everyone no matter what their tastes.

  12. kavika says:

    Algorithms are hard, and take forever to get right. You don’t have a magic solution, and neither does Valve.

    Valve holds a lot of power, and they’re asking us to help them wield it. If they want our help building algorithms, we should help them, so we can help ourselves. That’s really the only way you, me, and Valve can responsibly get this right.

    Let’s judge them, but let’s be fair, too. Let’s not assume malice where stupidity would explain it.

    No matter how much data they have, and how many vectors they identify, they’re not psychic, not seers, and don’t have santa-god-like powers. Let’s not scrap the whole thing just because Valve didn’t give us exactly what every single one of us don’t even know we want, and gave it to us yesterday.

    • ColonelFlanders says:

      I dunno, that deep dream algorithm is a pretty accurate visual representation of being fucked up on mushrooms, so there’s hope yet.

    • April March says:

      Man, I’d be a lot more kind towards Valve if I suspected malice. That’s because to match the results of a little malice it takes a lot of stupidity.

      Also, don’t pretend that their attempts to have me spend more money on their store are “helping me”. There are a lot of sites that recommend cool games that I might like because they want their users to have great experience; Steam is not one of those, and don’t forget it.

      • kavika says:

        What are those results? How have you been harmed by Valve? Show me where they touched you, on this doll…

        > There are a lot of sites that recommend cool games that I might like because they want their users to generate ad revenue for them, or somehow kick start their career as a journo, or maybe just get them free shit and get them a badge to E3/pax/etc

        FTFY

        Hate capitalism, not capitalists. Unless they’re doing something truly unethical. If you can tell me what Valve is intentionally doing unethically, I will probably understand your point better.

  13. April March says:

    Well, I’m not a games journalism, so let’s see what it offers me.

    Trine 2, because I played Risk of Rain and Magic: The Gathering. I’ve played MtG for ten minutes (or, how Steam says it, 0,6 hours… I think my math is right). All of these games have online coop modes. At least, I think, since I never played any of them, and any resemblance between the coop modes of Trine 2 and Magic: the Gathering are likely to be mostly academic. Of course, there’s no way for Steam to know that I tried Trine’s demo on the Xbox and didn’t like it. Still, fail. ❌

    Teeworlds, because I played Broforce. Between the similar perspectives, contant explosions and map-making tools I can see how the two games might seem similar to an alien unfamiliar to Earth culture to whom the concepts involved had been briefly explained. ❌

    ibb & obb, because it’s an online indie platformer, like Towerfall and Risk of Rain. This one is pretty spot on! ⭕️

    Welcome to Moreytown, a new game recommended by Choice of Games, which I follow. Technically a good suggestion; in practice, I already knew about it, exactly because I follow Choice of Games. I’ll be nice and give them a ⭕️ anyway.

    Full Mojo Rampage, an indie cooperative twin-stick roguelike. Suprisingly, while I own a lot of twin-stick roguelikes, including Binding of Isaac, Neon Chrome and Moon Hunters, it lists Streets of Rogue (which is only twin-stick in the way that Team Fortress 2 can be played with a controller) and Eldritch (which only has indie and roguelike in common, as do like eighty percents of the game in the universe). But, importantly, a friend of mine has the game, so the suggestions bears some weight. Or would, if I ever played coop games online, and the friend who also has the game wasn’t without a working computer. One of those things Steam would be able to know about, at least. But it superficially seems like something I might like, so I’ll give it a ⭕️

    That’s what it looks like for someone who isn’t a games journalism. Though, since I don’t like Steam and bought over 90% of my library elsewhere, perhaps I’m bringing my own biases to the algorithm. (Perhaps, seeing how seldom I use Steam, the algorithm did the computer equivalent of a shrug.) Still, these aren’t half bad, even if I would never pay attention to any of them if it wasn’t for this article.

    • Premium User Badge

      Don Reba says:

      I’ve played MtG for ten minutes (or, how Steam says it, 0,6 hours… I think my math is right).

      It’s 36 minutes — you got to know 0.6 is more than half, right? :)

  14. Henas says:

    Not a fan of anime, but love TBS and have played plenty of them on Steam. So because I own and have played Valkyria Chronicles, Steam endless recommends anime/JRPG games. One game, out of the hundreds I own on Steam.

  15. Jezebeau says:

    As far as 911 Operator is concerned, it’s possible the friends who haven’t played it supported the Kickstarter and had their fill before it hit Steam. I’ve barely touched it since then.

  16. nottorp says:

    I’ll have to join the crowd that hasn’t checked out any game from the steam front page since they introduced this “smart” recommendation system. Absolutely nothing I’ve seen on there piqued my interest.

    To add insult to the injury, the Steam account also has games that my daughter likes and I don’t – so even if the discovery queue worked right, it would be half useless to either of us.

  17. Premium User Badge

    BattleMage says:

    I love how it asks “Is this game relevant to you?” without the possibility to answer that question and improve the recommendations.

  18. Merus says:

    Given that literally the only recommendation engine I’ve seen work is Spotify’s Discover Weekly, I feel confident that Valve is not going to crack it. Spotify’s algorithm can tell the difference between ‘this is good but obscure’ and ‘this is not good’ and that is not something Valve has the dataset for.

    Also I see Robin Walker’s not working on games any more. Good old Valve, hiring a bunch of game developers and getting them to build a distribution platform for game developer money.

    • Premium User Badge

      DelrueOfDetroit says:

      Spotify spent a tonne of time and money on that service, to the point that they were losing money on it. It’s also a lot easier to create algorithms based on music because you can quantify a rhythm better than you can a shooting mechanic.

  19. Premium User Badge

    DelrueOfDetroit says:

    Do Valve sell any ad-space on their front page? I always assumed those pop-ups when you open Steam or the top slideshow must be paid for.