Valve Slowly But Surely Making Steam User Reviews Useful

By Nathan Grayson on May 15th, 2014 at 11:00 am.

Art imitates life

With Steam becoming more of an “open” platform by the day, Valve needs to oil the joints of its creaking machine in every conceivable way, lest it scream and screech to a halt under the weight of progress. Its latest baby step? Improving Steam user reviews, which can be surprisingly non-horrible occasionally (the “most helpful” tab definitely, er, helps), but they’re still a very imperfect science.

Steam now indicates if user reviews come from an Early Access version of a game. It’s a small thing, but it gives the system significantly more clarity in light of the fact that the “complete” version of a game often differs tremendously from a wibbly, wobbly alpha.

It still doesn’t account for the fact that the Steam hivemind is quite hostile to a game, like, say Gone Home – where even the most “helpful” reviews are incredibly skewed – but that’s kind of what happens when only a certain segment of your community is especially vocal. It’ll be interesting to see how (or if) Valve accounts for factors like this while increasingly depending on the Will Of The People.

One step at a time, though. That’s always been Valve’s way. I just hope they take their biggest steps in the right places. I mean, the Steam store is pretty much steaming garbage at this point (at least, from an organizational standpoint), and that only stands to get worse as Valve opens it up and lets developers have their own storefronts, put any game on the store, etc. The front page needs an overhaul, as do many other aspects of Steam. Valve is quietly tinkering away – tightening single screws as it sees fit – but here’s hoping a full overhaul is coming. It’s long overdue.

, , .

111 Comments »

  1. Premium User Badge

    Cinek says:

    “when only a certain segment of your community is especially vocal”
    - By “vocal” you mean “actually voting instead of staying ignorant”?

    • Premium User Badge

      jrodman says:

      Voting not helpful.

      • secuda says:

        Especially when we are looking at Greenlight games.

        • Premium User Badge

          Cinek says:

          so… you want Steam to create “special care zone” for Greenlit games?

          I’m fine with the current system. Majority of greenlit games appeal only to a narrow scope of audience – and the reviews show it perfectly. For majority these are crappy/uninteresting/ungames. Sure, that’s the McDonalds audience of PC gaming, but it doesn’t really invalidate their point of view.

          • subedii says:

            That’s the thing. Most of the complaints I see aren’t so much with the fact that the review system exists, it’s that most reviews are crap and written by people who are no good at reviewing anything, frequently confusing their own subjective complaints (or praise) as being objective issues with the title they’re reviewing.

            The thing is it’s not the storefronts job to deem everyone else’s perspective except for the ones you agree with as appropriate. It’s up to you to READ the reviews and see which ones are good and useful, and which address about the points you want to be informed about. A useful review is one where you can assess the likelihood of whether or not you’d like the title, regardless of whether the reviewer did. I’ve seen reviews of titles that I’ve liked which were non-the-less negative, but I could understand the reviewer’s perspective and what aspects they appreciated in it and why. And vice-versa.

            Yes most of the reviews are crap, but if I’m going to buy the title, I need to be the one making the assessment of whether it tells me what I need to know. And Valve can’t force people to write good reviews any more than they can force them to make good games (subjective as they both are).

          • Premium User Badge

            jrodman says:

            Be careful though, subjective reviews are actually fine.

            However, as you say, when they muddy the issue by not realizing that they write poor reviews.

          • subedii says:

            Actually I believe that all reviews of games are subjective to a greater or lesser degree. Unless you’re talking about bog standard facts (this game doesn’t have a singleplayer component, that game is first person), most of the rest, particularly how good a game is, is all subjective. It’s just that the good reviews will point out what aspects they found to be subjectively good or not, and say why.

            A review that says “9/10 GAME AWESOME!” Is useless to me.

            Whereas a review that says something like: “this game makes improvements on [prequel / other title / comparison point] by better implementation and new mechanics [a,b,c], but it also has issues in that [x,y,z] aren’t as well implemented as they could be because”…

            … holds a lot more value for me. Maybe [a,b,c] is what I’ve always wanted in a game. Maybe [x,y,z] is something I’ve always happily tolerated or don’t care about anyway. Or maybe the opposite. But at least I’ve got a better idea of whether I might be interested.

          • Premium User Badge

            jrodman says:

            I’d sum that up as “content-free reviews”.

          • YogSo says:

            @subedii:

            Actually I believe that all reviews of games are subjective

            All reviews, except the Objective Game Reviews, of course, which proved, once and for all, that Deus Ex is objectively one point better than Human Revolution ;-)

    • SkittleDiddler says:

      Wow, only 2 of the 50 top-voted reviews for Gone Home are actually positive?

      It’s like there’s no middle line when it comes to Steam’s crappy voting system – it’s either haterade or fanboyism that decides the top spots.

      • Chalky says:

        I am having trouble working out if this is intentionally ironic.

        • SkittleDiddler says:

          Say “ironic” enough out loud and it loses all meaning.

          • Premium User Badge

            jrodman says:

            Does this work for the whole world? If so please let me know how many times I must do it.

          • Premium User Badge

            RobF says:

            What about if I say “Candyman” out loud lots? Same or different?

          • Ahtaps says:

            Careful, if you say “ironic” out loud 3 times, you’ll summon Alanis Morrisette.

      • Shuck says:

        That’s the problem of online reviews and responses in general – it attracts the people who feel unusually strongly, and doesn’t represent the majority of players. This is why – and this sounds terrible – most game developers ignore community feedback, because it’s inherently misleading.

      • Arona Daal says:

        Steams voting System let´s you only choose between “recommend”(Good) and “do not recommend”(Bad).
        That makes it unnecessary difficult to quickly express a “middle line” Opinion about a Game.
        As someone who has written and read quite a few Steam Reviews, i think more Options would be nice.

      • Premium User Badge

        caff says:

        I’m quite sad that Gone Home has been subjected to such negativity, but I suspect this has come about through youngsters with a desperate desire for a “game” rather than an “experience”.

        • dskzero says:

          The controversy regarding its lenght, subject matter, and the media frenzy loving it make any kind of review for such a game useless. That said, in this case, people vote with their wallet.

    • battles_atlas says:

      You get that ‘voting’ and ‘staying ignorant’ are not opposites right?

      • Baines says:

        Come, you know that the only people who are intelligent are those who share your opinions.

        Even Nathan knocks “helpful” negative reviews of Gone Home in the article

        On the other hand, I look at Gone Home’s “Most Helpful” reviews and I see reviews that are negative, but rather informative of why they are negative. As for even the positive reviews of Gone Home being a bit negative, they are acknowledging that Gone Home isn’t going to be a game for everybody. Both to me are helpful.

        • Premium User Badge

          jezcentral says:

          Indeed. I really liked Gone Home, and want everyone to play it, but I wouldn’t recommend it outside of a sale. A 90 minute game with no replayability for 16 quid? That’s an almost Thirty Flights Of Loving lack of value for money. (And, no, I don’t know why I like the former and hate the latter. Maybe I should write a review, and work it out).

  2. magogjack says:

    That review must be for either DayZ or Rust…

  3. AngoraFish says:

    Steam reviews have a huge number of flaws, the most prominent being that reviews that don’t recommend a game inevitably get marked as unhelpful by fan-boys and drop off the first page incredibly quickly (while one word positive reviews get up-voted).

    This is presumably because fan-boys are more likely to be scanning the various pages associated with their favourite game, while people who dislike the game have long since moved elsewhere.

    In practice, voting on helpfulness is simply taken by the bulk of users as an opportunity to vote on whether they like the game or not by up-voting those that agree with them and down-voting those that do not, with little reference to the quality of the actual review.

    The second most problematic aspect is that the reviews are organised to show submissions from the last week up-front by default, which means that even detailed, well thought out and comprehensive reviews drop off the front page quickly in favour of one and two line dross that happens to be more recent. For this reason the addition of an “early access” tag is neither here nor there as far as I can see… those reviews will be far out of sight very quickly once a game leaves early access anyhow.

    The fact that reviews are quite difficult to find in the first place also makes reviewing an exercise that ends up being more about personal venting and reflections, with a bit of advice to friends (with the reviews at least appearing on the friends activity list). The bulk of useful discussion of the games themselves remains on the much easier to find dedicated forums.

    The “Gone Home” problem, on the other hand, seems to me more a problem with differing taste, that is, RPS readers wanting a more sophisticated PC experience vs the bulk of Steam users who are fully entitled to consider Gone Home “not a game” and review accordingly.

    • Naed says:

      There are 2 fairly big problems (imho) with the system as it is.

      If a developer doesnt like a review, they can tell Steam that the review was written based on a beta/test version of the game and have your review banned because its not based on the newest version anymore.

      There is also the option for a developer to mark a review as “off-topic”, hence basically “hiding” it until clicked on.

      http://puu.sh/6YSZd.jpg

      • joa says:

        To be fair that review is off-topic, because all you talk about is how the developer handled community relations. Reviews should be for the game itself.

        • Naed says:

          Correct, because I wrote the steam review after I posted on reddit.

          I thought it would be fairly silly to write the exact same thing again. :)

          But go and have a read if you want the “rest” of it :)
          http://www.reddit.com/r/darkout/comments/1wp951/dear_dev_your_community_wanted_help_and_to_help/

          I would like to ask you where (if not in a review) should a player bring up things that they don’t agree with in regards to the game/dev

          In my specific case, the forum was not an option at that point.

          • sirdavies says:

            In the game’s hub, there’s a “discussion” section precisely for that type of stuff. Either way, what you wrote wasn’t a review, so the ban is legitimate.

          • Baines says:

            Said the game had promise, voiced concern over the dev, described being banned from two forums for asking about the state of the game.

            Sounds like the kind of information that should be made available to potential buyers to me. Even if it is the kind of information that certain types of publishers want to suppress, and which Valve is quite happy to enable suppression of.

            I understand why Valve ultimately banned the review, but that itself speaks to multiple issues with Steam. (Just as I can guess why the user was banned from the Steam forum. But while the banning might be legal, it doesn’t make it “right”, and speaks to issues with how Valve handles Steam’s forums.)

          • Naed says:

            “In the game’s hub, there’s a “discussion” section precisely for that type of stuff. Either way, what you wrote wasn’t a review, so the ban is legitimate.”

            Re:Sirdavies: No, the “off topic” is a question of debate, which I’m perfectly fine with.
            The ability for a dev to claim that their own software, in a earlier version is “magically” a test/beta version of their product and use that as an excuse to ban a review (while keeping other reviews done with the exact same version of the game online)…… that im NOT okay with.

            “(Just as I can guess why the user was banned from the Steam forum. But while the banning might be legal, it doesn’t make it “right”, and speaks to issues with how Valve handles Steam’s forums.)”

            Re:Baines: Please enlighten me with your guess? :)

          • Baines says:

            My guess(es)?

            For the review being banned, you mentioned being banned on the Steam forums for asking about the state of the game. Valve doesn’t like people contesting Steam forum bans in public, even if you are the one who has been wronged. (The more cynically minded might argue that Valve is more eager to bury evidence of mods, developers, or publishers abusing the system.) I believe the Steam forum rules even say that it is a violation. Disregarding everything else, once the review was flagged and brought to Valve’s attention, there was a good chance that Valve was going to ban the review for the mention of the Steam forum ban.

            For being banned from the Steam forums, if your Steam forum posts mentioned previously being banned from the game’s forum, then Valve could choose to side with the mod/publisher for reasons similar to the paragraph above. Alternatively, you could have fallen victim to the trolling rules. You can trip across that in a variety of ways, by making multiple posts that are negative about a game, by being one of many users being negative about a game, or simply by complaining about something that others have already complained about. Circumstantial evidence implies that Valve prefers to side with the publisher and mods over users, even when it is the publisher and/or mods that are in the wrong. (It appears to take flagrant repeated abuses of power resulting in an ensuing public ****storm, for Valve to act against a publisher or mods.) My guess is that that is what triggered the forum ban, that you were tagged for trolling or being hostile or whatever. (I’m not saying that you were being hostile, but that you were tagged for it.)

          • Naed says:

            Baines: Aye, all of that is in line with my own take on it :) With a bit of extra sauce here and there.

            Was just curious if someone (in this case you) had a different perspective on it :)

            Thank you for the response :)

    • Geebs says:

      Steam is an appropriate place for users to review a product based on whether they thought they got value for money, which pretty comprehensively explains the response to Gone Home.

      As to the business of how “problematic” that somebody’s TL:DR got moved off the front page due to the passage of time: gosh, it’s simply awful how everybody gets a turn!

      • Naed says:

        now if “only” it was time that moved things.

      • AngoraFish says:

        I can’t see how giving everyone a “turn” is in any way relevant. Surely the point of reviews is that they are supposed to be (most) “helpful” to potential buyers of the game, not to give equal time to the most recent random brain fart, no matter how glib?

        The theory behind up-voting system appears to be to give some prominence to the best reviews to enable users to sort the dross from the chaff – a system that Valve immediately undermines by then dropping those same reviews off the front page a week after they are posted.

    • joa says:

      Indeed, the response to Gone Home is acceptable (even though I quite liked the game) for the fact that it has been forced on everyone as something they should like. The majority of people viewing that page probably wouldn’t like the game. Therefore negative reviews are helpful. If the game had been promoted as something that’s perhaps not for everyone, then the page would have an audience more in tune with who the game was aimed at, and at least some of those reviews wouldn’t be necessary.

      • Premium User Badge

        RobF says:

        It’s not been forced upon anyone. Don’t be silly. People who liked it said that they liked it.

        • joa says:

          It’s not been forced on anyone, but it’s been given exposure close to that of the most popular conventional games, which naturally gives people the completely wrong idea of what kind of game it is.

          It’s like if Hipster Indie Flick was given as much attention as Batman. There would be a lot of disappointed people expecting a conventional movie. Movie journalists have already figured that out; I suppose games journalists haven’t yet.

          • Premium User Badge

            RobF says:

            Oh get away. You say “popular conventional games” like The Sims isn’t one of the biggest games on the planet. Or like millions of people don’t play casual games, like before Steam was a thing and mums and housewives *dominated* the market for digital games doing numbers most developers would *still* kill for and on and on.

            Every single game ever made is “not for everyone” and what makes up the world of gaming is broad and vast and encompasses many, many things. Whether you’re clicking on heads to blow them apart, clicking on things to find the clue to solving a puzzle or clicking on objects to make them disappear or just staring at some really cool pictures on the screen, there’s room for it all. No-one needs protecting from it. It won’t hurt them. It won’t diminish their favourite genre any.

            The very idea that we should funnel things, not effusively praise things we find good in mainstream press lest people accidentally think it’s going to play like Call Of Duty or something is incredibly dumb. It’s treating people like they’re dumb when they’re not.

            Shouting over Gone Home because it’s not the kind of game you think we should discuss in terms of what is a proper game is like standing in the jazz section at HMV telling people that “the Mumford And Sons is over there, love. Go over there, love. This isn’t music. They’re forcing John Zorn down my throat here. Go over there.”

            Your idea of what is popular is narrow and what you seem to believe is conventional is broken. Sorry.

            And people are praising Gone Home because they *like* Gone Home and that’s just perfectly fine. You don’t have to agree with that praise, like I don’t agree with anyone who says that Bioshock Infinite is anything but utter wasteful exhausting guff, no-one’s going to make you. But you don’t need to protect anyone from it either or warn them away.

            It’s OK to like different things in videogames and whether you like it or not, they’re going to keep expanding outwards regardless so it’s up to you to deal with it because it’s not stopping any time soon.

  4. Gothnak says:

    They should have really obvious stats indicating what % of people played the game for over 2 hours. I’d suggest most people turn off from a rubbish game by that amount of time.

    • Yglorba says:

      That would punish games that have been bundled, since a game that’s in a bundle has probably ended up in the libraries of a lot of people who are only tangentially interested in it (if at all) — they might have tried it once for a little bit, then forgot about it because it wasn’t really the reason they bought that bundle anyway (and it might be a genre they don’t like, etc.) That doesn’t mean it’s a bad game.

      • Chalky says:

        Yeah, I don’t want games getting penalised because I’m the sort of person who buys things on sale then doesn’t play them very much.

        Also, length of time played isn’t directly relative to whether it’s a good game for you. I’ve played hundreds and hundreds of hours of CK2 but I can understand how it wouldn’t be many people’s cup of tea – a rating system based on time played would probably put that game up there as the greatest game of all time, but shorter indie titles that might have a broader appeal would rank very poorly.

      • ScubaMonster says:

        No, he’s talking about reviews tagged to show how much play time they spent before writing the review and being able to sort it.

    • Naed says:

      playtime sorting is not the “best” way of sorting, at least not by default.

      Also, what happens if you play a game somewhere else, another platform etc etc, and then buy the game on steam in a bundle/sale/something and write a review.

      With the playtime sorting, that review, no matter your previous experience, would end up far down the list.

      Mind, im not saying that the current sorting scheme is good. :)

    • Geebs says:

      How then would somebody review a game that’s broken to warn other people of that brokenness before they put their money down? Case in point: the mac version of Proteus is broken and costs 7 quid.

      Edit: oh I see, you meant “rate by time spent playing”, not “you must have played this long to vote”. Point taken.

  5. The Dark One says:

    Wouldn’t it be neat if they could use which reviews you’d found useful in the past to weight the pool of reviews for the other games you were looking at? That way that certain segment could complain about the gynocracy, and us walking simulator enthusiasts could be well served, too.

  6. BobbyDylan says:

    TBH, Steam’s whole storefront needs an overhaul. Shopping in it is no longer like browsing the isles of your favorite store, but more akin to wading through Molasses fishing for diamonds.

    And quality control. Greenlight was a mistake.

    • Premium User Badge

      Gap Gen says:

      At least it’s not the Google Play store.

      • Eight Rooks says:

        Give it five years, if that, and it will be.

        Or alternatively we’ll have “The RockPaperShotgun Steam Store” (along with ten thousand others, bigger or smaller), and the user reviews on there will be basically the worst of the comments here magnified a thousand-fold.

        • Premium User Badge

          Gap Gen says:

          That’s an interesting idea; sites like RPS could collate their own sub-stores with games they’ve played to separate out the dross from stuff actually worth playing.

        • Shuck says:

          From what Gabe has said, what they’re looking to make would be worse than the Google Play store. They’re essentially turning Steam into the internet. In order to work, the Steam client would have to turn into a real web browser rather than a storefront. Instead of having an RPS store on Steam, you’d go read RPS and they’d have a Steam widget to buy games.

  7. GenBanks says:

    I hadn’t seen Steam’s ‘Gone Home’ user reviews, it seems a mob mentality has formed against the game. I’ve enjoyed it, but I suppose those reviews might partly be a backlash from people who see a grandiose quote from the New York Times (which is sort of the US equivalent of the Guardian in the UK) on the store page, and immediately roll their eyes at the whole ostentatious self-aware ‘this game is a piece of art’ thing. They are then biased against it, just as a journalist from RPS is (maybe?) predisposed to like that sort of game, since they come to these things looking to form a ‘cultured’ perspective. I suppose it’s a similar divide as you see in film, with ‘high-brow’ but less accessible titles getting ridiculed by some and praised unreservedly by others.

    I think ‘games’ is in some ways too broad of a category to describe it. The way I see it, there’s a divide between arts based and sports based games. Arts based games create enjoyment by helping the player’s imagination (Total War single player, Skyrim, Gone Home, certain MMOs), sports based ones encourage enjoyment of the game mechanics distilled to be a more precise form of challenge/competition (Call of Duty, StarCraft, Dota). There’s some gray area, and you can play things like Total War in a way where you are mainly trying to max out your skill, efficiency and performance, but generally I think people often have a preference for one playing style over another. Perhaps this can also partly explain hugely different conclusions over games like Gone Home?

    • Chalky says:

      I think it’s fairly understandable that someone who’s familiar with how most games work could be unhappy with Gone Home. From looking at it briefly, you could think it was an adventure game, or a horror game, or a puzzle game. Even if you were unsure, you could be forgiven for assuming that it would be at least one of those things.

      In reality, it is more of a story than a game and this is a pretty unique thing. That it is not what the majority(?) of player reviewers were looking for that when they bought it is not entirely surprising.

      • GenBanks says:

        I definitely agree, it’s sort of like going to watch ‘The Wind That Shakes the Barley’ based on positive reviews with the expectation of it being a normal war film, or ‘The Assassination of Jesse James’ expecting a conventional cowboy movie.

        But there is also a cultural divide with people who actively dislike things which don’t fit into the expected categories, not simply because they were surprised, but also because they see it as pretentious, which I think could be contributing to the negative reactions to ‘Gone Home’… Or ‘Dear Esther’.

        • Chalky says:

          I’m not sure there’s all that much wrong with giving a negative review to something you didn’t enjoy because you found it pretentious if you paid for it and were disappointed.

          Personally, I found Gone Home to be dull rather than pretentious, however, a lot of that distinction comes from perceived intent. If I thought the people who made it believed that they had made some sort of life changing masterpiece rather than a simple interactive story, then I probably would have called it pretentious, but since I didn’t see it in that way I wouldn’t describe it like that.

          I guess that is one way in which the media coverage probably did negatively affect the game – if you read a review saying it was amazing but simply found it dull, you may well say it was pretentious even if it was the review you read rather than the developers intent that made you see it that way.

    • MattM says:

      When reading user reviews on Audible I find that narrow audience, genre junk food gets 5 stars from fans who have low expectations and classics of sci-fi get 3.5-4 stars from well-read critics who are evaluating the book by a harsh rubric. I like both kinds of stories, but the star ratings are pretty much useless when I am trying to find a new book to read.

      Also on a related note,
      Many people gave “The Path” a negative review because it wasn’t a traditional game. But even on it’s own terms, I don’t think “The Path” succeeded. It was repetitive, slow, and unclear. Although there were branching story paths, your actions didn’t really connect to their outcomes. Instead your ending felt like a random reward. (I would talk about “Gone Home,” but I just got it in a sale and haven’t played it yet.) For a counter example, I think that “The Stanley Parable” was a non-traditional game that succeeded and was fun to play through.

  8. Asurmen says:

    Can someone explain the problems with the storefront? I’m the kind of person who goes into Steam wanting to buy a specific purchase or purchases. I just just search function, go to game place and buy it. No issues so the storefront problem goes completely over my head.

    • subedii says:

      Biggest issue I have with it is that companies at the moment are re-releasing a tonne of old games on Steam, and those get pushed to the top under “new releases”. I think that’s something Valve need to change.

      Other than that, I’m basically like you. I know what I’m seeking to get before I go to the store front typically. On the odd times I’ve been looking less specific, I’ve been browsing by tags (another thing that was slated to a RIDICULOUS degree when it first hit beta, and took all of… 48 hours to correct the major issues).

      • JohnnyPanzer says:

        I’m the opposite to you guys (a notorious browser) and I -still- don’t understand what the problem is. While I agree that a few minor improvments should be imlemented, I’ve never been found the store front to be a total mess. The few issues I have are:

        1. The release date thing you mentioned. While I have no problems with games recently being added to steam going to the top of new releases, I find it infuriating that the actual release DATE stated only covers it’s release on steam. I find it difficult to believe that a lot of people are more interested in when the game was added to steam as opposed to when it was actually produced.

        Including it in new releases makes sense, as the page primary function (in my opinion) is to show you games that you haven’t seen before due to the fact that they weren’t available unitll now. But stating that the game was “released” in may 2014 means I have to go through google to find out if it’s really a new game or an old game that I’ve never heard of.

        2. Less games included in more than one genre tab. Yes, some games truly deserve to be included in more than one genre, but when I browse RPG games I don’t find it helpful to also see every single game that has -any- sort of character progression. When I’m browsing for simulators I don’t care about all the games that -could- possibly be considered to -sorta- simulate some sort of real life aspect.

        3. Not being able to browse by negative genre selection. Being able to tell steam to show me all games that aren’t action games would be extremely helpful, as would the ability to mix and match my own tabs.

        Other than that, I’m perfectly happy with the store front.

        • subedii says:

          With regards to point 2 (and in a sense, point 3), that’s why I browse by tags instead of genres. Genres are so vague as to be effectively meaningless. But if I want to browse by horror games, or twin-stick shooters, or spectacle fighters, I can do that now. And again, as much as everyone pushed out a sheer wall of abject rage over it, I’ve found the tagging system extremely useful as a result.

          • Premium User Badge

            jrodman says:

            Unfortunately 30% of the tags are wrong.

          • subedii says:

            Looking at the store pages for almost all the games in my library, I’m extremely hard pressed to find incorrect tags for them. The most I can find is the occasional joke one (Saints Row 4 for example, apart from all the standard tags, has a “wub wub wub” tag). More to the point, if there’s an obscure new tag when you go looking at its tags, searching that tag will turn up no other titles for it. Which again, is part of the objective.

            Unless you’re taking objection to something like “Gone Home” having a “walking simulator” tag attached to it, which frankly I’ve never had issue with.

            Otherwise, I’d be interested in seeing all these games that have 30% of their tags being incorrect. Or maybe you mean that 30% of games have an occasional incorrect tag applied to them, but again, that’s not something I have major issue with.

  9. w0bbl3r says:

    When I see many reviews on the first page completely slating a game as if it’s the worst thing ever made (like plumbers don’t wear ties kind of criticism I am talking about), and then I look and see they have played for over 200 hours (I have seen one review from someone that had almost 2,000 hours), I can’t take anything on that page seriously.
    Someone loving the game, having played it for less than an hour? Nonsense.
    Hating it, yet played for more than 10 hours (except maybe an MMO)? Nonsense.
    Valve are “stepping away” from steam. But they aren’t “stepping away” from those huge profits.
    They expect the user to make their money for them now. Not that this is new, with them selling user made content for games for years now. But it’s getting beyond a joke now.
    They don’t police anything that is released on steam anymore (to the point where someone like that douchebag Muxwell can release games on there), not even making sure the release date is correct. Many games in the “new release” section are games 10 years old and more, with a release date of that day, even when you go to the games store page it sometimes still says it was released that day, even though it is clearly an ancient (and usually terrible) game that some douchebag has managed to snag the rights to for a fiver.
    Valve should get their shit together, or give steam to a company that will care. They are rapidly becoming one of my most hated companies in gaming. They used to be just about my favourite company.

    • subedii says:

      I’ve disliked a lot of games that have taken a lot of time. If you don’t play it through, typically it’s much harder to trust someone’s assessment of it.

      Bioshock Infinite for example, is something that every major publication gushed about as GOTY all years. I kept playing through trying to find the awesome gameplay that everyone seemed to be talking about, couldn’t find it. And then the ending came and made things so much worse.

      Regardless of your perspective on the game, mine was basically that it was the worst entry in the franchise. But I couldn’t really make that assessment until I’d, well, PLAYED it.

    • Baines says:

      They don’t police anything that is released on steam anymore

      Did Valve ever police anything released on Steam? They were quite willing to sell garbage games before Greenlight even existed. They quite happily sold and continue to sell games that are highly problematic on modern PCs, without any warning. They side with publishers against customers. They routinely make it troublesome or outright refuse to give refunds. Etc.

      Greenlight, and the recent flood of non-Greenlight old PC games, have only helped bring into the light issues that had already existed for many years.

  10. Premium User Badge

    9of9 says:

    I still don’t quite see why there it’s not possible to just rate a game that you’ve played. Reviews are nice and all, but it seems unreasonable that I have to actually write out a proper review if I want to have any influence on the game’s standing whatsoever. A simple rating system, surely, perhaps with automated prompts to remind people to toss their own opinion of however many stars in after a few hours of play, would give a somewhat more accurate view of whether the people buying it are, on the whole, satisfied?

    • Geebs says:

      I prefer the “recommended / not recommended” approach. Otherwise everything ends up being a 9.5, 7.0 or 3.0 and you might as well go to Metacritic if you want that.

      • Eight Rooks says:

        The argument is that “recommended/not recommended” turns into the exact same thing. You might think a “not recommended” review for Gone Home that’s hit “85% of people think this review was helpful” or whatever is a perfectly valid expression of the majority’s opinion that needs to be seen – I think it’s a knee-jerk over-reaction from terrified nerds fretting that these awful not-games about Stuff They Don’t Like are killing their hobby, one which needs to be shouted down for the good of the medium as a whole, and every bit as useless and unhelpful in the long run as “ZOMG Mass Effect 3 -1,000,000/10 !!!!11!1!1!!” user reviews on Metacritic.

        • Ravenholme says:

          Nice job trivializing everyone who might have a different opinion to your good self. Othering is so cool.

        • almostDead says:

          Here is a representative ‘terrified nerd’ from the Gone Home ‘not recommended’ front page. He sounds awful, reactionary, very immature, and all the other terrible things you imagine:

          Finally got around to playing Gone Home, after hearing a lot of praise from both friends and the general gaming press, having heard “Game of the Year” mentioned more than once.

          Firstly, calling this a game, at least in the traditional sense, is doing it a great disservice, as a lot of people will expect more out of it than it delivers, it should rather be described as a digital short story (and, short as in two hours, give or take, even when you’re really trying to prolong the experience) than a game.

          Gameplay can be described as walking into a room, meticulously going over every nook, cranny and crevice until you find a piece of information (almost always on a piece of paper), and then either listening to a short audioclip that furthers the telling of the overall story, or reading a tiny bit of information about one of the sub-plots.

          The main story is ok-ish, albeit very clichéed, and more or less all of the sub-plots are uninteristing and will leave you without closure, and you’ll just have to speculate about the whats and whys. I must give credit for the great voiceacting, though, the actress who voiced Sam did a pretty fantastic job throughout.

          Furthermore, it annoyed me immensely that a lot of the clues are found in highly illogical places, it complete breaks immersion for me to think that someone would leave information of a certain nature in a highly visible place, or that notes that are decades old would still be found where they would surely have either been destroyed, or removed by the persons living in the house.

          This is even more compounded by the fact that the developers have done a pretty good job of making the house seem very real, more or less all drawers, closets, microwaves etc. can be opened to search for clues, but then again, this was most probably done to further the longevity of the game, you spend a lot of time opening stuff that have no items of interest in them.

          All in all, I can’t really recommend this game, its an interesting way to tell a story, and if the story had been better, it could really have been an experience, but as it is, this game should in no way be a contender for a GotY title.

          • SkittleDiddler says:

            Oh, for Pete’s sake, there was nothing intrinsically wrong with that review. Beside the fact that you didn’t like it, that is.

          • subedii says:

            That’s his point. The article, not to mention a fair few commentors, are frequently conflating a negative view of Gone Home as being childishly reactionary against it for stupid reasons. Hence why almostDead posted that in response to Rook’s post. Because Rook said that the fact that negative reviews are getting plenty of approvals is merely evidence of ” knee-jerk over-reaction from terrified nerds” which should actually be “shouted down for the good of the medium”. By default he stated that it’s not a “perfectly valid expression”.

            Those are all his words, not almostDead’s.

          • SkittleDiddler says:

            Ah, my mistake.

      • Premium User Badge

        9of9 says:

        Recommend/Not Recommend muddle the waters, I think, on account of you having to make a judgement on whether some indeterminate other person would enjoy this game or not. You can’t really offer a recommendation the public in general and far too often you will find something that you will like, for instance, but can’t really recommend because you know you’re in a niche audience for it… it just confuses matters, I think.

        My point is more that it would be nice to collect much more data from Steam users. Greenlight games collect thousands of Yes/No votes, but released games only get a few reviews. Certainly, having an AppStore-type starring system has its own issues, but at least that provides a good basis for datamining existing ratings and trying to get more useful information out of it. When it comes to the slew of smaller, indie titles, I feel like you really need as wide a breadth of information as possible other than just ‘how many times this sold’.

        I bought Cognition a while back when it first came out, and loathed the game from its art down to its writing. More recently picked up Cinders, which was a very low-key release and quickly buried by a slew of other titles and absolutely loved it to bits (really, TellTale can learn a thing or two from it about branching narratives). My problem is that I – as an average consumer – don’t really have any straightforward way of registering my personal experience with these two games in any meaningful way. All I can do is write reviews for them, which has too much of a barrier for entry for my liking. I don’t see the point of writing a few quick sentences as a ‘review’ and I don’t have the time to contribute an in-depth discussion of a game out of my own good will.

        Regardless, even judging by reviews alone, there remains no way of sorting through existing games by player opinion, at least not that I can see. Which means good but low-key indie games seem to get drowned along with the dross, even in the case of something like Cinders, which presumably does not have overwhelmingly high sales, but does have overwhelmingly positive reviews.

        • Geebs says:

          I think that Valve’s intent is that the actual written review itself is the important part, while the yes/no binary choice is meant to allow people to clarify whether they are overall positive or negative (which can sometimes be hard to infer).

          I like this system because the value is in another person’s explanation of why they hold a given opinion.

          (additional bonus point: if you’re doing a score out of 10 you generally have to allocate some of your marks to the quality/competency of the game’s construction, which is why so many polished-but-dull mainstream games are in the 7-8 range)

          • subedii says:

            I think that Valve’s intent is that the actual written review itself is the important part, while the yes/no binary choice is meant to allow people to clarify whether they are overall positive or negative (which can sometimes be hard to infer).

            Basically my view on it.

            I mean crikey, I’m reading this on RPS right now, a site that specifically has a policy AGAINST point based ratings. Because an arbitrary value rarely tells me jack-all beyond whether the author liked or disliked the game to begin with anyway (So many, so many 8/10 – 10/10 reviewed games that I’ve disliked or outright hated over the years). Points certainly don’t quantify “how much I will enjoy a game”.

            After that, I’m meant to read the review and see if what the author takes us through gives me an idea of whether I’d like the game or not compared to what I’m looking for in the title.

        • Baines says:

          Greenlight’s voting is more worthless than Steam’s regular reviews. There, you have people voting on whether they like the concept of a game often without ever having played it or even knowing what state the game will be released in (if it is ever released). On top of that, you have the joke votes, the kneejerk votes, the vote-every-game-up votes… And then you have flood of “I bought this game in a bundle, so I’m voting it up in the hopes that I get a free Steam key if it gets Greenlit” votes.

          Valve’s not very good with interacting with the public.

    • Eight Rooks says:

      Yes, because the user reviews on Apple’s app store or Google Play are such a sterling indication of whether something’s quality or not. Oh, no, wait, no they’re not.

      There is no way to stop people being asses with something like this. Trolls will always, always, always find a way around whatever barriers you throw in front of them. It’s either accept that your efforts to shape people’s behaviour are never going to work 100%, or become a curator. I’d rather more things were curated (Apple’s “walled garden” bothers me very little, if at all), and I don’t think most end users are very good at curation/expressing their opinions, but I accept (if unwillingly) that’s not the way the internet is going

    • trjp says:

      Fuck ratings…

      People are dumb at the best of times – but ratings make them mega-dumb.

      I saw an AppStore review last week which said

      1/5 – I’ve not played this game but I had to post this because dumb people post 5/5s

      Then you get the nerds and their

      2/5 – I wanted this to be 3.364 but the system wouldn’t allow that so I deducted 1 point for their lack of floating point numbers and then rounded down.

      The only ratings I’ve ever seen which work are on GoodReads – except that people ignore how they’re supposed to work.

      They use 5 stars – but they mean

      1 star – did not enjoy
      2 stars – it was OK
      3 stars – I enjoyed it
      4 stars – it was great
      5 stars – I loved it

      Note the heavy positive bias – note that most people ignore the system entirely (they should change it to text now and see how many people revise 2/3/4s to 1s ;0

      • sekullbe says:

        My favorite dumb reviews are the flood of hundreds of 0/10s for ‘Out of the Park Baseball 2007′. It’s tied (with Half-Life) for highest-rated on Metacritic because it’s an excellent game in a tiny niche and only the kind of sites that would really love the game reviewed it. For some people this is Unacceptable! and Will Not Stand! so they 0-rate the hell out of it.

        Of course it’s the Internet and you can find a few hundred idiots to do *anything* but still it’s telling.

      • Premium User Badge

        Cinek says:

        For me 5 stars is:
        1. Complete crap. Never should come out of a drawing board. Poor execution of a poor idea.
        2. Poor execution, boring game, but got some good things in it (eg. underlying idea)
        3. Average
        4. Decent game, lacks in few things (in case of iOS games it’s usually SFX/soundtrack or lazy graphics)
        5. Great game, very fun to play, highly above average in terms of graphics and sound design.

        I like to score games as a sum of all things, not just whatever I enjoyed them or not – sure it’s a major factor, but not the only one.

        • Nest says:

          This is why “recommended/not recommended” is the best rating system for user reviews. Nobody can agree on what each of the points in a 5 star system really means, and the 100 point systems used by mainstream review sites are abused even by ostensibly professional reviewers.

  11. internisus says:

    The storefront was already an embarrassment before they changed the default tab to Top Sellers last week, effectively hiding New Releases and killing the visibility of lesser-known titles that have just come out in favor of already-proven commercial successes that simply don’t need this kind of a boost. Now it’s a complete disaster.

    I’m very surprised that there hasn’t been coverage of this in the gaming press complete with quotes from angry indie developers whose new games haven’t even been noticed because the store is busy reminding everyone that DayZ exists instead.

    • trjp says:

      That was a stupid thing to do – it made me change my default tab from ‘Store’ for the first time in 7 years.

      “Hey – people are upset that the main reason for being on Steam – getting on the New Release list – no-longer works”

      “Fuck em – remove the list – remove their heads”

    • Lemming says:

      ” before they changed the default tab to Top Sellers last week, effectively hiding New Releases”

      Excuse me my hyperbole detector appears to have exploded. Don’t talk bollocks. It’s right there, in a bunch of tabs you have to scroll down past the ‘featured’ section to see anyway. If you’ve got that far, it’s safe to say you’re looking for more than what is immediately right in front of your nose.

    • Jalan says:

      Agreed, having the focus shifted by default to the top sellers is not something I find helpful and would much prefer an option to at least choose which tab I wanted to take prominence in profile settings or something (perhaps a job best suited to something like a browser extension such as Better Steam since Valve is unlikely to do such a thing on their own). The top tab has always been a “no shit”/”well, duh” kind of showcase in my eyes (stuff like DayZ and any other pre-order title or current running deal(s) are guaranteed to hog the spotlight) and I’d much rather disregard the tab entirely in favor of seeing what’s been recently released or even what’s forthcoming at some point.

  12. phenom_x8 says:

    Yeah sometimes I read user review for the game I’ve bought in Steam , you know just to make sure I didn’t get the wrong choice or just to know somebody who have the same thought with me about the game. For the game that I’m not already bought, my trust are lied with you ,RPS.

    Btw, waiting for Dreadout WiT you’ve been wrote sometimes ago
    http://www.rockpapershotgun.com/2014/05/06/ghostbustr-survival-horror-dreadout-haunting-next-week/

    You know ,just to confirmed it. Because many reviewers I already read said that the game quite different with many todays horror game that relied on jumpscare. It more akin with Silent Hill or Fatal Frame with different unique ghosts and some puzzle that makes you thinks harder to figure it out

  13. derbefrier says:

    I dunno I think its not bad. Sure there are some stupid “reviews”. Those arer easy enought to spot and ignore. Generally if the review is longer than a paragraph its safe to say its probably worth reading. Also don’t forget at the bottom of the store page you can sort by positive and negative reviews if you want a more balenced view of the game. I mean its like any other iser review site, if you know what to look for it can be helpful. If you are just going there to see how many people are trolling a game you love you’ll hate it.

    • subedii says:

      This is basically my take on it. I prefer Steams review system to basically any other online store I’ve seen to date (maybe I just haven’t been exposed to the really “top notch” stores or something?).

      Limiting user reviews to a simply “yay” or “nay” is a big plus for me. Point based scoring systems are something game reviewers have been railing against for years, and for good reason. The core of the assessment is always going to be ridiculously subjective. The key thing is actually READING the reviews and seeing how that lines up with what your expectations are and what you’re seeking from a game. A good review will give you an idea of whether or not you’d like to play the game, regardless of whether the author enjoyed it. Scoring beyond that is pointless. Most of the games I’ve ever enjoyed have had a “controversial” review status or “split” audiences, where some will love and others will hate it depending on what they’re after, and a point scoring system can’t capture that. Heck I remember back when Planescape: Torment came out and I kept seeing reviews negatively comparing it to Baldur’s Gate because it wasn’t as open ended in gameplay. So it would often hit the 80% mark instead of BG’s 90%+. But that’s just because they’re trying to apply a score in a ridiculous “cover all bases” way that’s mushes everything together, not necessarily what the reader (or even the reviewer) finds most and least important in the title and by what merits it should be judged.

      They made a conscious decision not to do the ‘gamerankings’ thing (still disagree that you can see the Metacritic ranking for games in places though) and have games ranked by score, and they’ve purposely avoided aggregating positive / negative responses to give the game a score.

      This is also helped by the fact that I can sort by positive and negative reviews so I can see what people’s positives and negatives are. EDIT: And the fact that those from my friends are shown first.

      Honestly, the big complaint seems to be that most of the reviews are crap and non-informative. Well, yeah. Very few people can actually write any kind of decent review, otherwise they’d be doing it professionally. I think the issue only crops up because you can’t simply look up a “score” for the thing on sale (like in other online stores) and make your decision solely off of that. You actually have to READ some of this stuff, see if the author does a decent job of covering the points of the game you’re interested in, and make your assessments off of that.

      • Premium User Badge

        Cinek says:

        “Limiting user reviews to a simply “yay” or “nay” is a big plus for me.” – totally agreed.

      • toxic avenger says:

        You act like reading a handful–say 10 reviews–is an arduous task when you weigh it against the proposition of blindly spending 60 USD. Since when did reading/writing become so difficult that even the thought of which causes people to recoil in horror? Even moreso, when did expressing such an opinion stop being embarrassing?

        “I’d like to make a less informed opinion about where I drop 60 or so dollars. In fact, make the system even less representative of the product by obscuring it with a simple number average that provides no context (was the game a parody, was it funny to people with an IQ below 40 only?) whatsoever!”

    • DanMan says:

      I don’t read them, if they’re too long though. I’m not interested in reading a long winded review from the average internet person. I much prefer reading multiple ones from multiple people which are on the short side. Positive and negative. How else can you possibly get a somewhat useful overview?

  14. trjp says:

    Greenlight’s question is

    “Would you buy this game”

    I think that’s dumb (you don’t know price/availability etc.) but they should have stuck with that idea and asked

    “You bought the game then – how was it?”

    [Amazeballs?] [Meh?] [IWANTAREFUND]

    Indeed, it should allow you to change your mind, logging each choice against your playtime…

    IMO you Never ask people for their opinion – who cares about that – just ask them how they FEEL :)

    • trjp says:

      p.s. and replies should be Twitter-style and short – we don’t want an essay, just tell us, just something like

      “I wanted Dungeon Keeper 3″

      or

      “It made by balls tingle!”

      • toxic avenger says:

        No? Count me out for the “I want short reviews” crowd. I don’t have a preference, really. I just want them to be well written and come from a place of honest intentioned curiosity and reason.

  15. Tei says:

    This is a matter of education, and education is a unending task.

    People need to vote from a NPOV way, not by hate/love positions.

    • Premium User Badge

      Cinek says:

      If people could actually do it in most of the topics of their lives – world would be a better place.

    • trjp says:

      You will never educate people to stop having opinions

      You might get them to learn to voice their opinion in a constructive manner – but then you’ll never teach them to edit well

      Web 2.0 introduced this idea of ‘include the user in everything’ – with ratings, likes, comments etc. It never once considered that what 99% of people think, isn’t worth reading.

      People now believe they’re entitled to speak on every topic – despite the fact they usually have nothing to say…

  16. SKapsniak says:

    I really don’t find much point in reviews from random people. Sorting those reviews of random people, by the reviews of the reviews by random people, is meta-silliness.

    Reviews — outside of any literary merit — are helpful, when I have an idea of the tastes of the reviewer. A reviewer who likes everything I like, and hates everything I hate is a reviewer that’s worth paying attention to, as is a reviewer who reliably has the opposite tastes to me. If I agree with one chap’s taste in shoorters, but not in RPGs, but another chap gets the RPGs spot on, but has an deep abiding love puzzle games which never like at all, and I know these things, then we’re getting somewhere.

    Given all the information Valve has (games lists, played time, tags added, recommendations), it should not be beyond them to come up with a tailored list of particular reviewers that that are going to be helpful to me because they have similar (or opposite) tastes. Until they do that then the reviews are fairly pointless.

  17. David Shute says:

    The problem with the Gone Home reviews (which is part of the problem with user-reviews in general,) is that they’re 10% people actually reviewing the game, and 90% people reviewing the genre. If you’re already familiar with the genre, then that latter 90% is beyond useless – I get that a lot of people don’t like them, I really do, but in my case, I already know I like going-for-a-walk games; I just want to know whether this is a good one.

    This is why we have professional reviewers. If I want to know whether a new JRPG (for example) is any good, I’ll read reviews by trusted, reliable reviewers who understand and enjoy the genre, and will judge the game on how it compares with others. The opinion of a reviewer who thinks all JRPGs are cliche-ridden grindy snorefests might well be valid, but it’s simply not relevant or constructive to my own purchasing decision – it’s just noise.

    With user reviews you can’t do that – you’re faced with a mountain of conflicting opinions – all you can do is to trust the average opinion (which, by definition, will be bland and conservative,) or cherry-pick particular reviews from random people you have no reason whatsoever to trust. And the more niche the genre is, or the subject matter, or the art style, then the less helpful the user-reviews become.

  18. twobitcoder says:

    Useful in what sense? In comparison with, say, Amazon reviews? The highly bogus, manipulated, fraudulent “reviews” on Amazon are completely useless. So I hope that isn’t what this author was thinking of as a comparison.

  19. Keyrock says:

    My biggest problem with Steam Reviews, and it’s a problem that won’t get solved without having a dedicated staff to comb through every review or some kind of filtering process that reviews have to pass before they get posted, is that Steam Reviews has become a competition to post the funniest or most “clever” meme or joke you can, because LULZ, or whatever. I guess there is enough like minded individuals out there (I’m assuming these people are mostly 12 year olds, either physically or mentally) that these useless joke reviews get a bunch of “helpful” votes, putting them front and center, while actually helpful, detailed, constructive reviews get bumped down.

    • toxic avenger says:

      This.

      Though, I hope (misplaced, of course) that unpopular reviews that aren’t pointless won’t be at risk by this new news. I wrote a review for the game Underrail, which for various reasons I completely loathed, and was subsequently downvoted into oblivion. Was I harsh? Absolutely. Do I think it was unwarranted? Absolutely not. Was it unpopular? Absolutely! I just see this kind of review going the way of the American buffalo, where only fanboy praise or hate one way or another is the only thing that’s left. Hopefully, that isn’t the case.

  20. Lemming says:

    “Valve Slowly But Surely Making Steam User Reviews Useful”

    Oh for fuck’s sake. They’ve always been useful.

    Unless you’ve got Asperger’s, who honestly can’t tell the joke ones from the real ones, and when looked at as a whole, don’t they all still tell you what to expect from a game?

    Perhaps we can all behave like intelligent people who’ve learned to do more than simply turn on our PCs instead of pretending to be critical of something that really isn’t that big a deal.

  21. ssbowers says:

    Garbage you say! I just got Ubersoldier 2 for $1.50! Not the greatest game in the world, but for $1.50 it is the best investment in my life.

  22. MacTheGeek says:

    My view of the Steam storefront is rather like Winston Churchill’s view of democracy. It’s the worst storefront ever, except for all the others that exist.

  23. DoingitRight says:

    People do not like a game that you only like because it was marketed at you directly by hitting your trigger issues, and it is automatically because they are all bad people and not because the game only appeals to a small minority?

    The hypocrisy is delicious.

  24. sabrage says:

    This’ll be especially helpful when the first game makes it out of Early Access.