Page 1 of 7 123 ... LastLast
Results 1 to 20 of 127
  1. #1
    Secondary Hivemind Nexus
    Join Date
    Aug 2013
    Location
    Germany
    Posts
    1,710

    Game ratings. What's so bad about them?

    First up, I don't want to convince anyone that RPS should have ratings. It's their decision, and I'm fine with that.

    But I think ratings do serve a purpose, so I don't understand why a lot of people despise them so much. Rating stuff like graphics, sound and other technical stuff like options (what? your console port is limited to 1080p resolution?) makes sense to me. You can see at a glimpse how up-to-date a newly released game is compared to other games of the genre at the time, or in other words: how much work has been put into them.

    That said, it may not make sense to have an overall score. Especially if it's an average of the partial scores, 'cause a game is rarely the sum of it's parts. The whole experience needn't be influenced a lot by the technical details.

    But maybe ratings are not the best way to communicate those details either. Maybe it should be a pro/con listing? I do think that you shouldn't have to read a whole article to find out about a few bumps you may run into. Thoughts?

  2. #2
    Secondary Hivemind Nexus QuantaCat's Avatar
    Join Date
    Jun 2011
    Location
    Vienna, Austria
    Posts
    7,558
    because ratings dont mean anything. Its always a personal affair.
    - Tom De Roeck.

    verse publications & The Shopkeeper, an interactive short.

    "Quantacat's name is still recognised even if he watches on with detached eyes like Peter Molyneux over a cube in 3D space, staring at it with tears in his eyes, softly whispering... Someday they'll get it." - The Conclave

  3. #3
    Secondary Hivemind Nexus
    Join Date
    Aug 2013
    Location
    Germany
    Posts
    1,710
    My point was that the technical level (how well the game has been made, as in: craftsmanship) IS quantifiable. You disagree?

  4. #4
    Secondary Hivemind Nexus Tikey's Avatar
    Join Date
    Jun 2011
    Location
    Argentina
    Posts
    3,396
    Because cultural works aren't quantifiable.
    Because the technical aspect is only important as a support for the whole experience.
    It's impossible to accurately condense a whole experience as a number. What makes a game worth playing usually differs from person to person. A number doesn't tell me anything. I might be interested in world building rather than if it supports the latest DirectX. Or I might forgive a few bugs if the game has an interesting mechanic. You can't put that in a number, that's why the article exists in the first place. And that's also why not putting a rating sends an important message. "This is a place that cares about the content.", and that's an important distinction to make.

    Quote Originally Posted by DanMan View Post
    My point was that the technical level (how well the game has been made, as in: craftsmanship) IS quantifiable. You disagree?
    Yes. A lot of Obsidian's games tend to be regarded with high praise despite their technical issues. Reducing them to their techinal aspect is a disservice. A game isn't it's engine or code.
    I go back to the previous paragraph. A rating doesn't add anything to the words written in the article. If there are technical issues or prowess I expect them to be covered there, a number doesn't help me at all.
    Last edited by Tikey; 09-06-2014 at 03:16 PM.

  5. #5
    Secondary Hivemind Nexus Sparkasaurusmex's Avatar
    Join Date
    May 2012
    Location
    Texas
    Posts
    1,474
    There isn't anything inherently wrong with ratings. This problem didn't just come from nowhere. It took some building up. But at this point aggregate sites like metacritic become influential, which also isn't so bad on it's own (and useful for gamers) but this influence sways game design, and causes publishers to not only aim for sales, but specific metacritic averages. This opens the door for corruption and questionable publisher behavior (see Fallout NV Bethesda and Obsidian).

    So the problem with rating a game is that the focus tends to shift over to that number, instead of the reviewer's experience itself. Aggregate sites basically just take averages, but a score may represent entirely different things on different sites.

  6. #6
    Secondary Hivemind Nexus
    Join Date
    Nov 2012
    Posts
    2,575
    Nice post, 7/10.

    It wasn't helpful, right?

  7. #7
    Secondary Hivemind Nexus QuantaCat's Avatar
    Join Date
    Jun 2011
    Location
    Vienna, Austria
    Posts
    7,558
    Quote Originally Posted by Sparkasaurusmex View Post
    There isn't anything inherently wrong with ratings. This problem didn't just come from nowhere. It took some building up. But at this point aggregate sites like metacritic become influential, which also isn't so bad on it's own (and useful for gamers) but this influence sways game design, and causes publishers to not only aim for sales, but specific metacritic averages. This opens the door for corruption and questionable publisher behavior (see Fallout NV Bethesda and Obsidian).

    So the problem with rating a game is that the focus tends to shift over to that number, instead of the reviewer's experience itself. Aggregate sites basically just take averages, but a score may represent entirely different things on different sites.
    Everything zenimax does is questionable.
    - Tom De Roeck.

    verse publications & The Shopkeeper, an interactive short.

    "Quantacat's name is still recognised even if he watches on with detached eyes like Peter Molyneux over a cube in 3D space, staring at it with tears in his eyes, softly whispering... Someday they'll get it." - The Conclave

  8. #8
    Lesser Hivemind Node
    Join Date
    Jun 2011
    Posts
    701
    Quote Originally Posted by DanMan View Post
    My point was that the technical level (how well the game has been made, as in: craftsmanship) IS quantifiable. You disagree?
    How do you rate it though? You can't quantify the number of bugs, because even the developers don't know how many there truly are. Docking points per bug isn't representative; a bug which makes the character's head inflate for half a second under specific circumstances is nowhere near the severity of a crash bug or a save corruption bug.

    How do you objectively rate graphical fidelity? Average out texture sizes and give a score the higher that is? That's going to be meaningful to exactly nobody. The problem is that even something you could call quantifiable like graphics or bugs is in fact not, because what you're reporting is your appreciation of those aspects.

  9. #9
    Secondary Hivemind Nexus QuantaCat's Avatar
    Join Date
    Jun 2011
    Location
    Vienna, Austria
    Posts
    7,558
    Quote Originally Posted by FriendlyFire View Post
    How do you rate it though? You can't quantify the number of bugs, because even the developers don't know how many there truly are. Docking points per bug isn't representative; a bug which makes the character's head inflate for half a second under specific circumstances is nowhere near the severity of a crash bug or a save corruption bug.

    How do you objectively rate graphical fidelity? Average out texture sizes and give a score the higher that is? That's going to be meaningful to exactly nobody. The problem is that even something you could call quantifiable like graphics or bugs is in fact not, because what you're reporting is your appreciation of those aspects.
    also, remember that guy that came here offering a hypeless review of wasteland 2? Youd end up with that. Look how long he lasted.

    You cant put objective numbers on a subjective matter.
    - Tom De Roeck.

    verse publications & The Shopkeeper, an interactive short.

    "Quantacat's name is still recognised even if he watches on with detached eyes like Peter Molyneux over a cube in 3D space, staring at it with tears in his eyes, softly whispering... Someday they'll get it." - The Conclave

  10. #10
    Secondary Hivemind Nexus
    Join Date
    Aug 2013
    Location
    Germany
    Posts
    1,710
    Some of you seem to deliberately misunderstand me. Didn't I write that I can see how an overall, numerical score may not make sense?

    Again. There ought to be something between simply scoring a game with "7/10", and writing a wall of text you have to sift through to get to the bits you want to know.

    Maybe I'm not interested in reading all the witty remarks or following any other tangent the writer goes off on (is that correct english?). On the other hand, a "7/10" doesn't tell me anything about highly subjective, personal no-gos (or is it no-goes?) that the game may contain. I can be very picky when it comes to sloppy game making.

    For some games I want to know everything before I buy it, for others I just want to know if they didn't mess up completely to make a buying decision. The former mostly for games I usually don't buy but am intrigued by, the latter for games I play regularly and am already interested because of the setting or something.
    Last edited by DanMan; 09-06-2014 at 03:40 PM.

  11. #11
    Secondary Hivemind Nexus
    Join Date
    Dec 2012
    Posts
    1,209
    Technical stuff is meaningless too, as it is the effect of its application that matters. You can't rate a cake over the list of its ingredients. Graphics can be top notch, technically, giving all teh effects and all teh pixels, but the overall effect can be disappointing due to bad art design. Sound stopped being technically relevant to everyone but audiophiles somewhere at sound blaster 16.

  12. #12
    Secondary Hivemind Nexus QuantaCat's Avatar
    Join Date
    Jun 2011
    Location
    Vienna, Austria
    Posts
    7,558
    Quote Originally Posted by DanMan View Post
    Some of you seem to deliberately misunderstand me. Didn't I write that I can see how an overall, numerical score may not make sense?

    Again. There ought to be something between simply scoring a game with "7/10" and writing a wall of text you have to sift through to get to the bits you want to know.

    Maybe I'm not interested in reading all the witty remarks or following any other tangent the writer goes off on (is that correct english?). On the other hand, a "7/10" doesn't tell me anything about highly subjective, personal no-gos (or is it no-goes?) that the game may contain. I can be very picky when it comes to sloppy game making.

    For some games I want to know everything before I buy it, for others I just want to know if they didn't mess up completely to make a buying decision. The former mostly for games I usually don't buy but am intrigued by, the latter for games I play regularly and am already interested because of the setting or something.
    thats why usually, on RPS, you can read the closing thoughts, just read the last paragraph.

    Again, I do know what you mean, the score just doesnt make a difference.
    - Tom De Roeck.

    verse publications & The Shopkeeper, an interactive short.

    "Quantacat's name is still recognised even if he watches on with detached eyes like Peter Molyneux over a cube in 3D space, staring at it with tears in his eyes, softly whispering... Someday they'll get it." - The Conclave

  13. #13
    Secondary Hivemind Nexus Drake Sigar's Avatar
    Join Date
    Jun 2011
    Location
    Jolly Ole England
    Posts
    3,429
    Gaming enthusiasts tend to judge scores based on their internal belief of what an 8 or 9 means, when the meaning of a score is entirely dependent on the reviewer. There is no universal system. To get any accuracy from a seemingly arbitrary number, you need to know the reviewer’s tastes and how much they conform with your own. Otherwise there’s no solid context. Scores aren’t useless but they do tend to breed constant misunderstanding because of the way people treat them.

  14. #14
    Secondary Hivemind Nexus
    Join Date
    Aug 2013
    Location
    Germany
    Posts
    1,710
    Quote Originally Posted by FriendlyFire View Post
    How do you rate it though? You can't quantify the number of bugs, because even the developers don't know how many there truly are. Docking points per bug isn't representative; a bug which makes the character's head inflate for half a second under specific circumstances is nowhere near the severity of a crash bug or a save corruption bug.
    Thanks for the proper reply.

    I didn't have bugs in mind when I wrote that, since I expect the devs to fix them after release eventually (unless they have a track record of not doing that). Which raises another interesting question: should reviews be updated, if the game was broken and got fixed later on, improving it significantly But I digress.

    That's where the reviewer comes in. In the end, he/she would have to decide about a bug's severity. In the end, they would have to be considered, yes, according to their severity. But I've already acknowledged that in my initial post when I referred to pro/con lists. So yes, maybe quantifying things in numbers is too difficult, even for technical things. So a bullet point on the "con" list "(annoying|significant|game breaking) amount of bugs" maybe?
    Last edited by DanMan; 09-06-2014 at 03:58 PM.

  15. #15
    Lesser Hivemind Node
    Join Date
    May 2013
    Posts
    648
    Regardless of my personal opinion, which I won't write because I've been flamed enough for it, this discussion is interesting on a wholly different level to me. It's basically a 1:1 Copy of the everlasting war between Quantitative and Qualitative Scientific Research. It perfectly mimics the arguments, it perfectly mimics the baseline issues with either approach and it mimics the line in the sand as well.

    If several decades of bickering have proven one thing in this debate though, it's that neither side is entirely correct and both approaches have their merit. However that is not possible in gaming, because even the slightest divergence of the perceived value of a game, be it through a rating or a written word, causes some people to go completely ballistics.

  16. #16
    Secondary Hivemind Nexus
    Join Date
    Dec 2012
    Posts
    1,209
    Pro/con lists are certainly better than scores, are used by some reviewers. The closing paragraph on RPS often serves as something similar too and I do sometimes skip right to it for that reason.

    Of course, scores are not totally useless, they can be used as a very loose guide when casually searching for something new when bored. But in anything more serious they cause more harm than good.

  17. #17
    Even for the technical aspects of the game it becomes very questionable.

    In order to really rate the technical aspects of an application, whether it is a game or not, you have to know what the requirements really are. Unfortunately, in games that dips into a lot of vague and subjective things.

    Even non-game software that is theoretically far more straightforward and objective than games has problems with this.
    All the time there are problems with changing requirements and not having a clear picture of what an application should and shouldn't do. You often need to exert a not-insignificant amount of effort to keep that potential chaos from becoming chaos.
    Even when you are trying to get something already made, often times you find you can't use otherwise very "good" applications not because they are in any way "bad" but because they don't fit your exact needs even if on the surface they seem to accomplish what you want to do.
    The most reliable way to know if an application will really fit your business is still to literally bring it in and use it for a while to see how things go.

    "Review scores" (or rather their equivalents) are often unreliable even in business software which, while problematic in its own ways, is much more objective, straightforward, and specific about what it does than games can ever be. If such evaluations are of limited reliability there, how would it work for games?
    Simply put, it doesn't.

    With games it goes from being potential mess to utter chaos. Games are an entertainment medium and an art form with numerous potential customers and pictures of what "quality" looks like. The task of figuring out what a game should and shouldn't do becomes pretty much impossible at that point.
    Sure you can point out games that are clearly broken and non-functional, but you can do that without a rating system. For the rest there's no way you are going to get enough precision to present a rating system and honestly claim its accuracy.

  18. #18
    Secondary Hivemind Nexus Fumarole's Avatar
    Join Date
    Jun 2011
    Location
    California
    Posts
    3,803
    I'm cool with a simple recommend/don't recommend concept, but not a score like it was a test.

    What ever happened to the RPS Optimus Prime Thumbs o' Doom anyway?
    The Medallion of the Imperial Psychopath, a Napoleon: Total War AAR
    For the Emperor!, a Total War: Shogun 2: Fall of the Samurai AAR
    The Red Blades, a Battle Brothers AAR

  19. #19
    Secondary Hivemind Nexus Wenz's Avatar
    Join Date
    Oct 2013
    Location
    Gorizia, Italy
    Posts
    4,859
    I'm sure nearly all gaming sites have their own rating system based on some standarts. Some of them may lack consistency between rates and standarts but a single number wears off such thing and gets thrown around anyway, or so I think when a friend of mine working as a gaming journalist laments most reviewers around don't know what they are talking about (that goes for Italy at least).

    Also, as sparkasaurusmex said, more and more games get 8 or something. Metacritic is a bit more subtle than that because while it doesn't rate games it shows a percentage of how many good/mixed/bad rates are out there and thus how popular the game is.
    Last edited by Wenz; 09-06-2014 at 04:17 PM.

  20. #20
    Secondary Hivemind Nexus
    Join Date
    Feb 2012
    Location
    Stockton-on-Tees, UK
    Posts
    2,885
    Review scores, review score aggregates, and user review aggregates are all useful tools if you know how to use them.
    Irrelevant on further examination of the rest of the thread.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •