Review Score Controversies: A Guide

By Alec Meer on November 4th, 2011 at 11:00 am.

__________________

« | »

.

189 Comments »

  1. Jim Rossignol says:

    I am sure I have bored the internet with my anti-scoring rants before, many times. But the “controversies” over review scores are only half the problem. Even general discussion of the merits of a game get caught on “is it really an seven?” when such a discussion is basically meaningless and say NOTHING about the game in question. See Gillen’s Boiling Point reviews on Eurogamer for the purest deconstruction of scoring.

    Mark out of 3D space, or nothing.

    • JuJuCam says:

      The best comments of Eurogamer are the ones that compare how the game in question ranks vs Mafia 2 based on score alone…

    • Richard Cobbett says:

      I enjoyed the reaction to me giving The Witcher 2 89% (for specific things that kept it out of the 90+ bracket) which was treated as the worst score imaginable! I checked a week later. Its Metacritic score was 89%, so under the inexplicable rules of troll logic, I think I can say I was objectively correct.

      Of course, it’s now down to 88% so I feel much shame for my overmarking.

    • Richie Shoemaker says:

      Ah, the old 89%.

      For some reason publishers think giving 89% is just about being mean and vindictive, but that 89/90 divide is (or was) a significant threshold. It’s the difference between nomination and Oscar winner.

      Better do do away with awards entirely if you ask me.

    • diebroken says:

      Well you could always follow up a review with ‘A Defence’ review! *ZING!* ;)

    • atticus says:

      @Richard: Remember the TW2 controversy, and I guess the biggest issue is that almost nobody has a proper sense of scores anymore. On a scale of 0% to 100%, 0% – 85% is shit, 85% – 90% is OK, and 90% – 100%. is good.

      Metacritic may be responsible for some of this focus, but I also feel that reviewers must take their part of the blame as it seems scores just gets higher and higher without proper justification. Your colleague McCormick owes me money for a new sweater, after I wept blood all over one of mine while reading his review of Dragon Age 2.

      Scores are losing their meaning and are becoming just a source for rage and trolling. The future is scoreless reviews and robot dogs.

    • HexagonalBolts says:

      Jim, I very much agree, but the review score does sometimes have its place – if something receives a high review score I at least expect it to be finished to a high quality with few bugs, with a good balance between gameplay elements, a sizeable amount of content, and a consistent difficulty level – or otherwise possess some element of an outstanding quality to redeem a lack of polish. Although these things are inextricable from the gameplay experience as a whole, they are components that we can often get a feeling of.

      One prime example of this is Empire: Total War, it was a game with considerable bugs at almost every level of gameplay – fort battles were completely bugged, the enemy AI was incapable of loading units onto boats in a naval battle game and the battle AI was entirely incompetent. There was a lot of aspiration in there, but it was almost impossible with the vanilla version of Empire to have game that wasn’t deeply deeply frustrating, and it was irritating that the game still received 9 out of 10 in reviews almost everywhere.

      Do you know what I think would also be interesting? If metacritic worked out the average score that games got in reviews, and then rated games in relation to this average.

    • kyrieee says:

      Disagreeing with a score is not more strange than trying to score it on such a granular scale in the first place. What’s the difference between a game that gets 55% and one that gets 54%? Using a 1-100 scale to score games is like trying to gauge the size of something down to a micrometer using your eye. I think you run into that problem as soon as your scale goes beyond 1-5.

    • President Weasel says:

      Eurogamer tends to score low anyway, so all the internet hate and “Y U drag metacritic score down U BAD MANS” rants tend to rather miss the point: the Eurogamer scores, being generally lower than other sites’, will tend to drag all metacritic scores down a fraction, and the more popular the game is the more reviews there are that are likely to be put into the “pot” for the metacritic average and the smaller the effect the outlying Eurogamer at-least-making-a-nod-towards-using-the-whole-scale score actually has.

      I do worry about people that decide they love a game, and know what mark out of ten it objectively deserves without ever having played it. This actually worries me more than the fact that they can say something as subjective as a review score can be “wrong” in any meaningful way – at least “well I have played it extensively for two weeks and my personal opinion is that this is an 8 or 9 therefore I feel you have reviewed this game poorly” makes some kind of sense, on the surface.

      Review scores are subjective, and unless they’re given by a journalist who can be shown to have barely played the game and not grasped how it works they can’t really be “wrong” in any sense. However metacritic scores are a major yardstick for success or failure these days, and review scores from respected publications can affect other review scores and the tone of publicity for the game, and influence sales. It’s hard to blame publishers for being obsessed with them.

    • MichaelPalin says:

      @Richard Cobbett They use the same logic to 89% as they do with the prices. “Hey!, it’s not 70€, it’s 69.99€”. Next time you give a 89%, give it a 89.99%, that will teach them!

    • Ergates_Antius says:

      My question is: Is there actually any controversy here? A bunch of hysterical screeching zealots on an internet forum isn’t controversy, it’s just a bunch of gobshites proving how stupid they are. Who gives a shit what they think? They’re fuckwits, their opinions are worthless.

    • Vurogj says:

      Maybe their opinions are worth 89%, but you refuse to use a review marking system, you mean mean person you.

    • Daave says:

      I wish I got 89% on my degree.

    • Kdansky says:

      You should give it a 89.999… and see the flamewar erupt about whether that is or is not the same as 90, and whether that makes any difference, and if that’s the correct score to begin with.

      Spoiler: It’s equal to 90 if you know Math, and less than 90 if you are ignorant.

    • Beva says:

      I find the review system with scores lacking as well, I just don’t think that treating reviews like subjective opinions and then pasting a score out of a number at the end works. I want it to either be an opinion piece or a consumer targeted review.

      If you choose to go with the score at the end then we need to instill some sort of system that would at least try to have objectivity as a goal, in the same way that all (academic) critique has criteria that makes it valid.

      And while THAT Eurogamer review was well written, it was at the same time a bit annoying due to it’s inconsistency within itself (even if, to be fair, this isn’t really what most of the rage was pointed towards). By that I mean that the game was described as peerless at what it actually does, while most of the negativity had to do with what the game was not, and what it was not trying to do.

      This initself is fine, in an opinion piece or a WiT, but in a consumer oriented score-based review it forces me, the reader, to get to “know” the author of the review, so that it’s possible for me to make a judgment about whether I’m going to buy a game or not.

      What I’m trying to get at is: what if the answer to the question in the pic is niether “yes” nor “no”, but rather “I don’t know”. Reading reviews to confirm my love/hate for a game is pointless, I find.

    • ThTa says:

      Word. (That’s a thing that people still say, right?)

      But it’s not just videogames, either. Just this week, I got into an arguement about movie scores: “Well, I thought Real Steel deserves a 4/5 at the very least!” because he personally enjoyed the heck out of it and (thought he) saw certain underlying plot threads he was sure most reviewers had missed. To which I replied, “Glad you enjoyed it, now stop being so bothered about scores.” But he said that he simply wanted other people to share in his enjoyment, and that other people actually are bothered by scores, and might actually be aversed by them, unknowingly missing out on what he considered a gem. We both agreed that scores are silly, and should be abolished.

      And yesterday, I actually typed out a somewhat long comment in response to another, on a tech site, where the commenter argued that the site in question should “Bring back scores” because they’re easier or something. He claimed he’d go to other sites for reviews, because they did offer scores. Another commenter actually agreed with him, and said the site was omitting scores because low scores do not please their advertisers. (This actually caused one of the site’s editors to respond, and refute that claim)
      I obviously argued that scores are, in fact, silly, and that the site should be commended for not using them.

      I’ll post a link to the latter happening, if any of you’d like. Though my comment was eaten by the site’s ridiculous paging system. (I may be able to retrieve and repost it here, though me doing so is again, up to you)

    • Froibo says:

      Ah, this explains all the talk about the 1% lately.

    • Milky1985 says:

      “Eurogamer tends to score low anyway”

      No they don’t there average score (as in score for an average game) a year ago was 7.

      This is half of the issue, reviewers are SO afraid of annoying publishers that they give out high scores to everything that gives em cash .

      People seem to shout a lot “thats why we need a 5 star system”, i ask them how will this fix the issue in anythign other than the short term?

      All that will happen is that the “average” will move to 4 stars rather than 3 cause of publisher pressure.

      There are 2 solutions to this, 1 is that publishers should nto be allowed to blacklist peopel for saying bad things about them, maybe some sort of central place where review copies go to and then get disributed from.

      Secodnly people who playu the games should put up information abotu what they liked and disliked about the game, with no scores (user score on metacritic is a bad idea, user + and minus with reasons is a better idea).

      And if you liekd the game but other people didn’t (and thers no chance for reasonable debate) just deal with it and move on, there are more important things in life to argue about than if game X should get a 7.5 or a 7.9!

    • nootron says:

      Wot I Think is that WoT is a better review system than a review system.

      You tell me if you had fun playing it, if you wish you were playing it right now, what other games it reminded you of and that pretty much tells me whether I need to play it or not.

    • Reapy says:

      I think for me I pick a game based on, first in the preview if it is a setting and/or feature set that i am excited about. Then just 1 form of video review so you can hear broadly about the game, followed by watching a few videos of it in action to decide, and that is about it.

      WoT I think’s are great reading, and remind me exactly why I like playing games, but letting myself get steered into buying things on them has always puttered out, mostly due to the fact that my gaming landscape is different than the writers. I don’t have a set of friends to game with at hand due to everyone just being too busy and opposite of schedule, and the times I can get online are not till later at night.

      So RPS has gotten me solium inferium, bloodbowl, frozen synapes etc, and I’ve liked the games and appreciated how well they are made (well bloodbowl more like i had never played the old school game so was experiencing that). ANYWAY wot I think’s are fun reads and much more open honest opinions about games, AND if I find that I enjoy the same sorts of aspects they liked from the game, it can push me over into buying it.

      Really there is room and need for both styles out there imho.

    • theleif says:

      The Boiling Point reviews was what got me to read Eurogamer in the first place. And reading Eurogamer was what led me to RPS.
      Thank you Gillen.
      And damn that monster.

    • Klatu says:

      Long time lurker, first ever post yada yada; can any of the very fine journalists (paricularly those ex-PCGamer types) please enlighten me as why PCG persists in giving numerical scores at the end of reviews? The shitstorm which those numbers unleashed after Mr Cobbets very fine review of the Witcher 2 and the arguably slightly less fine review of Dragon Age 2 by Mr McCormick are two recent examples which might have been down-rated to shit-sqalls if the non-review reading members of the Internet nasty squad didn’t have a couple of digits to hang their rage on.
      I believe Rich and Richard would have had far fewer personal and professional insults directed towards them if archair reviewers had to actually read the review.

    • Juan Carlo says:

      I know it’s been discussed to death, but that McCormick review of DA2 was really, really, awful (so I actually think PC Gamer deserves to have it brought up to them at every turn, as it was a huge gaff on their part that they need to learn from). And I don’t mind at all that he liked the game and gave it a 94%. It’s fine if you like a game or dislike it. The problem was that, as is expected with most reviews, he really didn’t bother to point out any of the games glaring flaws. Even if you absolutely love a game you usually should at least make an attempt in the review to weigh a game’s positive aspects against its potential flaws. The fact that DA2 has so many massive, unmissable, glaring flaws (and I mean obvious ones that most people would recognize as flaws and which don’t depend on mere taste—like bugs, reusing assets, a tiny and lifeless game world, etc, etc) seems to suggest that McCormick was either too stupid to see them or he just doesn’t know how to write a proper review. But, even stranger, he also doesn’t even really give any evidence to support why he likes the game in the review. It’s just full of idiotic hyperbole and reads more like a PR fluff piece than a review. After reading it my first thoughts were: either PC Gamer has fallen from grace hard since their glory days f the 1990s, or this is a fluke and this McCormick fellow just has an uncle on the staff who got him the job, or something.

      I have no problem with the Witcher 2 review, though. I do think that one is more up to PCGamer’s standards. Even though I really, really, love WItcher 2 (I actually think it’s the best game released this year and is revolutionary in terms of video game narrative), if I were forced to write a review of it I would have to grudgingly acknowledge that it did have some pretty big flaws on release–and so I agreed with most of PC Gamer’s criticisms in that review even though I love the game inspite of them.

      As to scores in general, I agree 100% with people who say they are meaningless and agree with all the arguments against them. But, that said, I still like them. Yes, that may be awful and uncouth of me. Perhaps by liking them I am contributing to the dumbing down of culture. But even though I agree that they are incredibly flawed and imperfect, I still like them as an easy shorthand to judge a game’s quality at a glance. I’ll still read reviews, but I don’t think scores are meaningless. Yes, they are imperfect, but I do actually think that more often than not the metacritic score of a game will be a fairly good indicator of its quality (or, in the least, one helpful tool among many by which to judge quality before buying a product)–so scores can actually be a pretty good tool for consumers to use sometimes (provided, of course, that people approach them with some critical thinking skills and realize their imperfections and merits as they are).

      The only reason why scores can be so controversial is because readers of the review, I think, take them in the wrong way and put too much value in them. Which is their own faults, really.

    • drewski says:

      I think, for whatever it’s worth, that you can write a bad review without being a bad reviewer.

      I bet anyone who’s written more than about 25 game reviews has at least one they’d kind of like back.

    • atticus says:

      @Juan Carlo: Your comment, sir, is my point exactly . I therefore judge it brilliant!

    • lijenstina says:

      The percentage points as scores are an equivalent of an actor wearing a lab coat in a commercial.

    • HeavyStorm says:

      This has already become a very large post on a very large thread, but I couldn’t help writing…

      First off, I need to play devil’s advocate here. There’s one thing good about scores: it is its intention: giving readers a quick conclusion. I need to know if ten different games are good? I drop by Gamespot and check the score.

      But hell, how flawed it’s. As I’m pretty sure most people will/have pointed out, there’s no scale: your average might be 70% while mine could be 50% and I may have many outliers at 95% while you have your 3rd quartile at 95%… So, it’s doesn’t work.

      Secondly, I guess that people expect too much of a score… Will I like a 99% or even a 10/10 game? You sure? I mean, EVERYONE should, right? It’s the best of the best and… well, and I don’t like sport games. At all. So if you get a FIFA Soccer or maybe an NFL title that has a perfect score, still I wouldn’t like it. And it’s the same for every genre out there. More than that, even — there are themes, mechanics, and a lot more of stuff involved in liking a game or not that are linked to personal tastes.

      Third and final, a score should (IMO at least) measure how good a game is, and not how much the reviewer liked it, right? Game critics should try to have a bit of a neutral view over a title, otherwise we get DA2 incidents. If that’s right, then again, a large score means nothing to most people. I’m currently playing Dead Island online with my friends, and heck, it’s a poor game. Really, I’d give it a 5/10 and I’m being nice. But damn, isn’t it fuuuun? So, we, seasoned gamers, know that a score means nothing and that bad games are fun to play and good games may be boring for some of us.

      What, however, I miss, is something like a bottom-line (preferably on the top) so that I can quickly check out what the game is all about in a few seconds. Like, a summary of the reviewer experience.

      Oh yeah, and with all that said, producers, please, give us back Demos!!!.

      (oh, and, is that picture from Visio?)

    • Mattressi says:

      This article/comment thread got me interested in finding out the average Metacritic score. I tried a few things, but it was too damn difficult to come up with. Instead, I took a look at the list of scores for games released within the last 90 days (http://www.metacritic.com/browse/games/score/metascore/90day/all?view=detailed&page=3).

      I counted 19 games with a score below 50 (the lowest of which was 32). I counted the number of games on one page (there’s 4 pages – too much to count for me) and counted 424 game scores. So, a total of 4.5% of all game scores for games released in the last 90 days have a score below 50. The lowest score after two pages (212 scores) was 75. So, 50% of all the games have scores of 75 or above.

      My concept of “average” has just been completely destroyed.

    • drewski says:

      Your problem is assuming that there should be any kind of normal distribution across released games using the entirety of the percentage scale.

      Review scores tend to weigh heavily toward the top end of the percentage system because generally shit games – games that would trend under 50% – get canned before release or, if they make it out, don’t get reviewed because publishers drop them without marketing or distribution to review outlets.

      It is entirely normal to expect most games released to be well above 50% – publishers don’t release broken games, and most review outlets consider a 5 or equivalent to essentially be a pass or the bare minimum standard for a released game.

    • Mattressi says:

      Maybe I’ve been looking at it the wrong way – I’ve always assumed that when reviewers say 5 is “average” they literally mean “average” – saying that a game with a score of 5 is the average game on the market. Perhaps they mean the colloquial “average”; which somehow seems to mean “sub-par” to most people. That would explain why review scores tend to be between 7 and 10, rather than using the full scale.

      But I don’t see how this kind of scale could work. You can mark someone’s mathematical workings or possibly even an essay (if looking for key points or something…I don’t know, essays bore me) with a score based on merit rather than based on relative performance to others – but I don’t understand how you can ‘mark’ a game based on anything other than its performance relative to other games. Funness, interestingness and prettiness are pretty difficult to quantify, but it’s easy to compare two games in terms of how fun they are, etc.

      As for games reviews being normally distributed – assuming that “average” means a literal average based on the “goodness” of all the games reviewed by a publication, surely they’d be normally distributed – unless reviewers are including in their ‘average’ calculation all games that are never released due to being too crap, which would surely mean that all review scores should be high 90′s at least, given the amount of horrible games made by undergrad computer science students?

    • jrodman says:

      “An seven”? *twitch*

      I rather think you did that to me intentionally. *pout*

  2. MiniMatt says:

    Ah but that flow chart doesn’t cover the possibility of my opinion being infinitely superior to that of the reviewer’s.

    I must therefore shout from the rooftops how much more valid my opinion is UNTIL THE LITTLE PEOPLE LISTEN TO ME.

    And preferably bow down too.

    • aronbarco says:

      As a philosopher, I feel obliged to say that opinion and critics should not collapse into the same meaning.

      We have a large category of [art, games, etc.] critics that don’t know the basics of their historical and hermeneutical responsabilities, an therefore they produce fragile reviews such that any joe feel that their opinion is equivalent to it.

      On the matter I suggest the reading of Gadamer’s Truth and Method.

    • MiniMatt says:

      Whilst I detect a distinct lack of bowing in my presence your response has given me cause to look up the definition of “hermeneutics” and as such you shall be spared my vengeance come the revolution.

    • Vagrant says:

      But then he immediately misspelled something! Off with his head!

  3. N says:

    True dat’ yo.

  4. Phoshi says:

    RPS I don’t think you understand

    somebody disagrees with me on the internet!

  5. CMaster says:

    Reads like a 4

    • LionsPhil says:

      Yeah, it doesn’t make sufficient allowance of the role of reviews as not just “here is my opinion” blogging, but also buyer advice. I care if a review is grossly misleading (or simply not informative) about how much fun I can expect for my money.

      I’d say it’s a 4.1, though, and will fight you until the end of time and space over that 0.1.

  6. greg_ritter says:

    But how can I know, if the game I’ve played is any good? What, decide myself? That’s borderline communism!

    • Maldomel says:

      Imagine, not only do we, customers, have to buy games, but we have to play them with no clues whatsoever. Oh, Inernet, why are you so cruel, do you enjoy tormenting us poor brainwashed netizens?!

    • LionsPhil says:

      If only it were possible to try portions of games in advance, for free.

      Sadly that is not as universal as it once was, especially for titles which are full-price “AAAA” and thus represent the biggest risk to buy blind. (Also, pre-ordering completely scuppers it, and pre-order exclusives and discounts try to tempt you to do so.)

  7. Kandon Arc says:

    But the internet must validate my purchases!

  8. Richard Cobbett says:

    WON’T SOMEONE PLEASE THINK OF THE METACRITIC SCORE THAT HAS BEEN STUPIDLY CO-OPTED INTO A MARK OF QUALITY THAT MUST BE BOOSTED AT ALL COSTS INSTEAD OF BEING TREATED AS A SIMPLE SNAPSHOT THAT DISPLAYS A WIDE RANGE OF OPINIONS AND WILL INEVITABLY HAVE OUTLIERS ON BOTH SIDES BECAUSE THAT IS HOW CRITICISM WORKS?!

    • Baka says:

      That was too long and complicated for me. Could you please trim it down to a number?

    • Richard Cobbett says:

      3.141592653589793238462643383279. Easy as pi.

    • Teddy Leach says:

      METACRITIC IS NAFF, INNIT.

    • phlebas says:

      Metacritic is useful – you can click through to the reviews, read the outliers and trusted sources to see why people did or didn’t like it. Only looking at the average score is (admittedly encouraged) misuse.

    • LionsPhil says:

      Metacritic needs to show the standard deviation as well as the mean.

    • InternetBatman says:

      @Lionsphil

      That is one of the best ideas I’ve read in any thread about scores.

  9. Ian says:

    Has there been a new notable review score controversy that I’m blissfully ignorant of, then?

    • greg_ritter says:

      Uncharted 3.

    • Vexing Vision says:

      Yeah. Google for Tom Chick and Uncharted 3.

    • The Tupper says:

      I don’t like going elsewhere for my gaming news. Those places scare me.

    • Fede says:

      That was fun, but also a little sad.

      Especially because they should know by now that Tom Chick’s opinion sometimes is very different from the mainstream (which is good).

    • MrGreen72 says:

      I know that Joystiq’s 8 on Uncharted 3 was responsible for several aneurysms on NeoGAF.

    • Khemm says:

      And now that many of those NeoGAF zealots have actually played the game, they complain about a lot of things (dumb story, input lag, troublesome aiming, ripping U2 off in many places) – suddenly, Eurogamer’s 8/10 sounds like a fair score to them.
      Yet before release it was like: NOT A 10/10???!!! Best game eveeer, how daaare they? Game of the decade yada yada…

    • dreadguacamole says:

      Not to mention the AV club, which gave it a 50
      (Well, actually, they gave it a C, which apparently should translate to 75%)
      There were something like 600 comments within the first few hours, mostly from people who registered only to complain… so much bile.

    • Oozo says:

      At least the ratings by Tom Chick and the AV Club were lowish. Simon Parkin gave the game 8 out of 10 and was, to say it politely, treated rudely by the commenters. At a time when most of said commenters couldn’t possibly have played the game. (Granted, his text had an angle that could be perceived as problematic, but it was not exactly a reasonable discussion of this fact…)

    • iucounu says:

      The comments on that article do give the lie to the idea that making people use their real names enhances civility. Thinking of one *amazingly racist comment* in particular.

    • V. Profane says:

      Wups, didn’t notice the AV review was already linked to.

      It’s 1381 comments now. The next highest for a game is 443 for a positive review of Arkham City.

    • slM_agnvox says:

      I remember reading Tom Chick in Computer Games Magazine when I was a kiddo, say ’96-2000. Was a great mag, though I’ve since looked it up and think I may have been one of a minority of readers. Turned me on to a lot of great games though, like Kohan and IL-2. Tom Chick had the back page humour article too, was consistently pretty funny.

      Sounded like a good review to me, I know I’d be frustrated by all the things he was frustrated by were I ever to play this game. The comments were sad, couldn’t read more than a dozen.

      RPS, still looking for another writer? Pick Tom Chick!

    • Highstorm says:

      Been playing Uncharted 3 the last few nights, and he’s pretty much spot on in that review – though personally I don’t even enjoy the gunplay (but then, it’s on a gamepad). I’ve found myself slogging through the jumpy bits and the hordes and hordes of dead-eye, grenade-spewing, made-of-steel enemies to get to the cutscenes as the characters are fun to watch. But then it turns out there’s nothing of any real interest there. It’s the same tale told twice before (and better) with a thin coat of paint splashed on top.

      Then I realized I just spent $60 to watch a mediocre movie and got a little depressed.

      But yeah, arbitrary review scores, wow! U3 got a 10 on IGN (the review has 7,799 comments at time of writing) which I rather expected and find to be wholly undeserved. This is why I love RPS and their WiT – it’s like getting a recommendation from a friend you trust rather than some guy in a cheap shirt, trying to sell you snake oil.

    • Jason Moyer says:

      I feel uncomfortable seeing Tom Chick’s name without “of Gamesdomain Deus Ex review fame” behind it.

      If he gave the greatest game of all time a 5/10 surely a 4/10 is pretty good, right?

    • Vinraith says:

      Tom’s an odd reviewer, in that he is willing and able to give you an honest opinion regardless of the hype, the fans, or any other damn thing. That’s genuinely hard to do, without dropping to the level of being a contrarian. He also uses the whole 1 to 10 scale, instead of the 7-10 model.

      Tom Chick, Slayer of Sacred Cows would be a good general title for the guy. The amount of rage you still see about his Deus Ex review is proof enough of that. It is, after all, objectively the best game ever made, and anyone that disagrees (and especially anyone heretical enough to actually point out its glaring flaws) must be burned in effigy for all time. Hilarious.

      I don’t always agree with him, but I respect anyone that can piss off that many people for that long just by stating an honest opinion.

    • Jenks says:

      He’s not a “slayer of sacred cows,” he’s a guy playing the metacritc metagame. There are a lot of people who don’t understand this and wind up unintentionally trolled.

      Disclaimer: I don’t own a PS3 and I’ve never played an Uncharted game.

    • Srethron says:

      @slM_agnvox: I subscribed to PC Gamer US, Computer Gaming World, and Computer Games back then. They all had different good and bad points, but Computer Games was the most reasonably adult and grew to be by far my favorite. Steve Bauman, Tom Chick, Jason Cross, Cindy Yans… those were the days.

  10. TheFlyingWooly says:

    Don’t make me read words, just give me a number!

  11. Vexing Vision says:

    There’s three kinds of people. Opinion-makers, opinion-followers and the loney hermits.

    If I see no scores, how can I validate my opinion? How do I know if I am right? How do I know I enjoyed the game as much as I was supposed to enjoy it? Maybe I was missing out? Maybe I enjoyed it too much?

    This chart leaves me with existential game-enjoyment angst. :(

  12. McDan says:

    Jim’s right though, scores are pointless. They’re still just opinions like the rest of the review. At least RPS is honest about saying it is just what they think or their impressions of a game. They’re just numbers! Arghhhh!

  13. Jockie says:

    Eurogamers comments sections have been ruined entirely by ths phenomenon, with pithy “better than x game then?” comments after every single review, then repeated and rehashed ad-nauseum.

    It’s partially down to Metacritic, because as much as we like to pretend review scores don’t matter, they obviously do to the money-men and to quite a few gamers.

    It’s weird that games criticism seems to have so much clout (probably to do with the amount of web-trawling gamers do), because pretty much every hollywood blockbuster film gets middling reviews (Pirates of the carribean series is a good eg), but still go on to make many millions more than those that are critically accaimed. Whereas in the gaming industry, a game that scores less than 80 is seen as a flop.

    • Archonsod says:

      The irony is it doesn’t have that much clout. If every publication gave Battlefield 3% it’d still sell by the bucket load. On the other hand, give an obscure indie title 99% and you might convince an extra six people to buy it.

    • wicko says:

      To be fair, the entire purpose of review scores is for comparison, so those commentors aren’t entirely in the wrong.

      But really, poor scores for Uncharted 3 just means they didn’t bribe anyone.

  14. mangrove says:

    There’s only one true scoring system: thumbs up or thumbs down.

    You don’t even need a review attached, maybe just a loud dominant grunt followed by the gesture.

    • Khemm says:

      Something in between would be nice:
      -thumb up (great)
      -thumb diagonally up (good)
      -thumb horizontally (average)
      -thumb diagonally down (not recommended)
      -thumb down (avoid)

      I think something like that would be really nice, certainly better than meaningless 7.5/10 or 8/10.

    • LarsBR says:

      Thumbs up and thumbs down would just be metacriticed to 0% and 100% or something equally silly.

      Khemm’s is basically the letter-grade system, which has been metacriticed as well: http://www.metacritic.com/about-metascores

    • Carra says:

      Yeah, it basically comes down to: do I advice you to play the game or not?

    • iucounu says:

      The ZZAP!64 scoring system was interesting in that it combined a 0-100% scale with little pencil drawings of the reviewers pulling faces or giving the thumbs-up/down. Compared to games reviews these days, it’s quite odd; the article body copy is completely devoid of editorialising, and the actual criticism is restricted to mini-columns from each reviewer.

      I often find when I recommend games to friends it’s often in terms of how much it’d be worth paying for it. Perhaps that’s the answer to the scoring conundrum: This is a £15 game, that’s a £30 game. Scaled to the disposable income of the person reading the review. (Not really.)

    • Novotny says:

      9 million upvotes for mentioning ZZap!

    • terry says:

      Your Spectrum had this exact system – HIT for “I liked this game” or MISS for “I didn’t like this game”. Of course that was in the cottage industry days of gaming and it was later changed to something far more complicated and misleading.

    • Hodge says:

      Zzap! was great, but in hindsight it’s also partly responsible for the score-obsessed mire in which we find ourselves.

      GRAPHICS 90% SOUND 97% PLAYABILITY 89%. LASTABILITY 91%. VALUE FOR MONEY 95% OVERALL 94%.

      All of which was quickly adopted by rival mags once Zzap! started selling big numbers. It was a well-intentioned response to dismissive, two-sentence reviews that games got in ‘serious’ mags, but well over the top.

      They also handed out awards to games that received 90% or more which led to the exact “WHY DID GAME X ONLY GET 89% OMG IT’S A CONSPIRACY” type whining which Richard talks about above.

      Still, the three-comment system was indeed fantastic, and theirs were the only reviews I even remotely trusted.

  15. MonkeyMonster says:

    42?

  16. Skeletor68 says:

    Pleh to review scores! They seems like a cheap tactic to stir up conversation and controversy a lot of the time anyways. Personally, the review text is an awful lot more helpful in deciding. For example, I’m pretty sure something like Mount and Blade would get low scores on graphics etc. and bring down the overall mark but that is completely missing the point of the magic of the experience.

    IGN’s ‘Dark Souls v Skyrim’ article really made me laugh and seems to be along the same lines of applying sensational headlines to non-isses. Two completely different games, but good for getting your community to blast each other into pieces with capitals and exclamation marks.

  17. kraken says:

    Well I can see the issue after buying a 50€ game based on positive reviews and then discovering it’s terrible.
    A few years back every game had a demo so you could test it out yourself, but now it has become a rarity. So how are you gonna decide which game is worth it?

    For example, Rage had very good reviews, despite the release bugs debacle, the contestable textures quality and limited gameplay. And I’m pretty sure quite a few release buyers got it with confidence of the reviews and were utterly disappointed.

    • Archonsod says:

      By making my own mind up rather than relying on reviews primarily.

    • Apples says:

      How are you going to make your own mind up without buying the game first? The only thing there is to go on is reviews before you buy/see/read something, surely. Unless you go down the ‘pirate first, buy later’ route.

    • jRides says:

      If you know who you trust when reading a review then maybe its worth it, as your comparing to that reviewers previous. PCGamer US reviewers are complete idiots who score based on hype and marketing whereas I pay more attention to reviews written by the fellows at PCG UK.

      I think a reviewing scheme based on biscuits would work best. Review score: “Custard Creams” Then I’d know where I stand as Custard Creams are disgusting and I could berate those who strangely like custard creams as being obviously wrong. Everybodies happy and my exclamation point key gets a good run out.

    • faircall says:

      Indeed, the days of the demo allowed us to make up our own minds. Question is, did that mean better or worse sales, I wonder.

      I reckon these 6-8/10 games that people apparently refuse to buy would really benefit from proper demos, because games in that range are usually flawed but very fun to play…(Evil Genius, Vampire Bloodlines, Wheel of Time, Call of Cthulhu all spring to mind)

      Although, said games often end up on Steam sales, so I suppose it kind of works in our favour in the long run…

  18. Teddy Leach says:

    And that is why How To Play don’t give scores, which means you actually have to (gasp!) read the words. It also means the more simple-minded readers won’t be able to make up their minds as to whether or not they want to get outraged.

    • Vexing Vision says:

      It also means that, unlike IGN, Gamespot, Eurogamer or Metacritic, you have no traffic.

      Why, yes, I DO specialize in mean. :P

    • VelvetFistIronGlove says:

      This is also why Rock, Paper, Shotgun has no traffic.

  19. jezcentral says:

    We live in a world that has people who couldn’t/still can’t take Daniel Craig as Bond, because the colour of his hair was a slightly different shade of brown.

    To these people, something like a review score must be a central pillar of their existence.*

    *I was a teenager fighting to understand the world too, once, so I can sympathise to some extent.

  20. Aquarion says:

    Which is nice, but unless it starts out as “free to play” unless I want to get past a certain point or level (or “Demo” as it used to be known), I rely on games reviews to tell me what the game is actually like (Although it doesn’t take into account things like “Will play like a sick dog stuck in a slow motion universe on your PC, Aquarion. Stay away” as well as a demo would).

    I mean, are you actually suggesting I read the entire content of a review and form my own opinion based on the reviewers thoughts and experiences without some kind of numeric evaluation? Gosh.

  21. Maldomel says:

    I think the REAL problem lies in how people aren’t reading reviews at all when there is a score in it.

    Also, I give this flowchart a 8/10.

  22. Drake Sigar says:

    I think we’d all be a lot happier if scores were completely abolished and people were forced to look at what the reviewer actually said.

    • BAshment says:

      but all those letter combinations tire me out.

    • Michael Hoss says:

      You know, that’s the problem. A market gets what it deserves. Magazines can’t go away from numbers so fast (even if most of them want to – true story!). The readers would fuck them off. Most gamers (exclude RPS readers here please) are stupid as hell and mastrubate on numbers.

  23. bear912 says:

    I remember reading an article recently that made the claim the fact that Metacritic scores matter to (parts of) the game industry breaks Metacritic…

    I thought it was a rather apt observation.

  24. Augmentation says:

    The problem with this flowchart is that most of the times people reading the review of a game can’t know if they enjoy it or not purely because the game hasn’t been released yet. So it’s down to sheer expectations when people get uppity about the scores, especially in Uncharted 3′s case on Eurogamer.

    Personally it seems that the sites that don’t have review scores tend to have the best discussions surrounding the reviews. A peeve that I have about scores is that they’re limited to the seven to ten range in most cases so a lot of the discussion of the review revolves around it not matching the aggregated scores despite a five on one site being equivalent to a seven on a so-called seven to ten site. (And don’t get me started on the equivalence of scores across sites…)

    • Groove says:

      That level of problem blows review scores out of the water.

      People getting angry about someone insulting a game they haven’t played yet…..wow. Stepping back to think about that, it’s pretty terrifying, the power of advertising.

      It’s like a blind person defending red as the best colour. Reviewing colours might be a foolish endevour, but the blind man is still the craziest person in the room.

    • ninjadeath says:

      I know what you mean, on the PC Gamer review of BF3 yesterday somebody claimed that it shouldn’t have got 80% when MW3 (am I allowed to mention that game here) is a far more polished product.. I asked him when he had played it due to the fact it wasn’t released for another week and I am still waiting for his response.

    • Milky1985 says:

      “So it’s down to sheer expectations when people get uppity about the scores, especially in Uncharted 3′s case on Eurogamer.”

      I think it was worse on eurogamer because that place’s comment threads are full of sony fanboys and apologists. I had to cancel my account due to the hate directed at me for trying to have debates about sonys treatment of peopel during the whole geohotz incident (some guy posted details i thought were private but were not like my emial address, then refused to remove them when asked, adn the moderators did nothing, this was ironically about an argument about the ps3 root key and how i didn;t see a problem with it being out there and he did)

      Uncharted 3 bveing a ps3 poster child getting anythign less than a 10 was always going to get hate at that site becuse of the commentors there.

      Just don;t say that they are the sony defence force, they will deny it all the time they are beating you with there ps3s!

  25. Quine says:

    I think the old Optimus Prime thumbs were the way forward.

    Get a bunch of reviewers with differing levels of enthusiasm for a game together to discuss what worked or didn’t work for them, with some good anecdotes and digressions and a generalised ‘would I buy it’ roundup at the end. Then there’s something for every fanboy and hater to get behind.

  26. JackShandy says:

    You’re poking holes in the very idea of commenting on the internet!

  27. Dominic White says:

    I’m so glad that the site I write for doesn’t make me slap an arbitrary numerical rating on my reviews, because it’s almost never that clean-cut. There’s always elements that will put some people off more than others, some things that are key to some folks but a complete non-issue to me, and so on.

    Read the damn review itself. Takes a little longer, but it’s much more worth the effort.

  28. salad10203 says:

    I prefer a wider variety of scores. For instance, I have no problem with BF3 getting low scores because they are pointing out its flaws. Meanwhile, the 10′s being given to Uncharted 3 just sound ridiculous, there is no way that game is perfect. That being said, your graph is a little off, people go to reviews a lot of the time when they have not played the game. Only trolls and flamers argue over review scores.

    • N'Al says:

      A 10 out of 10 doesn’t imply that the game is perfect, though (at least not on most review sites’ rating scales), merely that the game is the best of its type currently in the market – according to the reviewer, obviously.

  29. Cinnamon says:

    It’s a funny old business really. Why should we care about anything, all of life is ultimately deeply meaningless and futile. But if some people are demented enough to care about game reviews then at least it provides work for people who write them.

  30. coldvvvave says:

    edit

  31. mrdan says:

    Seems like sites bring this upon themselves. Comments are there (presumably) to discuss the review and for people to give their opinion even if it’s contrary to the reviewer’s or the majority’s. Review scores, if given, are part of the review. So if you don’t want people commenting on them either stop giving scores or disable comments. Otherwise you’re basically complaining about people using the system for what it’s designed for, just not in the way you like.

    Personally, I never read comments on reviews because I’m simply not interested in them no matter the score. I don’t read meteoritic, just different reviews from a few sites I’ve used before and can gauge how much weight to give their opinions.

  32. Binary77 says:

    I’ll take an “If you liked…” over a score anyday of the week. At least that gives me an impression which i can relate to from my own experiences.

    • Radiant says:

      I know right?
      Or even a ‘if you liked X, Y or Z you’ll like A’ and then review the game against it’s peers.
      Do a ‘Two games one slot!’ with mw3 and bf3 and watch the controversy cascade into your comments section.

    • Groove says:

      I prefer to review against apples. If it isn’t better than an apple then I don’t want to play it.

    • Radiant says:

      Yeah but being judged by twelve of your apples when you shoot someone could be kind of one sided.

  33. Roshin says:

    Because a 10 means the game is perfect and flawless and loved by everyone in the whole world.

  34. Tams80 says:

    So what are people’s thoughts on the ‘Puppy – Unicorn’ scale? I feel it has been grossly misused by RPS.

  35. Screamer says:

    An 8? I don’t even…..

  36. HermitUK says:

    It’s all in the numbers.

  37. Juuuhan says:

    I really wish they could stop with the score, can someone seriously state that they enjoy every game with a high rating and hate every game with a low rating? No, it’s personal preference, and people pay too much attention whether a game have a high score to motivate their buying decision and almost none on the actual review.

    Rather than scores can’t they focus more on “If you liked those games you will also enjoy this game”

  38. Saiko Kila says:

    I don’t understand why Steam lists the metacritic score at all. Why is that? If any score is necessary (and apparently it is, because they list it) it should be users score, like Amazon, ex-Impulse/GameStop or GamersGate, I think. It’s harder to buy users than to buy score-giving reviewers, for starters. And if I care about opinions of professional reviewers, I really should read the review and not the score. And that’s why I read RPS.

    I suppose only impulse buyers need that score. Oh, wait, I may have answered my own question…

    • V. Profane says:

      I’ve always found that bizarre. Fair enough if it’s a high figure, but you’re trying to sell me Silent Hill: Homecoming, do you really want to advertise the fact it has a 64?

    • iucounu says:

      The way it works for me is, I start out on the Steam product page, usually wondering whether to buy something in a sale. I tend to then click through to Metacritic, because I’m less interested in the raw score than I am in reading the reviews themselves. It’s interesting to be able to triangulate what my likely reaction is going to be out of the outliers. I think I’d do that even if there wasn’t a handy link in Steam, so it just makes my whole decision process more convenient.

      I’d wager that most people who are looking at Metacritic scores at all are going at least a little deeper, and using a game’s page more as an index of criticism than as a place to find some sort of definitive numerical verdict. I could be wrong though.

  39. Stevostin says:

    If I like something, I want to promote it to encourage the industry to do it more. If I don’t, the opposite. That’s basically what every RPS journalist does, and wants to deny to their readers, apparently ;-)

    • adonf says:

      I think the problem here is not with comments in general. The comments themselves are a valuable addition to the review, unless they focus only on the score. Well I don’t read other gaming sites so I may be wrong, but that’s what I understood.

  40. Michael Hoss says:

    I remember myself giving Operation Flashpoint: Dragon Rising a 6.9 out of 10. The first comments blamed of giving a way to low score. later on the comments said I rated the game to fucking high. Funny thing is the commenting people never read the already available comments so they couldn’t see that there COULD be some kind of other opinion.

  41. tinners says:

    Unfortunately if you make the mistake of scrolling down past the end of a eurogamer review you are just asking to get enraged. Why do we do this to ourselves?

  42. Wooly Wugga Wugga says:

    I personally disagree with scoring a review. Much of what makes a game good is objective and how do you accurately convert objective to subjective?

    One thing that does get my hackles raised is when a reviewer clearly gets his facts wrong in the review or tries to pull an Old Man Murray style edgy review. (*cough* Fallout New Vegas : Wot I Think *cough*)

    • Metonymy says:

      There’s something fascinating about the way people still argue this subject. Fallout 3 was a substantially better game, NV was a substantially better RPG, this has been argued thousands upon thousands of hours across who knows how many forums and boards, and that specific conclusion is always reached after people stop acting like fools.

      That original WIT was complete accurate, even if he failed to stress that NV had better rpg elements.

  43. Metonymy says:

    To answer the question at hand, inflated scores lower the perceived value of truly good games. How can I possibly know that “Bastion” (or whatever) is a good game, when hundreds of terrible games get the same score?

    It seems that reviews struggle to write more interesting pieces, but in truth, they need to be less interesting and more specific.

    IE, I want to know what SPECIFIC games are similar to the way the (game being reviewed) handles:
    -control
    -gameplay
    -progression
    -story
    -multiplayer
    -anything else

    For example, the review for Borderlands should say:
    -It controls like Call of Duty
    -It has item drops like Diablo, but with less effect variety from the items
    -It has skill trees like WoW, but only a few talents affect the way you play
    -you can only level up from similar-level enemies, like WoW
    -It has a linear story in a setting similar to Fallout3/NV
    -It has CT’s new game+, but with almost no variation from the first play

    See, that’s an informative review, that instantly tells me whether it’s a game I want to play.

    • hoobajoo says:

      Doing a review in the style of “it’s like game X but with elements of game Y” is the most boring, intellectually bankrupt way of doing a review. It’s just as bad as assigning a score to every element (GAMEPLAY SOUND PRESENTATION etc) and then just adding/averaging those scores together for a bigger score. As if games are things like automobiles or appliances, to be evaluated objectively and pragmatically.

      Each game is a whole, and needs to be evaluated only as a whole; you can have a game where each constituent element is solid, and the game itself is mediocre, because a game is NOT the sum of its parts. Any review that breaks a game down to atomic elements is inherently fallacious, and doing it solely by comparing it to other games is adding boring to its litany of sins.

    • Metonymy says:

      One method doesn’t exclude the other.

      You’re suggesting that a game can have excellent features even if some areas fail, and you’re also suggesting that it can have a holistic quality that is greater that the individual components suggest. If this rare situation exists, and it obviously sometimes does, all the writer has to do is say “BUT…”

  44. Jhoosier says:

    The only way I would accept a numbered score would be if there were a proper rubrik to sort it out. Like game-crippling bugs gets you -2 points. Multiplayer not working? -1 point. Is single-player story non-existent? -1 point. Does the voice acting make you want to stab out your eyeballs? -5 points. Open-world sandbox with Burt Reynolds and dildo bats? +10. And so on. Perhaps someone could work out rubriks that pertain to different genres. A single-player platformer doesn’t need photorealistic facial animation, nor would it rely on having a decent game matchmaker. So points could go towards something else that platform players care about. Anyone else?

  45. Sander Bos says:

    So this diagram represents… what? Is it an argument against comment sections on gaming websites, or against reactions to web articles in general? Because it seems no matter how you go through the flow diagram, you are not supposed to give ‘a remark, observation, or criticism’ (definition of comment) on whatever you just read.

    I for one, like to *read* reactions to what others have written. For instance, with posts on RPS on a game I should check out, I will always check the comments first to get some second opinions whether it is really worth my time.

    Probably my cynical negative nature, but I don’t understand how a ‘hooray for everything’ commenting/ reaction philosophy (or in the case of Alec’s flowchart, a ‘silence is golden’ commenting/ reaction philosophy) contributes to a better more informed world.

    • VelvetFistIronGlove says:

      It’s an argument against arguments about review scores.

      Discussing the game is fine. Bring your own thoughts, hopes, fears, likes, dislikes, about the game to the comments.

      What’s a stupid waste of everyone’s lives is arguments over review scores.

    • Sander Bos says:

      And how about meta-arguments about review scores, like this post, do they help?
      Seems to me like you can make the same chart with ‘Do you care about review score discussions?’ as the top question…

    • VelvetFistIronGlove says:

      Yeah, I think this post is a little bit pointless, since the arguments are not actually taking place on RPS. It’s preaching to the choir. But I’m willing to forgive it, since I got a chuckle from the diagram.

  46. Dominic White says:

    Another impressively stupid review controversy is over on countless Edge Magazine comment threads – a lot of people are angry that the mag gave Bayonetta a 10/10 score. Amusingly nobody seems to be able to think of a single game in its particular subgenre that is better, but somehow that 10 is just undeserved!

    A lot of the problem seems to come down to people being stupid and not being able to figure out what a score means – it’s not the universal appeal of the product. It’s how much the reviewer liked the game, and how well it stacks up against other recent releases in the genre.

    • Hindenburg says:

      Ninja Gaiden Black, at least as far as the “game” bit is concerned. `netta beats the crap outta it when it comes to aesthetics, obviously.

  47. aircool says:

    I’ve always hated scoring.

    That’s it… nothing more to say.

  48. Lambchops says:

    I rate this diagram 3.4 quibbles out of -1 billion flumallos.

  49. Nemon says:

    [color body peach of importance and respect]

    crap, didn’t work…

    I love how RockPaperetc doesn’t include scores like that, it keeps all us internet angry men focused on the text instead of a number.

    [/color body peach of importance and respect]