Page 1 of 10 123 ... LastLast
Results 1 to 20 of 196
  1. #1
    Network Hub Jakkar's Avatar
    Join Date
    Mar 2013
    Location
    Swansea
    Posts
    319

    Professional versus User Reviews

    Fallout 4's Metacritic reviews by major publications,

    Metacritic's user-reviews section for same.

    Again, the great gap. Its width will vary over time, pushed back and forth in the schoolyard of public opinion by the many winds. We'll see the small-time pro-reviewers who received no advance code weigh in - cynical, unrushed, not living on amphetamines and skimming surface texture in order to make a pre-release publish. We'll see a few 're-reviews', quasi-retrospectives from the bigger sites that enjoy keeping their communities on their toes, fanning the flames of scandal just a touch.

    The users, meanwhile, will mob furiously in one direction then another - we'll no-doubt see the younger generation who see the term 'fan-boy' as something other than an ad-hominem attack rally to the game's defense despite having not yet received adequate pocket money to invest. We'll see a jaded mass attach themselves to the prevailing wind of a given season and swamp the review aggregators with vitriol bearing little resemblance to their own experience with the product - or lack thereof.

    Nonetheless, if there's anything to be drawn from public opinion on videogame releases, this moment in a game's shifting reception is as valid as any.

    What I query may appear to have an obvious answer, but I'd like to do so from an innocent perspective and dissect it, with the aid of anyone interested in weighing in, with an open mind and as little cynicism as I can manage.

    Why the gulf between public and professional opinion, in general? Is it passť to mention paid review scandals, or childishly sour of me to entertain the notion that such a broad swathe of those given pre-release code are barred from giving a negative review? Some of you will, I'm sure, think I'm silly to even question it. "Of course it's all bloody corrupt! Welcome to Earth!", but I'm equally sure others will disagree, and I'd like to hear from both camps and any I've not mentioned.

    Secondly, I wonder... with RPS as my primary source of gaming news, opinion and community and in part because RPS themselves have weighed in on the highly positive side of opinion on this project... and many other major releases of recent years - something I have again found unexpected, despite it being some years since RPS truly upheld their reputation for calling out such behaviour and erring toward the more analytical, cynical, irreverant and hard-to-please in their opinion-pieces.

    There was a time when no game could easily please the RPS team without being something - for example - massively technically flawed but overflowing with soulful energy, a clever message, or a zealous sense of fun that bucked a predicted trend and pleased the 'IThinker' despite itself. Or, indeed, simply a truly excellent game by multiple measures, upon that rare occasion. The overhyped were called out, the misleadingly advertised needled with a sharp point, and the merely mediocre were heatedly sandblasted by John Walker's infamous Withering-Scornomatic 850.

    Some years have passed since I had that feeling, however - with glowingly positive reviews of a 'You might be expecting me to hate this, BUT, I don't, nyeh nyeh!' sort followed by a lengthy series of 'I am not going to mention all the bad things about this' statements of a curiously defensive nature while complimenting traits that had been criticised in similar form often by the same writer in the recent past.

    Meanwhile this curious shift in attitude toward mainstream 'triple A' releases, and disproportionate leaning toward positivity in general appears specific to 'Wot I Think' and 'Impressions' pieces, while the analytical cynicism remains quite a strong theme in investigative pieces and retrospectives on products and companies that have fallen from favour and financial/legal influence. Further, that RPS remain quite critical of less well-funded independent products, and those released by troubled and non-Western publishers, although an overall shift to broad positivity remains a very visible trend, with harsher opinions becoming increasingly rare, and lukewarm to overwhelmingly positive feelings about a broad range of games becoming so common that I've found myself less inclined to place any weight upon what I read here.

    It's a cruel thing to ponder, but I will at least wonder openly if I'm the only fan of RPS and long-term community member who finds themself wondering about this. Is it just me, being a paranoid grump, or have you felt RPS appears to be diluting their opinions over time, and failing to critically analyse new releases, particularly mainstream releases, with their old rigour and realism?

    And that perhaps this is an inevitability as the founders have phased out, moved on, or grown quiet, while new writers have arrived alongside the supporter program. And, of course, while the site's front-page coverage has shifted heavily toward e-sports and MMO coverage, Streaming culture and hardware/software and law controveries... and other Serious Matters of a more lucrative age of videogames.

    Hmm.

    /thinkyramblequeries

    Requisite TL;DR and summary; Why, obvious as it may seem, are we seeing exclusively positive reviews of demonstrably flawed mainstream releases, and is RPS following the same age-old pattern? We are each entitled to 'wot we think', yet does RPS seem to be thinking in a rather predictable way, more and more as years go by? Why, if so, and can an independent review and opinion source truly survive anymore, or will there always be a lifespan before things change?

    Thanks for reading. I look forward to your thoughts.

  2. #2
    Secondary Hivemind Nexus Jesus_Phish's Avatar
    Join Date
    Aug 2012
    Location
    Dublin, Ireland
    Posts
    4,817
    Right, but don't use metacritic ever.

    Sample user reviews that are affecting the total user score which sits at 6.0 right now.

    This guy gave it a 2
    In Fallout 5 the SPECIAL system will be simplified by having only one number, going from 1 to 10, that tells how cool is your character. Perks are completely removed.

    In Fallout 6 you only have one dialogue option. You can press X to initiate dialogue whenever you want. Game still get 10/10 reviews and it is chosen as RPG of the year.

    Fallout 7 have complete Facebook integration.
    This guy gave it a 1
    It's not a fallout game. simply, it's another shooter. I hope sometime Bethesda would sell this franchise to a company like Obsidian(they are more interested in creating rpg games).
    These folks gave it a 0
    Please Bethesda, let Obsidian make another Fallout game. This is just pathetic. Everything is poor in this game; graphics, writing, story, even the combat.
    This game doesn't work on my typewriter with 32-bit windows. 0 points from me. I like my little pony: friedship is war. That is a game we should all play.
    Review scores are stupid and in this day and age we've enough resources available that you don't need a review to inform your decisions.
    "Halo is designed to make the player think "I look like that, I am macho sitting in my undies with my xbox""

    Steam ID

  3. #3
    Secondary Hivemind Nexus TheDreamlord's Avatar
    Join Date
    Dec 2011
    Location
    London
    Posts
    1,948
    Quote Originally Posted by Jesus_Phish View Post
    Review scores are stupid and in this day and age we've enough resources available that you don't need a review to inform your decisions.
    Totally agree. Plenty of information available nowadays to easily help you form your own opinion. You (proverbial you) have a brain. Use it.

  4. #4
    Secondary Hivemind Nexus
    Join Date
    Feb 2012
    Location
    Stockton-on-Tees, UK
    Posts
    2,854
    Why are you surprised that RPS, a website that has a long history of really liking Bethesda games, should be positive about a Bethesda game that is very clearly in the distinct Bethesda style?


    I will continue to defend review scores, user reviews, and metacritic as useful tools. They provide helpful information quickly.
    Irrelevant on further examination of the rest of the thread.

  5. #5
    Secondary Hivemind Nexus neema_t's Avatar
    Join Date
    Nov 2011
    Posts
    1,759
    Review scores on their own are only useful if you know the person who scored it. That's why when John Walker expresses disdain for a game I immediately open a new tab to check it out because I know he's always wrong about everything as I've been reading his banal nonsense since 2010. If IGN, GameSpot, Polygon, all the big papers regardless of bias, every Steam review that wasn't written by a friend of mine and all of my students agreed that Fallout 4 was great, it wouldn't mean a thing to me because people - as in society in general and the media that it consumes - can be really fucking stupid and wrong. I mean all those people probably liked Skyrim too.

  6. #6
    Activated Node
    Join Date
    Oct 2015
    Posts
    45
    There is one obvious difference between the Pro and user reviews. The Pro's have played 30-50 hours+, the users probably 2-10 hours. A couple of the Pro's said that the game didn't really kick in until after the 10 hour mark so I expect the user review scores to pick up.

    Personally I've played 3 hours so far and I'm not too impressed but I expect (or at least hope) for that to change. I felt pretty much the same when I started playing Fallout 3.

  7. #7
    Secondary Hivemind Nexus
    Join Date
    Jun 2011
    Posts
    7,434
    Metacritic User Scores (and Steam Reviews and - for that matter - Amazon Reviews and other places) are often used to 'review bomb' games that someone took a dislike to. Not many 'professional' reviews will do that so you get a 'skew' for those games in favour of the pro reviews.

    Interestingly, you even get the opposite sometimes. Games which failed to appeal to the pro. reviewers but which gain a lot of traction with players will skew in the opposite direction. Those would be the equivalent of the 'cult' movies which bombed at release but became late-night/DVD success stories (almost every film Kevin Smith has done, for example)

    We've moved-on tho, I don't look at scores anymore - I just listen to what people have to say (yes, I stopped reading written reviews about 2 years ago too!!). People like ACG or Nerd3 or whoever you prefer will be a much better key into whether a game is to your tastes and worth your time.

    There may be a parallel with cars here. When I first got into cars (80s) there were some good, most average and some TERRIBLE cars. Since the mid 00s there really haven't been any 'terrible' cars and most cars available are what would have been considered 'good' in the 80s. Cars are no longer about which ones are unreliable or poorly specced or which ride poorly, they're about more subtle things like equipment/layout and avoiding known gotchas (such as VWs penchant for cheating emissions tests!!) When looking at cars you need a more subjective review now - you need to see if it has room for your kids/dogs and whether it has a cupholder where you prefer them to be etc. etc.

    Games are like that - there are few (tho Steam is trying to improve on that) "Bad games" now - things are more complex than that - scores don't reflect that so well and games which were 10s in the 90s would not be 10s now.

  8. #8
    Fuck steam reviews, fuck metacritic. I want opinions from people I know well enough to calibrate what that means, be they website writers, podcasters, forum goers, or water fountain* chatter.

    It's about knowing how your tastes clash with theirs, but better than all of them? A demo, free weekend or a system in a store, I like to find my impressions most of all[/narcissist].

    *See also: Pubs, Coffee Shops, Stood at urinals, Blood banks and Fight Clubs; essentially anywhere fluids go into or out of bodies.
    I'm failing to writing a blog, specifically about playing games the wrong way
    http://playingitwrong.wordpress.com/

  9. #9
    Lesser Hivemind Node
    Join Date
    Nov 2012
    Posts
    727
    1) Professional reviews generally use only the upper end of the 10-point scale, reserving anything below a 7 for the games that are "safe" to make fun of. (Yes, there are exceptions.) So you've automatically got a biased result.

    2) User reviews not only have sub-groups that are much more willing to use the lower end of the scale for anything that they don't like, they have sub-groups that will use the lowest end of the scale for anything that they don't like that others like. Making matters more chaotic, there are also sub-groups that only use the very top of the scale for games that they like, and will also use it for anything that they don't like that others don't like.

    2b) User reviews are subject to joke reviews. (This is a particular issue with Steam user reviews.)

    3) At least from appearances, originally carried by issues in text reviews and now reinforced by watching videos of professional reviewers playing games, many professional reviewers are actually pretty bad at the games that they talk about. Videos imply that they often aren't even good at the games that they are allegedly good at, while they can be downright abysmal at the rest, if they even give the rest any passing thought.

    4) Some dissension that exists among professional reviewer opinion is stamped out before the review ever sees print. For some professional reviews, you have one person writing the review, while another (or a group of people) decide on the score. This can include people who have a dramatically different opinion than the original writer, those who are scoring it for reasons outside of gameplay, and even people who haven't played the game at all. (Polygon's review of Dragon's Crown is a popular example. I've also heard stories from freelance reviewers, including one case where the rather positive review that he submitted for a game was edited into a negative review and given a low score.)

    Of course there are various other factors. Professional reviewer burnout can bias a score in either direction. User reviews can be biased by a number of people trying to "correct" what they see as bad professional reviews. (I don't mean just giving a game 1s or 10s, but instead stuff like just being more strict just because they feel the professional reviews overlook too many faults.) Etc.
    Last edited by Baines; 10-11-2015 at 04:17 PM.

  10. #10
    Network Hub Shadow's Avatar
    Join Date
    Mar 2014
    Location
    Buenos Aires
    Posts
    148
    Metacritic has the worst possible form of user ratings: anyone can rate, so that can lead to all sort of outlandish results.

    Professional reviews are fairly untrustworthy, prone to bribes from the big name publishers.

    Steam user reviews are imperfect, but taking the Positive-Negative average, they tend to be fairly accurate as a measuring tool for a game's general quality. Accuracy drops to some extent in the case of Early Access games, which can be reviewed throughout their EA period, and Steam doesn't bother to priorize actual release version reviews. They're all bundled together, providing a misleading impression.

    So your best bet is either Steam user review average, or an average of that and professional reviews. Metacritic is bull.

    Of course, reviews are inherently subjective, so you might still dislike a very well-received game. Or viceversa.

  11. #11
    Network Hub
    Join Date
    Feb 2014
    Location
    London, UK
    Posts
    352
    I think it's pretty misguided to conflate "user reviews" in general with "metacritic user reviews". It'd be like conflating "online communities" with "LoL's community circa 2013" or something.

    As Shadow says, Steam's user reviews often have quite a bit of value, and don't exhibit the same "rating a game as shit as a form of protest" (which is putting what happens on metacritic pretty generously) deal, because you actually have to own them. I'd say they're actually the third-best guide to quality, right behind "professional" reviewers you trust (virtually all of whom are essentially well-publicised amateurs, for better or worse), and recommendations from friends who you know aren't idiots about games.

  12. #12
    Network Hub
    Join Date
    Sep 2014
    Posts
    153
    Quote Originally Posted by Baines View Post
    3) At least from appearances, originally carried by issues in text reviews and now reinforced by watching videos of professional reviewers playing games, many professional reviewers are actually pretty bad at the games that they talk about. Videos imply that they often aren't even good at the games that they are allegedly good at, while they can be downright abysmal at the rest, if they even give the rest any passing thought.
    I would actually argue this is a good thing. While it is true that many games are designed to be hard and someone who isn't "good" may rate them more negatively than someone who is, I myself am merely average (as are most readers pretty much by definition). While I like watching and listening to the alpha-uber players as much as the next guy, their opinion of what is "good" and "challenging" is often a poor match for mine.

  13. #13
    Secondary Hivemind Nexus Wenz's Avatar
    Join Date
    Oct 2013
    Location
    Gorizia, Italy
    Posts
    4,065
    I'm not sure what's the deal with review bombs, like if it's just a reaction to pro critics, still a couple of things: metascores are not quality indicators and just tell you about how many are appreciating the game. It's all crap from dudes and gaming sites you never heard of which you don't need to read and it's only useful to some student taking an international marketing class who doesn't want to mail everyone for a month to build his poll as I am. Or some analysis shit.
    On the other hand some dudes on the internets are not very humble and will list you 100 games/books/trendy words/whatever to prove their experience or something but have no clue about games having rules and never learnt to play a single one in years, tldr learn stuff on your own and bother people when you actually need it if you have time to waste on metacritic and stuff (not you op just thinking of metacritic users).
    Last edited by Wenz; 10-11-2015 at 06:52 PM.

  14. #14
    Moderator Grizzly's Avatar
    Join Date
    Jun 2011
    Location
    Dutchland
    Posts
    4,374
    Most of the negativity surrounding the Bethouts is focusing on what the game is not: It's not a direct sequel to FO1 and 2, but rather a gross mutation; It's not an Obsidian game, it's not a particularely deep or philosophical game. But on the other hand, it's not trying to be. Bethesda is taking the 50s sci fi direction but going into comic book and cartoon territory rather then Obsidian's more philosophical musings. The role that the fallouts play in Bethesda's lineup is similar to the Far Cry: Subtitle games: More experimental and silly then their mainstream siblings. Most professional reviewers judge the game based on those metrics, and when one does that, the answer is often positive. If you look at Alec Meer's Wot I Think, it mainly focuses on points where it is better then FO3. The point is clear: It's better then FO3. If you don't like any of Bethesda's lineup you won't like this one, but spending an entire review debasing that the game is not <insert game here> is not anything particularely usefull for anyone: It's just pointless griping.

    Another factor to consider is that all reviewers have a degree of choice in what they review. The people for which playing a Bethesda in-house game is an experience similar to Shutter Island will not volunteer for having a go at another version.

    A bit of an inverse seemed true with Mad Max: A lot of the reviewers were using Shadow of Mordor as their metric, whilst the general public seemed to expect a bogstandard movie tie-in and were reasonably surprised.

  15. #15
    Lesser Hivemind Node
    Join Date
    Nov 2012
    Posts
    727
    Quote Originally Posted by SuddenSight View Post
    I would actually argue this is a good thing. While it is true that many games are designed to be hard and someone who isn't "good" may rate them more negatively than someone who is, I myself am merely average (as are most readers pretty much by definition). While I like watching and listening to the alpha-uber players as much as the next guy, their opinion of what is "good" and "challenging" is often a poor match for mine.
    From a consumer perspective, it can be either good or bad. But regardless it is potentially a factor in why professional reviews are different from user reviews. Even if the masses contain a lot of average to bad gamers, user reviews will see at least some representation from skilled gamers (and at the very least will for better or worse see parroting of skilled gamer opinion.)


    On the matter of whether it is good or bad, like I said, it depends. Yes, such reviews fit a larger demographic. At the same time, such reviews can misrepresent a game due to the reviewer's lack of skill or knowledge. If you've watched many YouTube reviews (or opinion pieces for those who don't like to call their work "reviews"), then surely you've seen cases where the reviewer misses something obvious and then berates the game for things that the game isn't even guilty of. Such reviews can also miss serious design and difficulty issues, because the reviewer never had the skill, or simply never paid enough attention, to realize the issues existed.

    In some cases, such reviews are worse than useless, causing damage to a decent game or overhyping a poor product.

  16. #16
    Lesser Hivemind Node
    Join Date
    Nov 2012
    Posts
    727
    Quote Originally Posted by Shadow View Post
    Steam user reviews are imperfect, but taking the Positive-Negative average, they tend to be fairly accurate as a measuring tool for a game's general quality. Accuracy drops to some extent in the case of Early Access games, which can be reviewed throughout their EA period, and Steam doesn't bother to priorize actual release version reviews. They're all bundled together, providing a misleading impression.
    Steam user reviews, with 9,775 total reviews and a nearly 3:1 ratio of positive to negative, gives Bad Rats a "Mostly Positive" rating. Bad Rats is the poster child for joke reviews, but is certainly isn't the only bad game that has been pushed up to Mixed or Positive territory entirely through joke reviews.

    Steam user reviews have also been manipulated by developers/publishers in the past. Some developers abused Steam's flagging system to remove negative reviews while leaving positive reviews. You also have the same kind of tactic seen elsewhere, where right after release a game sees a bunch of positive reviews by employees, friends, and even newly created accounts. That can get overwhelmed with time, but smaller games tend to see real reviews come in at a slower pace.

    Early Access is a review issue no matter how you handle it. People who write a negative Early Access review aren't likely to come back and rewrite it after a game sees official release. If Steam removed EA reviews post release, or simply weighted them to be much less important, then a developer with some thought could take advantage of the system at least long enough to inflate his rating for a while. (This also introduces the question of what to do when a game sees a post-release overhaul. The same logic that devalues pre-release reviews post-release applies here. And if you do implement review devaluation, what do you do when a publisher abuses it?)

  17. #17
    Secondary Hivemind Nexus
    Join Date
    Jun 2011
    Posts
    7,434
    A lot of reviews removed by developers were removed fairly tho - the rules of the review system require you review the product at-hand, not the developer and their previous practices etc. etc.

    So review bombing games because of something the developer did with another game or over something you personally didn't experience are fair grounds for removal of the review.

    People seem to think they can express their opinions everywhere as some form of 'free speech' (showing they have no idea what that means) - it gets a bit tiresome because a lot of what people are upset about is bollocks and nonsense.

    EVERY Ubisoft game is review-bombed because it requires UPlay, UBI are completely within their rights to ask for those reviews to be removed as it has nothing to do with the game they're reviewing (so long as it works - which is mostly does)

  18. #18
    Lesser Hivemind Node
    Join Date
    Nov 2012
    Posts
    727
    Quote Originally Posted by trjp View Post
    A lot of reviews removed by developers were removed fairly tho - the rules of the review system require you review the product at-hand, not the developer and their previous practices etc. etc.
    Both positive and negative reviews can violate Valve's rules. However, it seems positive reviews hardly ever get removed, presumably because developers hardly ever flag positive reviews.

    And to be fair, it sometimes isn't hard to violate Valve's rules when dealing with certain developers. For example, if you mention that the Steam forum for the game is being heavily censored, then Valve will very likely remove your review if it gets flagged. (I don't know about these days, but Valve used to be very strong handed about silencing complaints about moderation.)

    It doesn't particularly help that Valve doesn't appear to even have clear rules for what you can and cannot say. The "Rules and Guidelines" link above the review box just links to Steam's general guideline post, which was written to cover forum posting and never rewritten as Valve expanded Steam's support for user generated content. The official rules link for reviews doesn't actually say anything about what is and isn't a valid review.

    Nor does it help that Valve is notoriously inconsistent in how it applies its rules.
    Last edited by Baines; 10-11-2015 at 10:47 PM.

  19. #19
    You need to realize that *any* effective ranking system is going to be gamed by all crowds: Obsessive fans, obsessive haters, as well as sneaky developers and journalists.
    There are now plenty of games around that don't fit into Metacritic's 90 range, or even Steam's Very Positive range, but are still enjoyed by many. The only effective way to improve the situation is to simply provide more information for players, and overall that's been done pretty well. More reviews are giving non-declarative scoring (ie, "Yes/Wait for sale/no") and persuading people to read their text, more reviews are giving little Pro/Con summaries at the end, etc. Plus, people can watch others play. There's no better demo, in my opinion.

    The one group that has of course never gotten better is gamers themselves. There's a whole lot of very dumb campaigns made against games for very little reason. You know why I hated Evolve? Because asymmetric multiplayer is hard to do fairly, and the game has a really high skill floor to play competently. NOT because it happened to sell skins as well. Someday, gamers are going to have to stop pre-emptively hating a game for having DLC or making a console port. Tomb Raider 2013 was a cautionary tale - games gotta make more money.

  20. #20
    Network Hub spoken_starfish's Avatar
    Join Date
    Jun 2014
    Location
    Australia
    Posts
    126
    I'm not sure why everyone seems to be reading the Fallout 4 WIT as positive. It seemed it seemed squarely in "mixed" territory, while acknowledging that fans of the series will enjoy the improvements.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •