Results 1 to 20 of 124
Thread: Professional versus User Reviews
10-11-2015, 12:55 PM #1
Professional versus User Reviews
Fallout 4's Metacritic reviews by major publications,
Metacritic's user-reviews section for same.
Again, the great gap. Its width will vary over time, pushed back and forth in the schoolyard of public opinion by the many winds. We'll see the small-time pro-reviewers who received no advance code weigh in - cynical, unrushed, not living on amphetamines and skimming surface texture in order to make a pre-release publish. We'll see a few 're-reviews', quasi-retrospectives from the bigger sites that enjoy keeping their communities on their toes, fanning the flames of scandal just a touch.
The users, meanwhile, will mob furiously in one direction then another - we'll no-doubt see the younger generation who see the term 'fan-boy' as something other than an ad-hominem attack rally to the game's defense despite having not yet received adequate pocket money to invest. We'll see a jaded mass attach themselves to the prevailing wind of a given season and swamp the review aggregators with vitriol bearing little resemblance to their own experience with the product - or lack thereof.
Nonetheless, if there's anything to be drawn from public opinion on videogame releases, this moment in a game's shifting reception is as valid as any.
What I query may appear to have an obvious answer, but I'd like to do so from an innocent perspective and dissect it, with the aid of anyone interested in weighing in, with an open mind and as little cynicism as I can manage.
Why the gulf between public and professional opinion, in general? Is it passť to mention paid review scandals, or childishly sour of me to entertain the notion that such a broad swathe of those given pre-release code are barred from giving a negative review? Some of you will, I'm sure, think I'm silly to even question it. "Of course it's all bloody corrupt! Welcome to Earth!", but I'm equally sure others will disagree, and I'd like to hear from both camps and any I've not mentioned.
Secondly, I wonder... with RPS as my primary source of gaming news, opinion and community and in part because RPS themselves have weighed in on the highly positive side of opinion on this project... and many other major releases of recent years - something I have again found unexpected, despite it being some years since RPS truly upheld their reputation for calling out such behaviour and erring toward the more analytical, cynical, irreverant and hard-to-please in their opinion-pieces.
There was a time when no game could easily please the RPS team without being something - for example - massively technically flawed but overflowing with soulful energy, a clever message, or a zealous sense of fun that bucked a predicted trend and pleased the 'IThinker' despite itself. Or, indeed, simply a truly excellent game by multiple measures, upon that rare occasion. The overhyped were called out, the misleadingly advertised needled with a sharp point, and the merely mediocre were heatedly sandblasted by John Walker's infamous Withering-Scornomatic 850.
Some years have passed since I had that feeling, however - with glowingly positive reviews of a 'You might be expecting me to hate this, BUT, I don't, nyeh nyeh!' sort followed by a lengthy series of 'I am not going to mention all the bad things about this' statements of a curiously defensive nature while complimenting traits that had been criticised in similar form often by the same writer in the recent past.
Meanwhile this curious shift in attitude toward mainstream 'triple A' releases, and disproportionate leaning toward positivity in general appears specific to 'Wot I Think' and 'Impressions' pieces, while the analytical cynicism remains quite a strong theme in investigative pieces and retrospectives on products and companies that have fallen from favour and financial/legal influence. Further, that RPS remain quite critical of less well-funded independent products, and those released by troubled and non-Western publishers, although an overall shift to broad positivity remains a very visible trend, with harsher opinions becoming increasingly rare, and lukewarm to overwhelmingly positive feelings about a broad range of games becoming so common that I've found myself less inclined to place any weight upon what I read here.
It's a cruel thing to ponder, but I will at least wonder openly if I'm the only fan of RPS and long-term community member who finds themself wondering about this. Is it just me, being a paranoid grump, or have you felt RPS appears to be diluting their opinions over time, and failing to critically analyse new releases, particularly mainstream releases, with their old rigour and realism?
And that perhaps this is an inevitability as the founders have phased out, moved on, or grown quiet, while new writers have arrived alongside the supporter program. And, of course, while the site's front-page coverage has shifted heavily toward e-sports and MMO coverage, Streaming culture and hardware/software and law controveries... and other Serious Matters of a more lucrative age of videogames.
Requisite TL;DR and summary; Why, obvious as it may seem, are we seeing exclusively positive reviews of demonstrably flawed mainstream releases, and is RPS following the same age-old pattern? We are each entitled to 'wot we think', yet does RPS seem to be thinking in a rather predictable way, more and more as years go by? Why, if so, and can an independent review and opinion source truly survive anymore, or will there always be a lifespan before things change?
Thanks for reading. I look forward to your thoughts.
10-11-2015, 01:01 PM #2
Right, but don't use metacritic ever.
Sample user reviews that are affecting the total user score which sits at 6.0 right now.
This guy gave it a 2
In Fallout 5 the SPECIAL system will be simplified by having only one number, going from 1 to 10, that tells how cool is your character. Perks are completely removed.
In Fallout 6 you only have one dialogue option. You can press X to initiate dialogue whenever you want. Game still get 10/10 reviews and it is chosen as RPG of the year.
Fallout 7 have complete Facebook integration.
It's not a fallout game. simply, it's another shooter. I hope sometime Bethesda would sell this franchise to a company like Obsidian(they are more interested in creating rpg games).
Please Bethesda, let Obsidian make another Fallout game. This is just pathetic. Everything is poor in this game; graphics, writing, story, even the combat.This game doesn't work on my typewriter with 32-bit windows. 0 points from me. I like my little pony: friedship is war. That is a game we should all play."Halo is designed to make the player think "I look like that, I am macho sitting in my undies with my xbox""
10-11-2015, 01:11 PM #3
10-11-2015, 01:22 PM #4
- Join Date
- Feb 2012
- Stockton-on-Tees, UK
Why are you surprised that RPS, a website that has a long history of really liking Bethesda games, should be positive about a Bethesda game that is very clearly in the distinct Bethesda style?
I will continue to defend review scores, user reviews, and metacritic as useful tools. They provide helpful information quickly.Irrelevant on further examination of the rest of the thread.
10-11-2015, 01:33 PM #5
Review scores on their own are only useful if you know the person who scored it. That's why when John Walker expresses disdain for a game I immediately open a new tab to check it out because I know he's always wrong about everything as I've been reading his banal nonsense since 2010. If IGN, GameSpot, Polygon, all the big papers regardless of bias, every Steam review that wasn't written by a friend of mine and all of my students agreed that Fallout 4 was great, it wouldn't mean a thing to me because people - as in society in general and the media that it consumes - can be really fucking stupid and wrong. I mean all those people probably liked Skyrim too.
10-11-2015, 01:34 PM #6
- Join Date
- Oct 2015
There is one obvious difference between the Pro and user reviews. The Pro's have played 30-50 hours+, the users probably 2-10 hours. A couple of the Pro's said that the game didn't really kick in until after the 10 hour mark so I expect the user review scores to pick up.
Personally I've played 3 hours so far and I'm not too impressed but I expect (or at least hope) for that to change. I felt pretty much the same when I started playing Fallout 3.
10-11-2015, 01:41 PM #7
- Join Date
- Jun 2011
Metacritic User Scores (and Steam Reviews and - for that matter - Amazon Reviews and other places) are often used to 'review bomb' games that someone took a dislike to. Not many 'professional' reviews will do that so you get a 'skew' for those games in favour of the pro reviews.
Interestingly, you even get the opposite sometimes. Games which failed to appeal to the pro. reviewers but which gain a lot of traction with players will skew in the opposite direction. Those would be the equivalent of the 'cult' movies which bombed at release but became late-night/DVD success stories (almost every film Kevin Smith has done, for example)
We've moved-on tho, I don't look at scores anymore - I just listen to what people have to say (yes, I stopped reading written reviews about 2 years ago too!!). People like ACG or Nerd3 or whoever you prefer will be a much better key into whether a game is to your tastes and worth your time.
There may be a parallel with cars here. When I first got into cars (80s) there were some good, most average and some TERRIBLE cars. Since the mid 00s there really haven't been any 'terrible' cars and most cars available are what would have been considered 'good' in the 80s. Cars are no longer about which ones are unreliable or poorly specced or which ride poorly, they're about more subtle things like equipment/layout and avoiding known gotchas (such as VWs penchant for cheating emissions tests!!) When looking at cars you need a more subjective review now - you need to see if it has room for your kids/dogs and whether it has a cupholder where you prefer them to be etc. etc.
Games are like that - there are few (tho Steam is trying to improve on that) "Bad games" now - things are more complex than that - scores don't reflect that so well and games which were 10s in the 90s would not be 10s now.
10-11-2015, 02:51 PM #8
Fuck steam reviews, fuck metacritic. I want opinions from people I know well enough to calibrate what that means, be they website writers, podcasters, forum goers, or water fountain* chatter.
It's about knowing how your tastes clash with theirs, but better than all of them? A demo, free weekend or a system in a store, I like to find my impressions most of all[/narcissist].
*See also: Pubs, Coffee Shops, Stood at urinals, Blood banks and Fight Clubs; essentially anywhere fluids go into or out of bodies.I'm failing to writing a blog, specifically about playing games the wrong way
10-11-2015, 04:14 PM #9
- Join Date
- Nov 2012
1) Professional reviews generally use only the upper end of the 10-point scale, reserving anything below a 7 for the games that are "safe" to make fun of. (Yes, there are exceptions.) So you've automatically got a biased result.
2) User reviews not only have sub-groups that are much more willing to use the lower end of the scale for anything that they don't like, they have sub-groups that will use the lowest end of the scale for anything that they don't like that others like. Making matters more chaotic, there are also sub-groups that only use the very top of the scale for games that they like, and will also use it for anything that they don't like that others don't like.
2b) User reviews are subject to joke reviews. (This is a particular issue with Steam user reviews.)
3) At least from appearances, originally carried by issues in text reviews and now reinforced by watching videos of professional reviewers playing games, many professional reviewers are actually pretty bad at the games that they talk about. Videos imply that they often aren't even good at the games that they are allegedly good at, while they can be downright abysmal at the rest, if they even give the rest any passing thought.
4) Some dissension that exists among professional reviewer opinion is stamped out before the review ever sees print. For some professional reviews, you have one person writing the review, while another (or a group of people) decide on the score. This can include people who have a dramatically different opinion than the original writer, those who are scoring it for reasons outside of gameplay, and even people who haven't played the game at all. (Polygon's review of Dragon's Crown is a popular example. I've also heard stories from freelance reviewers, including one case where the rather positive review that he submitted for a game was edited into a negative review and given a low score.)
Of course there are various other factors. Professional reviewer burnout can bias a score in either direction. User reviews can be biased by a number of people trying to "correct" what they see as bad professional reviews. (I don't mean just giving a game 1s or 10s, but instead stuff like just being more strict just because they feel the professional reviews overlook too many faults.) Etc.
Last edited by Baines; 10-11-2015 at 04:17 PM.
10-11-2015, 04:54 PM #10
Metacritic has the worst possible form of user ratings: anyone can rate, so that can lead to all sort of outlandish results.
Professional reviews are fairly untrustworthy, prone to bribes from the big name publishers.
Steam user reviews are imperfect, but taking the Positive-Negative average, they tend to be fairly accurate as a measuring tool for a game's general quality. Accuracy drops to some extent in the case of Early Access games, which can be reviewed throughout their EA period, and Steam doesn't bother to priorize actual release version reviews. They're all bundled together, providing a misleading impression.
So your best bet is either Steam user review average, or an average of that and professional reviews. Metacritic is bull.
Of course, reviews are inherently subjective, so you might still dislike a very well-received game. Or viceversa.
10-11-2015, 05:51 PM #11
- Join Date
- Feb 2014
- London, UK
I think it's pretty misguided to conflate "user reviews" in general with "metacritic user reviews". It'd be like conflating "online communities" with "LoL's community circa 2013" or something.
As Shadow says, Steam's user reviews often have quite a bit of value, and don't exhibit the same "rating a game as shit as a form of protest" (which is putting what happens on metacritic pretty generously) deal, because you actually have to own them. I'd say they're actually the third-best guide to quality, right behind "professional" reviewers you trust (virtually all of whom are essentially well-publicised amateurs, for better or worse), and recommendations from friends who you know aren't idiots about games.
10-11-2015, 05:53 PM #12
- Join Date
- Sep 2014
10-11-2015, 06:00 PM #13
I'm not sure what's the deal with review bombs, like if it's just a reaction to pro critics, still a couple of things: metascores are not quality indicators and just tell you about how many are appreciating the game. It's all crap from dudes and gaming sites you never heard of which you don't need to read and it's only useful to some student taking an international marketing class who doesn't want to mail everyone for a month to build his poll as I am. Or some analysis shit.
On the other hand some dudes on the internets are not very humble and will list you 100 games/books/trendy words/whatever to prove their experience or something but have no clue about games having rules and never learnt to play a single one in years, tldr learn stuff on your own and bother people when you actually need it if you have time to waste on metacritic and stuff (not you op just thinking of metacritic users).
Last edited by Wenz; 10-11-2015 at 06:52 PM.
10-11-2015, 06:29 PM #14
Most of the negativity surrounding the Bethouts is focusing on what the game is not: It's not a direct sequel to FO1 and 2, but rather a gross mutation; It's not an Obsidian game, it's not a particularely deep or philosophical game. But on the other hand, it's not trying to be. Bethesda is taking the 50s sci fi direction but going into comic book and cartoon territory rather then Obsidian's more philosophical musings. The role that the fallouts play in Bethesda's lineup is similar to the Far Cry: Subtitle games: More experimental and silly then their mainstream siblings. Most professional reviewers judge the game based on those metrics, and when one does that, the answer is often positive. If you look at Alec Meer's Wot I Think, it mainly focuses on points where it is better then FO3. The point is clear: It's better then FO3. If you don't like any of Bethesda's lineup you won't like this one, but spending an entire review debasing that the game is not <insert game here> is not anything particularely usefull for anyone: It's just pointless griping.
Another factor to consider is that all reviewers have a degree of choice in what they review. The people for which playing a Bethesda in-house game is an experience similar to Shutter Island will not volunteer for having a go at another version.
A bit of an inverse seemed true with Mad Max: A lot of the reviewers were using Shadow of Mordor as their metric, whilst the general public seemed to expect a bogstandard movie tie-in and were reasonably surprised.
10-11-2015, 07:31 PM #15
- Join Date
- Nov 2012
On the matter of whether it is good or bad, like I said, it depends. Yes, such reviews fit a larger demographic. At the same time, such reviews can misrepresent a game due to the reviewer's lack of skill or knowledge. If you've watched many YouTube reviews (or opinion pieces for those who don't like to call their work "reviews"), then surely you've seen cases where the reviewer misses something obvious and then berates the game for things that the game isn't even guilty of. Such reviews can also miss serious design and difficulty issues, because the reviewer never had the skill, or simply never paid enough attention, to realize the issues existed.
In some cases, such reviews are worse than useless, causing damage to a decent game or overhyping a poor product.
10-11-2015, 07:49 PM #16
- Join Date
- Nov 2012
Steam user reviews have also been manipulated by developers/publishers in the past. Some developers abused Steam's flagging system to remove negative reviews while leaving positive reviews. You also have the same kind of tactic seen elsewhere, where right after release a game sees a bunch of positive reviews by employees, friends, and even newly created accounts. That can get overwhelmed with time, but smaller games tend to see real reviews come in at a slower pace.
Early Access is a review issue no matter how you handle it. People who write a negative Early Access review aren't likely to come back and rewrite it after a game sees official release. If Steam removed EA reviews post release, or simply weighted them to be much less important, then a developer with some thought could take advantage of the system at least long enough to inflate his rating for a while. (This also introduces the question of what to do when a game sees a post-release overhaul. The same logic that devalues pre-release reviews post-release applies here. And if you do implement review devaluation, what do you do when a publisher abuses it?)
10-11-2015, 08:22 PM #17
- Join Date
- Jun 2011
A lot of reviews removed by developers were removed fairly tho - the rules of the review system require you review the product at-hand, not the developer and their previous practices etc. etc.
So review bombing games because of something the developer did with another game or over something you personally didn't experience are fair grounds for removal of the review.
People seem to think they can express their opinions everywhere as some form of 'free speech' (showing they have no idea what that means) - it gets a bit tiresome because a lot of what people are upset about is bollocks and nonsense.
EVERY Ubisoft game is review-bombed because it requires UPlay, UBI are completely within their rights to ask for those reviews to be removed as it has nothing to do with the game they're reviewing (so long as it works - which is mostly does)
10-11-2015, 10:45 PM #18
- Join Date
- Nov 2012
And to be fair, it sometimes isn't hard to violate Valve's rules when dealing with certain developers. For example, if you mention that the Steam forum for the game is being heavily censored, then Valve will very likely remove your review if it gets flagged. (I don't know about these days, but Valve used to be very strong handed about silencing complaints about moderation.)
It doesn't particularly help that Valve doesn't appear to even have clear rules for what you can and cannot say. The "Rules and Guidelines" link above the review box just links to Steam's general guideline post, which was written to cover forum posting and never rewritten as Valve expanded Steam's support for user generated content. The official rules link for reviews doesn't actually say anything about what is and isn't a valid review.
Nor does it help that Valve is notoriously inconsistent in how it applies its rules.
Last edited by Baines; 10-11-2015 at 10:47 PM.
11-11-2015, 01:51 AM #19
You need to realize that *any* effective ranking system is going to be gamed by all crowds: Obsessive fans, obsessive haters, as well as sneaky developers and journalists.
There are now plenty of games around that don't fit into Metacritic's 90 range, or even Steam's Very Positive range, but are still enjoyed by many. The only effective way to improve the situation is to simply provide more information for players, and overall that's been done pretty well. More reviews are giving non-declarative scoring (ie, "Yes/Wait for sale/no") and persuading people to read their text, more reviews are giving little Pro/Con summaries at the end, etc. Plus, people can watch others play. There's no better demo, in my opinion.
The one group that has of course never gotten better is gamers themselves. There's a whole lot of very dumb campaigns made against games for very little reason. You know why I hated Evolve? Because asymmetric multiplayer is hard to do fairly, and the game has a really high skill floor to play competently. NOT because it happened to sell skins as well. Someday, gamers are going to have to stop pre-emptively hating a game for having DLC or making a console port. Tomb Raider 2013 was a cautionary tale - games gotta make more money.
11-11-2015, 02:05 AM #20
I'm not sure why everyone seems to be reading the Fallout 4 WIT as positive. It seemed it seemed squarely in "mixed" territory, while acknowledging that fans of the series will enjoy the improvements.