Results 1 to 8 of 8
  1. #1
    Activated Node
    Join Date
    Dec 2016
    Posts
    33

    Bethesda and No Advance Copies: Should reviewers award GOTY to such titles?

    So. PC Gamer (Which I haven't read consistently in many years) recently made Dishonored 2 their Game of The Year. What follows is a sectioned, directed non-rant. It is, however, quite long, so skip to the Proposal, Policy and Ask if you want.

    Background:
    By all accounts, Dishonored 2 is a good game. The same repeating 5 minutes I've been able to play it for (AMD R9 280, but still gets a solid 40-50 FPS before CTD. I bought it when it came out. I burned up the 2 hour refund window on trying to make it work. That's on me. And I'm not repeating that mistake. I've also periodically checked back on it, including this past weekend, to see how the patch progress was coming along. It hasn't really.

    Still no go. And I'm not alone. There are still substantial slices of people, some of whom pre-ordered, who can't play the game. Now, I don't want this to devolve into a troll war. I'm going to try very hard to not chase after any one developer too hard. So we're switching off this to the more salient topic:

    Dishonored 2 was one of the first games released under Bethesda's new "No Advance copies" policy, in which reviewers would not receive advance copies of games that they normally use to prepare their reviews, allowing Gamers a fairly complete view of what they can expect for their money.

    The crashing issues began to come out after day one launch, but gathered steam through social media. Granted, most reviewers that would receive have otherwise received an advance copy don't have shabby rigs, but I can't help but imagine a few of the more responsible ones wouldn't turf it onto a backup computer just to kick the tires on it (Reviewers who do this are actually really valuable). If this had occurred, perhaps I and others would have been able to avoid an aggravating potential loss of $59.99 and the profound disappointment of losing a weekend to the frustration of what is appears to be an engaging, fun game that I really looked forward to.

    The Issue:
    Which is why it kind of annoyed me when it got Game of the Year. I thought at first that it was just my bias. It frustrated me, so I'm frustrated at it. But the more I think about it, its really a fairness issue. I don't feel cheated by the game, its story, or game play. Again, what I've played intrigues me conceptually, and what I've seen online looks enjoyable. I'm rather fond of Dishonored.

    Instead, I feel cheated by the buggy, ill-conceived game engine, and by the publishing company which must surely be well aware of their QA deficiencies at this point. Normally, I'd turf to where such issues should rightfully belong: With the developers. . On reflection, I conclude its the latter: I've yet to encounter a Bethesda game that did not behave in some mildly breaking fashion even a few months after launch. So either they have created an incredibly deficient stable of developers who have somehow managed to make games that are well-regarded in terms of gameplay, or someone has messy relationships with their stable, and after Dishonored 2, I'm inclined to believe the latter. I'm adopting a position of "No Bethesda pre-orders, immediate refund if not working in 15 minutes." This is what I'm doing for me.

    Problem:
    Which brings us to the review policy. The thing which could've saved me all of this aggravation and frustration. To be clear, I'm not in the business1, merely a gamer. But I think sideways alot, and I'm good at analysis, and I like crunching on problems.

    So this is what appears to be happening in my view: This seems to me like an easy way to boost pre-order sales by creating an advance wave of hype/opinion that is composed almost exclusively of preferred sources, which allows the company to more easily manage expectations, as I imagine its probably easier to cajole or persuade a youtube streamer who is getting direct revenue than an actual professional reviewer, who is often dependent on Ads.

    This media management practice also prevents bad reviews from triggering refund waves prior to release, or dampened day one purchases. This is legal behavior, mind you. And its allowable in the market. But its still a sleezy, sleezy tactic, one that allows for maximized profit, with minimized backlash. This, in turn, leads to customers being actively denied the information required to make a fair value judgement. This, in turns, moves industry metrics regarding what a company reasonably assumes it can get away with releasing, in what state, regardless of customer reactions. If they feel they make soak a few bad reviews and save a month of compatibility testing, why not?

    This creates a market failure, specifically a structurally generated scenario of asymmetric information, which distorts the nature of the exchange and disrupts the power of the consumer. Non-nerd translation: By controlling information, they are able to leverage market position in which a higher percentage of customers make purchasing decisions that result in imbalanced exchanges: Customers feel cheated. Of those, a percentage will not seek redress or refund, resulting in profit for that company at a lower risk.

    But we still have a conundrum: How to respond? If we penalize the game, which may have artistic or critical merit irrespective of marketing practices, we could damage the industry, and it would not be fair to development studios. But if we allow it to go unpunished, then as gamers, our purchasing decisions will slowly be ceded to word-of-mouth and what random people on youtube with soundboards that interrupt their own reviews think - people who we don't know. If a streamer gets an under the table deal, its a controversy, and potentially a legal matter. If a professional site gets paid, it can destroy that publication. The risks of unethical behavior seem much higher for the established publication, something I think RPS has written about before (Tired, will need to check).

    Proposal
    : I would submit the following as a suggested Editorial policy for game review websites:

    In my view, awards such as Game of the Year, i.e general merit and achievement awards, serves as a major market driver for a publisher. Having that one statement "Game of the Year' or "#1 Game of 2017" is dramatically more powerful than "Best Soundtrack" or "Best Multiplayer Experience"

    I understand that also penalizes the developer. Games often become Game of the Year because of the overall high level of achievement of a studio. To be clear, to maintain the maximum possible praise for a developer, no game should receive a blanket ban from individual recognition for specific elements of a game.

    But this practice of companies picking their own heralds and critics is unacceptable, damaging to the industry, and jeopardizes the norms and institutions that have provided us with quality entertainment for years. This is a phenomenon brought on partly by new media, and is happening across the economic spectrum. The phony standards labs are popping again, fake non-profits are certifying each other, and Shaq is once again being contractually obligated to play basketball with animated stereotypes!


    I realize the universe of game reviewers is an eclectic, eccentric bunch of loosely affiliated snowflakes floating on a cloud that is, on good days, halfway volumetric in nature. But that does not preclude having a discussion to get out ahead of something that could potentially destroy the institution, and leave us gamers, and the developers themselves, at the whims and mercy of whatever scruples a publisher may or may not have in a given year. Starting that discussion requires a jumping off point, a place that you can plant a flag as "Too Far for Me" or "Not Far Enough"

    I have taken the liberty of preparing such a policy in a boiler plate fashion. I have also packed you a bag lunch1.

    Proposed Policy:


    [Establishment/Website/Reviewer/Editor's Name] believes in the basic mission of ensuring that our [Readers/Viewers/Subscribers/Semi-AutonomousGoldFarmers] deserves the best possible value for their gaming dollar. When gamers feel cheated, our service loses value, and the industry as a whole suffers for it. When gamers decide to pre-order or purchase on the launch, they depend on this service to inform their purchasing decisions, and it is our obligation to try to provide as complete a picture possible of the totality of that gaming experience.

    How does it run? Do the features work? Is it finished? Can my hardware handle it? These are all questions that gamers ask themselves, the product description, and whomever they trust to review the game. Forming a complete recommendation takes time, and it is not fair to the customers or to the development houses to compel that an opinion be formed only a day before or in the hours after a release.

    The customer suffers from feeling cheated when the product isn't as advertised, whether they are frustrated with a buggy release, or disappointed that the features they wanted aren't included or aren't to their satisfaction. Not all customers have universal rights of return, which are dependent upon where they live. It is unreasonable to expect every game to work all the time, but it is similarly unreasonable to expect a customer to jump through hoops disrupt their lives to return a buggy product.

    These expectations are valuable in and of themselves. Whether they are unfounded hype, legitimate praise, or harsh criticisms. From a technical standpoint, they are part of the product: Does the feature list on release match the advertisements? Does it run well at the bench-marked specifications? They are also, in their own way, part of the experience. Just as a gamer can be disappointed and frustrated, they can, hopefully more often, be joyful and ecstatic that a game turned out to be even better than they expected.

    Whether a customer chooses to partake of our service or not is irrelevant: If the information is being denied or manipulated for a customer, they are not receiving a total experience. Furthermore, if the reviewer is denied this information, we cannot render effective judgements on the totality of a gaming experience, which is why our patrons ostensibly [subscribe/read/freeload] our reviews in the first place.

    To that end, effective immediately, the Editorial Policy of [Insert Your Website/Fiefdom/Blog/WordpressCoven here] that no game which is not available for review 5 days prior to the release of a game can be eligible for awards in recognition of, in the Business/Calendar Year End or a 6 Month period thereafter, whichever comes first, shall be eligible for editorial awards for:

    1) General Achievement
    2) Rankings
    3) Game of the Year

    To be clear: This does not apply to individual achievements or conventional scoring mechanisms, example such as:

    1) Best in Category (Best SoundTrack, Best Multiplayer, etc.)
    2) Review scores
    3) General informal statements or endorsements
    4) Individual endorsements by writers/contributors

    An example would thus: While one member of our establishment may assert it was the best game of the year in their opinion, the game may not be labelled as our game of the year. This example is mentioned to prevent inevitable misunderstandings, and to preserve the freedom of opinion that our writers hold. Please do not hesitate to contact us if there are any misunderstandings, and know that regardless of this decision, we do enjoy your games, and we look forward to working with you to make sure our customers get the best possible opinions for their value.

    Sincerely,
    [Blank/ThatGuy/SomeGirl]





    The Ask:
    So this is nice, but what's my ask here? Its 4:40am, so I'll keep it short.

    If you have thoughts on it, comment!

    I would like to see this as a discussion point within the community of games journalists (Am I allowed to use that word? I want to use that word. It's been bugging me this whole time..), leading to some form of policy agreeable to a broad cross-section of the profession that can be adopted with as few defections or non-enforcement as possible. I recognize that its a loose amalgamation of really creative3 folks, but I feel that agreement and professional discussion is warranted, as this has the potential to develop into a broader problem that will, in time, undermine the industry.

    So why do I care?

    I like playing games that are smart, well-designed endeavors made by caring, loving people.

    I do not like having to shop from defective, deficient, repetitive things that are up-shaded hack jobs from last year with a new protagionist that I can buy one-time use stickers for in an environment where I'll be fighting so fast I won't have time to comment on the affordability of someone buying every hat in the game. When the creation of such products is rewarded by the market, the market responds.

    Having a healthy reviewing industry is, currently the most effective institutional bulwark against this.


    There are issues and criticisms one can make about this proposal, please make them! Its 5AM, so I haven't though of everything. I'll get started:

    What about indy devs that make a really epic game, but aren't experienced enough to even know what an advance copy is?

    Does this automatically mean that only games that are prominent for us to have heard of and sought to play the advance copies thereof can be considered for Game of the Year?

    Also, the text in the red needs rewording. Its too late for me to parse the legal phrasing for that.

    Your random reward for good readership is as follows, one of the original meme faces. Use it for your own reparte:


    1: I am the business. BOOM! Mandatory monthly science fiction reference nailed.
    2: In honor of the forum sponsors, it is an "International Abomination" consisting of an English Biscuit, an American Cost-co Muffin, and a Thermos of Bovril.
    3: Read: Individual voices often separated by distance and opinion. This is totally not a nice way of asserting that you're in any way shape or form disorganized, non-union transient workers that compete with each other over who can agree the fastest that sitting in front of a screen all day is the best job in the world.



  2. #2
    Network Hub
    Join Date
    Sep 2011
    Location
    Falkirk
    Posts
    175
    So... three things come to mind.

    One is that GOTY should be awarded to developers - not publishers - based upon the reviewer(s) merit of the game as a piece of entertainment software. i.e. including technical quality, but not considering the publishers PR strategy (including withholding reviews) or social media opinion. I'd suggest that such an award should not be awarded to any game released within 30 days or so, for obvious reasons... but more importantly that ways should be sought to more explicitly associate any award with the developer and reduce it's use as a tool by those who dictate marketing. I don't know a good answer to that one, though.

    Two is that we really want later reviews of games, not earlier - i.e. to review the game in the state it is in as released, as if the reviewer had picked it up at the shop or downloaded from a DD medium right then. That does require some maturity on the point of the journalist to not get sucked into the marketing of 'first EXCLUSIVE review here', and of the gamer to be prepared to sacrifice immediacy for waiting for reviews.

    Third is that it is extremely difficult to withhold a GOTY type judgement simply because of reported technical issues, if the reviewer cannot self-verify them. People have a habit of shouting loudest when annoyed, least when happy, and complaining unrealistically.

  3. #3
    Secondary Hivemind Nexus
    Join Date
    Feb 2014
    Location
    London, UK
    Posts
    1,867
    Whilst Bethesda's policy is anti-consumer crap, and has stopped me from buying Dishonored 2*, I don't think it has much to do with Game of the Year, either way.

    I also don't think it would have a positive long-term result for reviewers. Best case scenario is that a game that should be GotY is denied it universally by mainstream reviewer sites/magazines, and the publisher is somewhat miffed and might think about what they've done. But I mean, it's not like you actually need to be GotY to put out a GotY edition, is it? Pretty sure at least one GotY edition in history was for a game that wasn't anyone's major's GotY.

    That's literally the actual best case.

    A more likely case is that game that was potentially GotY is doesn't turn up as GotY on any major lists, and the good people of the internet, perhaps egged on by the sillier publishers/developers, decide that this game has been unfairly and cruelly denied GotY by the evil reviewer cartel and/or the whims of the publisher (rather than the entirely innocent and wide-eyed orphans who developed it), and basically throw a stupidity party, to which everyone is invited (or dragged). So it's not good for the reviewers, and in fact the game might even sell more copies than otherwise, as a sort of cause celebre.

    * = Seriously. Dishonored is in my top 20 if not top 10 games ever, but as good as the sequel might be, I'm just not buying it full-price if they're not letting reviewers see it until after release. Period. End of. Etc. In fact I also ignored the 33% off sale on Steam (despite the fact that my PC is now powerful enough to play it properly, thanks to a new CPU). I think more people doing that (as many here evidently did not, including some people who often say "Don't pre-order!" yet appear to have bought it day 1, which is effectively the same thing as pre-ordering in many regards, when there are no pre-release or release reviews.)

  4. #4
    Secondary Hivemind Nexus
    Join Date
    Nov 2012
    Posts
    1,261
    Before getting to the GOTY topic, something else needs to be addressed.

    You are assigning the blame for buying a buggy game at release on the lack of reviews at release. This is very much a similar situation to "don't preorder", where you could have saved yourself frustration if you had simply waited a few days (for adequate site and user feedback to accrue, as well as issue reports) before buying the title. Particularly for a game from a publisher that you acknowledge has a history of such issues, but in general as well.

    No one made you (or anyone else) buy Dishonored 2 at release. More than magazine/site awards or any other such stuff, the most effective way to get a message to publishers and dev studios would be for consumers to stop blindly buying their games on release day. If there are no reviews, or no time was given for reviews, or they didn't send out PC reviews, or they just have a history of problems, or whatever else, don't buy their games at release. Wait a few days and buy them. If enough consumers did that, with publishers seeing the upspike in sales two or three days later, then it might actually sink in that consumers don't trust the publishers.

    Now back to the GOTY thing.
    It doesn't really matter. Those site awards don't matter. The threat of locking publishers out of GOTY awards is a threat with no weight. Games tend to win GOTY because they are popular titles, and can survive without any down-the-road awards bump. Heck, publishers can (and will) just do their own self-proclaimed GOTY re-release if they want a "this is a really popular game" sales bump. The no-advance-reviews policies are in place to protect the known turkeys and known problematic titles. The former aren't going to win GOTY anyway. The latter may or may not, but I've already addressed the "may" and again the threat has no weight against the "not".

    Sites that *did* enact such restrictions would have to worry about backlash from both publishers and fans. Fans already go ballistic when a game that they like gets a 9 instead of a 10, and will similarly flip out if such a title is ignored for a GOTY award just because the reviewers weren't given an advance copy.

    This also touches on the similar issue of review embargoes. Your proposed restriction only applies to not supplying a review copy far enough in advance. But for those consumers that absolutely cannot wait to buy a game, a release day review embargo can be just as effective as not sending out review code.

    It also touches on the longstanding, though perhaps now more relevant than ever, issue of games that aren't the same in review form as they are at release. This use to be because publishers didn't always have enough lead time to send finished code to magazines (and were happy to promise that any issues would be fixed for the final code.) Today it is because many games see day zero or day one patches, sometimes substantial and even game changing ones. As well, sites and publishers have come to loggerheads over the issues with reviewing online multiplayer focused titles when there are only a few reviewers online. Publishers argue that it doesn't reflect the post-release environment where the games will be filled with interacting players, while reviewers (and consumers) argue that it doesn't reflect the post-release environment where the games will be unplayable due to network issues, plagued by griefers, seeing entire modes abandoned due to lack of interest, and the various other issues that only become apparent after release.

    If you are going to make a policy to "punish" publishers for not providing review copies in advance, then you need to address situations like the others above as well. Particularly if you want to present it as being "fair", but also because any publishers that *did* care would just shift to an alternative out if available.

  5. #5
    Activated Node
    Join Date
    Dec 2016
    Posts
    33
    Quote Originally Posted by LexW View Post
    Whilst Bethesda's policy is anti-consumer crap, and has stopped me from buying Dishonored 2*, I don't think it has much to do with Game of the Year, either way.

    I also don't think it would have a positive long-term result for reviewers. Best case scenario is that a game that should be GotY is denied it universally by mainstream reviewer sites/magazines, and the publisher is somewhat miffed and might think about what they've done. But I mean, it's not like you actually need to be GotY to put out a GotY edition, is it? Pretty sure at least one GotY edition in history was for a game that wasn't anyone's major's GotY.

    That's literally the actual best case.

    A more likely case is that game that was potentially GotY is doesn't turn up as GotY on any major lists, and the good people of the internet, perhaps egged on by the sillier publishers/developers, decide that this game has been unfairly and cruelly denied GotY by the evil reviewer cartel and/or the whims of the publisher (rather than the entirely innocent and wide-eyed orphans who developed it), and basically throw a stupidity party, to which everyone is invited (or dragged). So it's not good for the reviewers, and in fact the game might even sell more copies than otherwise, as a sort of cause celebre.

    * = Seriously. Dishonored is in my top 20 if not top 10 games ever, but as good as the sequel might be, I'm just not buying it full-price if they're not letting reviewers see it until after release. Period. End of. Etc. In fact I also ignored the 33% off sale on Steam (despite the fact that my PC is now powerful enough to play it properly, thanks to a new CPU). I think more people doing that (as many here evidently did not, including some people who often say "Don't pre-order!" yet appear to have bought it day 1, which is effectively the same thing as pre-ordering in many regards, when there are no pre-release or release reviews.)
    I do concede that there is a potential backlash there. The Game of the Year is more of an exemplar of what I'm talking about. And its certainly possible for advert splash screens to continue what they do - present the words like "BEST GAME EVER" in big, bold, high contrast letters, while putting the source in smaller font that only contrasts with the background if its a big name reviewer.

    I guess my big concern behind this proposal, is that I feel as if though we are continuing a slide away from where it was always assumed and apparent that people were making games for reasons beyond simply making money. Yes, it was their job to make games, but they wanted the games to work on as many computers as possible, not to maximize their marketshare in a quarter, but because they honestly believed in their product, and felt a duty to ensure that people were having fun. When people came to them with bugs and broken products, you could get a response, even given the primitive state of the industry media and forums of the time.

    I feel like the industry has been gradually moving away from that, to a place where developers don't have as much power to do that. One of the vehicles for controlling that market force is to have reviewers that decrease the information barriers for consumers - so that people can spot games that they don't think will work. They can most effectively leverage that power if the reviews they give are fully informed about the complete game experience. In an era in which massive open worlds, unlocks, progression systems, and all these different types of games are becoming more popular, it is important to note that these can't always be evaluated in a day.

    I agree that I really like Dishonored, and I really, really want to experience Dishonored 2. I know it probably won't be the same. But still, even if its only for Stephen Russell.

    Perhaps the profession of game's journalism can collaborate on a "Buyer's Bible" of simple suggestions and maxims, examples of how things have gone wrong in the past, things to watch out for such as judging your specs, navigating support tickets and refund policies, etc.

  6. #6
    Secondary Hivemind Nexus
    Join Date
    Nov 2012
    Posts
    1,261
    Quote Originally Posted by Insignus View Post
    Perhaps the profession of game's journalism can collaborate on a "Buyer's Bible" of simple suggestions and maxims, examples of how things have gone wrong in the past, things to watch out for such as judging your specs, navigating support tickets and refund policies, etc.
    That would require games journalists and sites to actually collaborate and cooperate, rather that continue to slit each others and their own throats.

    Heck, sites can't even stick to their own messages. Plenty of sites have done "Never fall for the hype" and "Don't pre-order" articles, while simultaneously publishing "Hype! Hype! Hype! I can't wait! Guaranteed game of the year!" previews, "Look at the amazing pre-order bonuses" news, "Here is our amazing 15-part strategy guide telling you everything you need to know about the upcoming game (article text supplied by the dev studio)", and the like. Maybe sometimes with a tiny "we don't promote preordering" disclaimer near the end, when the author or editor remembers.

    And how often have you seen games journalists swear off any responsibility, whenever anyone tries to direct responsibility their way? In recent years we've gone from denials of responsibilities and ignoring inconsistencies to sometimes outright shutting down the topic of discussion when it turned back to games journalists.

    I just don't see enough influential sites organizing and sticking with such a task. And if the sites didn't themselves hold themselves to certain standards, then any "Buyer's Bible" would be soon forgotten in the flood of continued hype (just as the random "Don't preorder" article gets forgotten amidst all the preorder-promoting hype articles.

  7. #7
    Secondary Hivemind Nexus Heliocentric's Avatar
    Join Date
    Jun 2011
    Posts
    17,554
    Valve will likely let you refund a game that won't run on a system that meets minimum spec.

    The 2 hour thing is just an automatic system to reduce workload.
    I'm failing to writing a blog, specifically about playing games the wrong way
    http://playingitwrong.wordpress.com/

  8. #8
    Activated Node
    Join Date
    Dec 2016
    Posts
    33
    Quote Originally Posted by Baines View Post
    That would require games journalists and sites to actually collaborate and cooperate, rather that continue to slit each others and their own throats.

    Heck, sites can't even stick to their own messages. Plenty of sites have done "Never fall for the hype" and "Don't pre-order" articles, while simultaneously publishing "Hype! Hype! Hype! I can't wait! Guaranteed game of the year!" previews, "Look at the amazing pre-order bonuses" news, "Here is our amazing 15-part strategy guide telling you everything you need to know about the upcoming game (article text supplied by the dev studio)", and the like. Maybe sometimes with a tiny "we don't promote preordering" disclaimer near the end, when the author or editor remembers.

    And how often have you seen games journalists swear off any responsibility, whenever anyone tries to direct responsibility their way? In recent years we've gone from denials of responsibilities and ignoring inconsistencies to sometimes outright shutting down the topic of discussion when it turned back to games journalists.

    I just don't see enough influential sites organizing and sticking with such a task. And if the sites didn't themselves hold themselves to certain standards, then any "Buyer's Bible" would be soon forgotten in the flood of continued hype (just as the random "Don't preorder" article gets forgotten amidst all the preorder-promoting hype articles.
    I agree that getting that level of coordination, even inside a publication, would be challenging, however, I'm not seeing much of a way beyond that. It's either this or cede the industry to preferred outlets (Some of which can be good, others, not so much. I'm not sold on the medium) and rent-a-writers.

    Could the burden be shifted also to distribution platforms such as Steam and GoG? I don't now if this is already in place, but it would make sense for steam to surcharge a vendor for a defective item (Brick and mortar does this) and in the case of steam, it can be justified on the premise of having to pay for staff hours to arbitrate and untangle the more complicated refund requests.

    On a related note: I wonder if Steam has the potential to integrate bug-reporting, and can use that to track crash data. That would be a mammoth soft-ware tie-in, but could be useful for general, bulk crash reporting, especially when coupled with Steam's hardware survey component. This could be used by both developers and steam itself to track if a product is encountering specific hardware-related disorders, by use of automated statistical routines. Surely someone has thought of this.

    On another related point: I think it'd be humorous if, on 10 scale reviews, a game could lose half a point on a review solely on the basis of "Game was released with/without a Clue." covering various items such as "Does it have extraneous MicroTransactions?" "Does it have remappable controls for PC on Day One?"

    You know, all the things that we used to scoff at and assert that No True Developer would ever include in their game.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •