An Ancient Argument
I've given myself 37 minutes to write this post in. You won't learn anything new from it.
Kieron's covered the problems with MMO reviews in a reliably masterful way, and there really isn't much to be said beyond it. All I can do is repeat some of what others have touched on a thousand times previous, with different words at a different time, and with my own sense of awkwardness and upset. I don't have an answer. That doesn't make it any less relevant.
Today's drama has been unpleasant, both personally and in terms of feeling really awful about how the two fundamental sides of the games criticism divide - the creators and the commentators - see each other. That's (almost) all I'm going to say on that specific matter.
What I will offer is comment on why that kind of situation can ever develop, which unavoidably becomes the eternal bete noir of how games journalists can possibly review MMOs.
They can't. They just can't. Not to anyone's satisfaction, at least. They have to please day-1 buyers, they have to please beta testers who pledged undying allegiance months ago, they have to please people typing "[game name] + review" into Google. They have to please people with £30 to spend, and please people with £300 to spend over a period of months or years.
That investment is why a company might ask that a reviewer give an online game more time, a chance to bed in and see how it takes shape once a community forms around it. That's also why someone might turn up on a site on a Monday morning, wondering whether they're about to drop their cash in the right place, then wander off when they don't get their answer. Developers need fair representation, but customers need timely buying advice.
21 minutes.
Over the last year, there have been multiple cases of MMO reviews being pulled or re-done because server logs suggest enough time wasn't given. There's an awful lot of validity in that kind of response, because MMOs do genuinely require more time, and because most publications don't provide enough expenses or time to do the job that's required. Trouble is, publishers still want positive reviews up in that sales-critical release week - and positive reviews of an MMO can be exactly as short-sighted as negative reviews, so awkward compromises are reached. Whether positive or negative, such reviews look at the potential of what the community might bring to bear, because the actuality of it simply doesn't exist yet. A prediction is never anything more than a prediction.
There is a trust issue, though. A presumption that journalists are looking for the easy route, that they won't appreciate what a game could be and, unstated but implicit, that they just don't understand. Unless they've played for whatever is deemed long enough, their experience and opinions can be seen as invalid, making judgment calls based upon what they do see isn't good enough, and accusations will inevitably ring that nothing can be said or appreciated until hour or day X. If a judgment is issued before then, it'll be called cruel, lazy and irrelevant. Such a response can be absolutely right - a lot of MMOs don't come into their own until the high-level stuff's experienced, and most of all until someone has built up and invested in both a character and a community. The trouble is presuming/deciding that that's the case, yet expecting that a journalist on a short deadline and the same fee as reviewing an eight-hour nothingy shooter can somehow get to that point immediately. That's why we get post-release embargoes.
16 minutes.
There have been enough mishaps and misjudgments to, perhaps, warrant this mistrust. Bad eggs exist in every field. But how do we earn that trust back, when the status quo remains snatching time on intermittent beta servers or private QA areas, cobbling together a sense of what might be rather than what is? The double-bind of doing an online game justice in the eyes of its creators and most passionate players, but also to provide useful buying advice to an audience on the day of a game's release, remains. Post-release coverage can be arranged, if budgets and deadlines are flexible enough, but so much hangs on that week-one coverage, for consumers and creators alike.
9 minutes.
Of course journalists want to do a good job, and celebrate what an MMO does well. Mandating that they do it under specific conditions and time periods isn't going to magically ensure that, though. There needs to be conversation, not accusatory server logs and absurd embargoes. Obviously that raises all manner of issues in and of itself, but in my experience, the question of how to review an MMO has never been adequately tackled by either side. Magazines and websites have deadlines, publishers and developers have server and population and privacy issues. So they butt heads and reach mutually uncomfortable compromises, and the net result is always that the game has to be reviewed by someone who hasn't been able to have enough access to the game.
The most fascinating element of today's fallout, to me, is that Euro journalists can now play APB's beta on US servers from the 26th, two days earlier than previously. Why wasn't that the case to start with? Because someone didn't want it to be, and someone else didn't want to pay for a reviewer to take that much time.
No-one wants to do a bad job. No-one wants to tear something down unduly. But I fear that, because of mistrust and limited access and painfully short deadlines, we're headed towards a point where that's the presumption. So instead of conversation, we get "you will do this." How is that possibly going to lead to excitement and betterment?
1 minute. Perfect.