Last week, Eurogamer made a change. It was, in fact, a monumental one. It was a simply stated conundrum: “How should we score an excellent game with severe networking issues? A flawlessly polished game with a hackneyed design? A brilliantly tuned multiplayer experience with dreadful storytelling?”
The result? They dropped review scores. It has been a long time coming. Joystiq, just before their unfortunate and unceremonious shuttering, came to the same conclusion after asking themselves the same question. Ars Technica, in fact, has never had review scores.
Let’s back up, however, and start with why review scores have stuck around so long. The lineage is fairly obvious. Movies, music, and books are often graded on scales of numbers and letters and thumbs, and those mediums and their critics have been around far longer than video games. It seemed like a good model to follow.
And it worked, mostly. Very early on it was a challenge to fit our products into the static models that preceded them. GamePro in the 80s had the gem of an idea to introduce a faceted approach, bringing about the much derided Fun Factor attribute. It was, however, an entirely valid attempt at remedying what turned out to be just a symptom of the problem.
It’s just unfortunate that it didn’t work. Like all movies aren’t made to make you laugh, not all games would (eventually) be made to be fun—or at least the kind of fun that made you make the GamePro guy face. So then some all-encompassing score was engendered out of that, or at least reinforced at the outlets that already had that.
But that sentiment—not all games are made with the same intent—would only truly come to a head many years later. The goal, for example, of Adventure is very different from the goal of Gone Home. It did, however, once again miss the mark.
The sentiment should have really been boiled down to “not all games are made the same nor for the same reason,” though that also wouldn’t be revealed until (relatively) far later. While almost all games used to fall in the same range of ambition, we now have mobile, indie, triple-A, and so many layer in between. Even the indie crowd is bifurcating into high profile and the lean solo or duo team shops.
There lies the first big problem with the review score. Just because the scope of a game is largely reduced either out of necessity or by choice, that doesn’t mean it should be knocked points or letter tiers from its final evaluation. Is Angry Birds any less of a game than World of Warcraft? Not even a little bit.
But then that surfaces the issue of merited value. Let’s say a site gave both Angry Birds and World of Warcraft a 6 out of 10, but the former only costs a dollar and the latter is a full $60 upfront along with $15 a month after that. So then where does that get factored in? Is that the duty of the reviewer or the reader to judge that monetary value? But they both got a 6 out of 10, so shouldn’t they both be of commensurate purchasing impetus?
The next folly of the system is that games were perceived to be as static as the entertainment mediums that preceded them and the scores that rose up alongside them. But even arcade cabinets weren’t immune to updates what with different boards being installed and different hardware sets for sticks and buttons and knobs. That still, however, meant the review had to live and die with the game itself.
Nowadays games come out with the transparent intent of updating on day one. They are made to change, to grow alongside player needs and demands. If not just bug fixes, there are substantial content additions, or even games that pivot almost their whole aim post-release. It’s a strange but inevitable development in a purely digital medium.
Polygon’s response was to implement an updating scheme where depending on updates or necessary analysis of massively multiplayer products. SimCity, for instance, went from a 9.5 to a 6.5 after three updates because what point is there to a well-designed game that you can’t play. It’s a strange system, though, where the main body of the review remains static and addendums appear at the bottom, leaving it as an exercise to the reader to superposition each layer of criticism.
It’s a historical record for a line of products that know no such thing. There is no great vault of games kept public at each versioned iteration. Without the playable context, the older critiques are merely a compendium of opinion without an anchor. It’s certainly not the fault of Polygon or anyone else who implements this design, but it only solves half the problem.
Then there’s the problem of the meta consequences of the scores themselves, namely Metacritic, a site that aggregates numerically compatible (and weighted) scores into averages. The idea itself is not a problem, but Metacritic, for one, refuses to break out of the realm of static scores in an age of dynamic products, an implement to supposedly “protect” writers.
This wouldn’t be worth contending if it wasn’t a known, no matter how rare, practice to give bonuses and raises to the scores. It is a wholly confounding exercise, basing compensation on the ability to appeal to the lowest common denominator in the best possible way. Not every game is going to be for everyone, so why try to force developers to make every square peg in in every round hole?
If not for that outside influence, review scores would be fine. It’s something akin to a personal gauge of value for time, money, and effort. So many things, though, have to be brought to transparency, such as the personal meters of the writers. That’s why the reviews of a site like Giant Bomb work so well. The personalities and tastes of that staff is well known, so its readership knows precisely how to process their words.
It’s not a singular problem with a singular solution. If it was, it would have hopefully been solved by now, or at least addressed and well on its way to Doneville. But there’s an entire business built around the scores that is making the practice toxic, and it’s a business built on an archaic foundation that is trying to rule a neo-populace. With any luck, there’s a revolution coming to knock down the doors.