Numbers Speak Louder Than Words?: In Defense of Rating Games However You Want

Since RPGFan's creation, we've been reviewing games on a "school system" scale: 0 to 100 percent, with the average (mean and median) being somewhere between 75 and 85 percent. We also break down five "subscores" before the overall score (for graphics, sound, gameplay, control, and story). The rating system we use has had its ups and downs. It's very "meta-friendly," since most aggregate-scoring sites will use percentages as well. But we've received plenty of complaints as well. They sound something like this:

"Why don't you hold 50% as the 'average' game's score? You're wasting the usage of the lower half, and cramming all your scores into the upper ranges."

"The school system suffers from the problem of 'grade inflation,' and it seems like you're going along with them!"

"How could you possibly rate one game 86%, and another one 87%, and objectively say that one is better than the other? Wouldn't it be better to just say that both games are a 4 out of 5?"

And I'm not here to defend our scoring system as the best. Because it's not the best. This isn't an editorial where I claim RPGFan has the best scoring system in the universe. There are flaws in it, just like there are flaws in any rating system.

But I am here to make two points. First point: it doesn't really matter how you rate the games on a numerical scale. You can do it out of 100, or out of 40, or 10, or 5. You can have halves (.5), quarters (.25), or even tenths (at this point, though, a ten-point scale with one decimal place may as well be a percentage scale). You can also have a simple "good" or "bad" rating, like Roger Ebert giving a thumbs-up or a thumbs-down, or Rotten Tomatoes (my favorite aggregate site for film reviews) having ripe or rotten (splattered) tomatoes. You could even do something really silly, like include negative numbers in your scale, or work in base 6 instead of base 10. As long as a person can quickly figure out from your scoring system that you generally like or dislike the game, and can look at the scores of many games to get a feel for what the spread and standard deviation of scores looks like for your repertoire of reviews, you're golden.

(And if a reader can't do that for most sites, might I suggest that they're probably not very good at RPGs, which are traditionally very math and logic heavy...for a game anyway.)

Our friendly rivals at RPGamer use a five point scale. If you take their review scores and our review scores, normalize our numbers (using an "A B C D F" scale to match the numbers 5 4 3 2 1), and compare, you'll see that our scores generally match. Is there some unnecessary complexity to a percentage grading? Yes, maybe. Is the whole range of numbers from 1% to 59% a big waste? Sort of (though it's fun to rank just how bad some of the worst games are). What matters is that most people with some level of common sense can understand either rating system without any trouble whatsoever.

Which leads us to our second point: the scores are a cursory tool at best, and are not even remotely the most important part of a review. After all, some publications don't do any ranking whatsoever (see The Escapist's extremely popular Zero Punctuation video reviews as just one example). Some people see the scoring as a hindrance, and would rather just convey their thoughts using words. After all, "the pen is mightier."

If you're genuinely interested in seeing what your peers and "professional" critics think of this or that game, you sure as heck better be digesting the written (or spoken) portion of the review. We've all seen knee-jerk reactions from angry gamers on message boards: "they gave Twilight Princess an 8, that's blasphemy!" ... "Grand Theft Auto 4 is by no means a perfect game, why is it getting perfect scores?!" And, after some inane arguments and insults are thrown back and forth, some well-reasoned person goes back to the review and says "hey, I actually read the review, and here's what I learned." ... problem solved. Reasonable, civil dialogue triumphs over numerical rankings.

And that's because, in the business of critiquing, there is no such thing as objectivity. Sometimes, we can get a sense that something is "great," and some things are clearly pathetic (if the game is broken, it doesn't deserve a good score). You absolutely need to know why the reviewer handed the game a high or low score. The reviewer might cite specific examples of a scene in the game, or a specific gameplay mechanic, that really impressed (or irritated) the reviewer. Other reviewers may be indifferent to said scene/mechanic and focus on another aspect of the game. As for me personally, I tend to get really wrapped up in an RPG's story. If the game isn't designed to have an epic plot, I won't make a big deal of it; but if the game is story-heavy, I will put a lot of my emphasis in the review on the plot and judge it according to my own tastes, as well as my experiences with past RPGs.

So to all the would-be game critics out there, go ahead and rate the game however you want. Use stars, or percentages, or thumbs, or tomatoes, or heck, don't use anything at all! What really matters is the verbal communication. What did you like or dislike? Where did the game fail to meet expectations (either of fans or of the developers themselves)? What could be done to improve the game? These are the things that matter in a review, and they cannot be conveyed through numbers alone.

- Patrick Gann



Back