Rating systems...opinions wanted

[sunteam_paul]

I'm mulling over changing the rating system for the updated PCE Software Bible. Marking out of 5 as it is seems alright, but I'm not completely satisfied with it. I've always believed that the text itself is the key to how good you think a game is and not the final rating, where 'middle' scores are always misinterpreted as meaning poor games. Ratings out of 100 or even 10 I think are next to useless; not only is it impossible to accurately rate something out of 100, but again anything below 75 or so is generally regarded as poor.

One idea is to have a simple rating of NO - MAYBE - YES represented by something like smily faces or thumbs up/down or somesuch. As most games would fall under the 'maybe' this would give a quick indication of what to avoid and what to go for.

Am I barking up the wrong tree if I simplify things? Do you think rating out of 3 or 5 is too restricting, and is 10 far too inaccurate?

[Turbo D]

I like the "out of 5" rating that you use and I've always felt your ratings were fair. I love how you use bonk meats as rating 

[SuperDeadite]

Personally I don't like ratings and never use such systems.  I think a simple text review is the best way to go, as it forces people to actually read your hard work and know what the game is actually about. 

Also it helps prevent people from directly comparing different games.  For example, lets say Dracula X gets a 5/5, and Cotton gets a 4/5, so Dracula X must be a better game then Cotton right?  Platformers and shooters should never be directly compared like that because they are totally different genres and have nothing in common besides being PC-Engine games.  Its like saying concrete bricks are better then tongue depressors.  They are both excellent building materials depending if you want to build a house for yourself or your G.I. Joes.

I hate reviews that say "Graphics look like game X, music sounds like game Y."  Such information is absolutely useless unless the reader has actually experienced game X and Y themselves.  Its much better to say "I like this game's music because...."   or "This game looks like crap because..."

If you want to use a rating system, the simple "Thumbs Up, or Thumbs Down" is the best.  The main focus of a review should always be is the game fun or not and nothing more.

Just my 2 cents, either way I've always loved your site. 

[sunteam_paul]

Personally I don't like ratings and never use such systems.  I think a simple text review is the best way to go, as it forces people to actually read your hard work and know what the game is actually about. 

This is true. However, if I had no ratings, then people would just keep asking for them. Thumbs up/down is a little extreme I think as there are games that are decent, but neither exceptional or poor. I would have a hard time chosing between the two, particularly on things like puzzle games (which I'm not a big fan of) without being unfair and unrepresentitive of how good the game actually is for its genre.

[esteban]

Personally I don't like ratings and never use such systems.  I think a simple text review is the best way to go, as it forces people to actually read your hard work and know what the game is actually about. 

Also it helps prevent people from directly comparing different games.  For example, lets say Dracula X gets a 5/5, and Cotton gets a 4/5, so Dracula X must be a better game then Cotton right?  Platformers and shooters should never be directly compared like that because they are totally different genres and have nothing in common besides being PC-Engine games.  Its like saying concrete bricks are better then tongue depressors.  They are both excellent building materials depending if you want to build a house for yourself or your G.I. Joes.

I hate reviews that say "Graphics look like game X, music sounds like game Y."  Such information is absolutely useless unless the reader has actually experienced game X and Y themselves.  Its much better to say "I like this game's music because...."   or "This game looks like crap because..."

If you want to use a rating system, the simple "Thumbs Up, or Thumbs Down" is the best.  The main focus of a review should always be is the game fun or not and nothing more.

Just my 2 cents, either way I've always loved your site. 

I totally agree. Rating systems fail because they really aren't a summary of the text (the content of the review)... unless you have a horribly written review.

Personally, though, I think that "Thumbs Up, Thumbs Down" is too constraining, so if you (Sunteam) were going to use a rating system, I think the "Yes, No, Maybe" ranking is the way to go. I honestly think that is the best way to use rankings, since I would put tons of games in "Maybe" and then explain (in the review) why the game might appeal to some niche audience despite the fact that it is universally panned by most folks (like China Warrior and Timeball, you bastards).

"Yes, no, maybe" converts to "buy, avoid, rent" with contemporary games, so I think it is a useful framework that folks can relate to.

[spenoza]

I do think moving away from numbers would help. I think the problem with people thinking 75 or less is a bad game comes from school system grades, where less than a C (70 or 75 or so) is "no credit". I think the best "rating" system would involve a scale of 1-5 but using words. Horrible - Poor - Fair - Good - Excellent. Fair is therefore a passing grade.

[esteban]

I do think moving away from numbers would help. I think the problem with people thinking 75 or less is a bad game comes from school system grades, where less than a C (70 or 75 or so) is "no credit". I think the best "rating" system would involve a scale of 1-5 but using words. Horrible - Poor - Fair - Good - Excellent. Fair is therefore a passing grade.

Hmmmm, I had never considered a "Horrible - Poor - Fair - Good - Excellent" scale before. That sounds pretty good, and much better than "5 stars" and other such poppycock.

Yes, poppycock is the word for the weekend .

[GUTS]

I like the school grading system way of rating, its quick and easy.  No need to re-invent the wheel, just go with what already works.

[Necromancer]

I do think moving away from numbers would help. I think the problem with people thinking 75 or less is a bad game comes from school system grades, where less than a C (70 or 75 or so) is "no credit". I think the best "rating" system would involve a scale of 1-5 but using words. Horrible - Poor - Fair - Good - Excellent. Fair is therefore a passing grade.

I like the 'Horrible, Poor, Fair, Good, Excellent' idea, but it needs to relate to the bonk meats - they're too damn cool to abandon.  Maybe rank 'em from Inedible to Delicious. 

[Black Tiger]

The reviews for my site (which are still currently hidden), score Virtual Console games out of 10, in increments of .5, like EGM does. I score regular Turbo/PCE games out of 100.

I also have thumbs up/down type system and describe 'who' would like to play each game.

I recommend most of the games with mediocre scores overall and explain why.

[Turbo D]

why must you hide it from me 

[CosMind]

i dig your concept of "no - maybe - yes", but would also add a final "must have" to the mix.

no - maybe - yes - must have

no = don't even bother
maybe = give it a try, you might enjoy it
yes = a definite play, possibly even a purchase
must have = a definitely must own

[Keranu]

Personally I prefer to avoid score numbers and such all together. A review is best "scored" from the text itself and adding a score number or something just seems to cause difficulty and contradictions with other game scores.

[spenoza]

I agree that numbers can be problematic because everyone interprets them according to their own scale, but at the same time, if the review is only in the text, it can be hard to extract an overall picture. I think there does need to be a one-work to one-sentence sum-up that covers, overall, whether the game is worth 2 beans or not.

[esteban]

I agree that numbers can be problematic because everyone interprets them according to their own scale, but at the same time, if the review is only in the text, it can be hard to extract an overall picture. I think there does need to be a one-work to one-sentence sum-up that covers, overall, whether the game is worth 2 beans or not.

Yeah, I think you're right. A pithy sentence or two would go a long way in helping people, accompanied by the short-hand score.