intfiction.org

A few talking points about ratings on the IFDB

1. The IFDB is pretty neat for a whole load of reasons. Chalk me up as a fan. But the question as to what rationale to employ when rating games troubles me.* At first, I thought of converting it in my head to a 100 point scale- with '3' being between 60-79%, so pretty good. But given that there is no zeroth rating, '3' is the middle star, and looks very much like 50%. There's a big difference between saying 'this game almost deserves 80%' to saying 'this game is around 50%'.

2. By my count there are 1670 unrated games out of 4064, 41% of the games. As a community tool, there's no use complaining that somebody should do something about this, but I'm wondering whether people think this is an issue, and whether there'll always be so many unrated games or whether people think that over time it will tend towards greater completion.

3. The ratings are obviously ratings about the whole game experience, but there are lots of different possible games. Do you rate a game based on how good it is as a game, or as a game of its type or even how good it was at conveying the experience the author intended to afford you?

4. A lot of the time I don't really want to give a rating, but I feel like I should because it helps improve the database as a ratings engine. A review is almost always more useful, but necessarily more time consuming to write or read. Perhaps I should tag more games- as this aids the IFDB qua database without appending a possibly misleading number onto a game.

*Well, 'troubles' is probably too strong. I'm certainly not lying awake at night over it.

Games with no ratings look sad and unloved. Even one star is better than none. Just go with your gut and pick a star.

For me, the stars are the single most useful metric for filtering games. I view it as a scale ranging from "Avoid" to "Strongly Recommend" and don't get too caught up in the specifics.

I tend to avoid reviews for games rated 4 & 5 stars, since I will likely play those games and form my own opinions. For games rated 1 & 2 stars, the reviews can offer a lot of valuable game design advice, and are worth reading even when the game may not be worth my time. 3 star games are a toss-up; if the blurb is compelling, they go on the wishlist; otherwise I spoil them with reviews until / unless I am persuaded to take a closer look.

So yes, reviews are nice, but ratings are also nice. With so many games to choose from, it's very helpful to have a metric that tells me at a glance how to approach a given title.

If you'd filter out all the obscure proprietary 80s and 90s games that often don't even have a download link, I think that percentage will drop considerably. (That's just a hunch based on my own observations, though.)

There are 3657 downloadable games and 1484 of them have no ratings; that's about 41%. If you limit the search to games published in 2000 or later the numbers are 1950 and 543 which makes 28%.

_________________
Vorple UI library — Beta testing site — Blog

A quick summation of a longer reply I was making! Fast figures Juhana! So a little under a third of all games are potentially worth playing and are playable but are unrated.


Yeah, maybe that's the best approach. I feel that I'm somewhat susceptible to ratings-creep. My earlier ratings are harsh, but now I've ended up giving out four stars all over the place, when on reflection I often liked the new four star games less than the old three star games!

Case in point: I've given four stars to some of the new TMBG one-move Fingertips games when I've thought they're pretty awesome for what they're doing, but I only gave three stars to Patanoir (mostly due to polish) when I'd take Patanoir over any of the Fingertips games any given day.

I think this is one of the things that IFDB gets right, or at least more right than the IF Comp style of scoring games relative to the set, such that you end up with ranked winners and losers.

It's perfectly fine in my book to give four or five stars to an outstanding one-move or one-room game, and three stars to a deeper, more satisfying game that nevertheless suffers from implementation issues. All you're really judging is whether time with this game will be time well spent; the longer the game, the harder it is to sustain the ratio of awesomeness to minutes played.

For me, a game doesn't really "exist" yet until it has at least, say, 9 or 10 ratings on the IFDB. Or at least, I don't imagine I can guess at any kind of consensus for how good it might be until that point.

Sadly, this means that there are even chunks of the Infocom catalogue that "don't exist" yet. On the other hand, I guess I have them to look forward to



Amen to all of that. I rate a game on how well I feel it succeeded as itself, and how much it rocked my world for the time I spent with it.

For me the stars are pretty simple:
1: Bad or Dull Game*
2: Yeahfine/Decent/Okay Game
3: Good Game
4: Very Good Game
5: Great Game

* If zero stars were possible, zero would be reserved for dull, which is a shade worse than bad in my book.

This means there's only one shade of "I don't recommend this," but that's okay, because how many shades of "I don't recommend this" do I really need?

_________________
Games, Fonts, and Font-Toys For Gamers - http://www.cumberlandgames.com

When I don't rate games that I've played, it can be for a number of reasons, which I don't always apply consistently:

• I played the game a long long time ago, and I can't remember it all that well, or if I can I don't really think that 2012-me would wholly agree with a self-important 17-year-old.
• The score's associated with a review that's very equivocal or has a lot of reservations. (For instance, one's appreciation of Encyclopedia Fuckme and the Case of the Vanishing Entree is going to depend so heavily on whether you find it sexy, revolting, triggery or none of the above, that rating it independently of that very personal reaction is going to be somewhat pointless.)
• I'm trying not to be a dick to a first-time author who seems to have made a good-faith effort but still produced something wretchedly bad.
• A five-star rating system feels too crude to summarise my feelings about the game, and any value I pick seems sort of arbitrary.

Not all of these are necessarily good reasons all of the time, but they have an effect. (And now you've guilted me into going through the games by year and seeing what I've played but not rated.)

We were actually just talking about this on the ADRIFT Forum. By the way... might wanna go through the list of unrated games and find which ones are written in ADRIFT. It's possible they have reviews posted elsewhere, be it on the ADRIFT Forum or Adventures page.

I tend to view the star system as a very subjective thing nowadays, keeping in mind that my opinion is likely to be just one of many. That way, I'm not too worried about my reviews being too extreme... if they are, someone will balance them out later, probably (unless it's an ADRIFT game). So my barrier for a three star rating is just my own personal enjoyment, like this:

1 star = I just don't like it. Period.
2 stars = I could like the game, perhaps with minor changes.
3 stars = I liked it, but might feel hesitant about recommending it.
4 stars = I liked it and recommend it.
5 stars = A game I think everyone should play, whether they end up liking it or not.

Personal enjoyment is an easy metric to use, and removes a lot of the anxiety I had previously about rating games.

It's fascinating to me that anyone could view it any other way. :/

_________________
Games, Fonts, and Font-Toys For Gamers - http://www.cumberlandgames.com