intfiction.org

My emphasis here is on the "very." That is to say, I used to view them as subjective, but still hold 5 stars up as a sort of "world record" rather than a personal "I really, really liked this game." Just three short years ago, I wrote on my blog: "A game would have to be nearly life-changing to score a 10. 10s I would only assign retrospectively to great classics." In practice, that meant I unintentionally low-balled every entry in the ADRIFT IntroComp.

Mr. Whyld also expressed a similar stance on ratings recently, saying: We also had people uncomfortable giving out low scores, for which Abbi Park developed a standardized disclaimer:Can't say much of this resulted in more reviews, though. Anyway, I don't know how instructive this info from the ADRIFT community is to IFDB, but I figure some of it ought to translate.

Hmm... 20 games written... 1 game "exists."

Well, in that case I don't find it puzzling at all.



I can think of one work of IF that literally changed my life (for the better, vastly) and I rate it three stars on the IFDB ... because it's good, you know? But only just. For me.



Heh. I see five stars as an exalted category, not a specific position. Besides, even if someone were to choose, for their own purposes, to reserve the five-star rating to mean their unique choice of "bestest ever," the IFDB allows us to change our ratings all we like ... so, anyone could choose to grant only a single five-star rating for his very-very-very favorite ... and then still leave room for that favorite to be unseated, as it were, by a later game.



Alas

_________________
Games, Fonts, and Font-Toys For Gamers - http://www.cumberlandgames.com

Yeah, this is pretty tricky. I tend to reserve 1 and 2 stars for things I think are not worth play or severely flawed in their present state, but the 3-5 range is vastly complicated.

The games I admire most are the ones that are both well-executed and trying to do something I think is worthwhile (where "worthwhile" might mean telling an awesome story, having something valuable to communicate about the human condition, breaking new ground in what the medium can do, presenting a really cool puzzle mechanic, et al). For very different reasons, I would consider "maybe make some change," "Make It Good," "The Baron," "Patanoir," and "Apocolocyntosis" all to be worthwhile in this sense, though I don't think them all equally well crafted. And they're pretty hard to compare to one another meaningfully at all, really.

While in theory I would like to have a system that graded craft and worthwhile-ness distinctly, rolling both into a single number is really hard. So sometimes I find myself giving 5s to games with gaping flaws because they've nonetheless got so much going for them, and sometimes I give a 4 to something that I thought was really solidly constructed but just not that significant; and sometimes it goes the other way and I penalize something conceptually awesome for its lack of craft followthrough. It's really hard to be rigorous about this, so I just have to hope it will all come out in the wash.

To complicate matters, I find in practice that I'm sometimes harsher on really good games than on fairly good ones. The only way I can explain this effect is to say that once a game passes a certain quality level, I am mentally rating it against all other awesome IF, rather than average-quality-this-year.

So, for instance, I gave a 4 to Blue Lacuna, which on one level is absurdly unfair because it is bigger, braver, more polished and generally more amazing than the vast majority of IF. On the other hand, there are things about the writing that I wish had seen a stronger edit; I think a lot of the potential of Progue is often unclear to the player on a single playthrough, and the game is long enough to discourage numerous playthroughs; there were all kinds of pacing problems in the midgame; and the piece as a whole had massive stretch marks from having grown so much during production. It could have been better at being what it was trying to be (though I'm not sure that could have happened without killing Aaron -- he'd put enough time in on the thing already, for sure). I totally understand how that can happen to a big project -- it's happened to some of mine -- so I'm all sympathy, but still. There's a lack of consistent vision, a lack of unity, that makes it a 4 when it's lined up with other great IF.

Finally there's the "spark of life" quality, which is not the same thing as being worthwhile, and is very very hard indeed to describe. But in essence, I find that there are some games that grab me with their creative spark, with the sense that the author was having fun or speaking from a place of true feeling. Sometimes goofy, pointless, badly made speed-IF has the spark of life; sometimes epic games three years in the making don't. If it's not there, I find the game a tiresome chore to play, no matter how carefully crafted it might be. But I also feel like it's not that useful to write a review saying so, because the complaint is so nebulous and also feels like I'm attacking the author directly as a creative person. But no spark, no 5, that's for sure.

The discussion about the relationship between numerical ratings and reviews comes up often on sites where long term reviewing is going on, and a frequent sum-up position is that the numbers make the most sense in the context of each review.

We can't know what invisible rules from their mental universe any person is applying to their own ratings (For instance, I've only given 5 stars to games which have percolated in my brain for ages, like Wishrbinger - is that fair? Probably not. Or that I almost never go above 3 stars for very short games) -- but if you look at the number in light of or just before the review content, then the sense (or nonsense) will quickly emerge.

So, re Ghalev's comment:



Do you find you really need that much consensus? Sometimes I can work out all I need cue-wise from the written part of a single two star review, depending on the writer. That is, I can get a good impression in one review of what's likely to interest me (or not) about the game.

So while a very high or low overall star rating can draw my eye, it's always the review content that does the business for me.

This does mean that old games which are often long on numerical ratings but without written reviews (plenty such games I've rated from memory to try to help IFDB, but not reviewed because the details aren't fresh in my mind, only my overall impressions of enjoyment or quality) often don't get a chance to draw me anew, unless I read written reviews of them elsewhere.

In my opinion, rating games in retrospect is a mistake.
Personally, I do it when they are still a warm body (or an agonizing one, meaning a game I can't get to finish is one I won't rate too much, eventually) and I still can feel the pain of killing them. If I did it later (maybe a lot of time later) I'd lose much of the emotion of being there in the first place. The overall feeling, the immersion.

In these very days I'm trying to evaluate games in a famous competition.


Also, as a personal note: I prefer stars to reviews when evaluating something. I understand that a review can give so much more insight (you can find some even in rookie and off-topic reviews like the ones of mine), but sometimes it all gets too bigger and precise and picky. I just wanna say "i liked it" or "i didn't like it" and to hell to all those explanations I've gotta give.

No; I apologize if I implied a need, specifically. Wasn't what I meant. Remember that I'm talking about getting a sense of community consensus, not about getting an idea about how much I'll personally like it or consider it good. The community and I have pretty different values and standards, but I do like to have an awareness of that territory



For me it's often just the tags :)

_________________
Games, Fonts, and Font-Toys For Gamers - http://www.cumberlandgames.com

Heh. I recently rated a game four stars, and said in my review it was probably worth two.

As a reviewer, I'd happily eschew the stars system if I could (however, seeing as it's there, I do make use of it. Every tool counts). As it is, if I end up rating something with more/less stars than I think are really worth it, I'll explain it in the review. And indeed, I expect the reader to take the review in consideration, not the star.

Hmm, reading these comments, it seems like it might be nice to split the IFDB rating system into a few categories of stars, like importance/notability, quality/craft/polish, and creativity/innovation. But that could be a lot of work and would trash/devalue the old 1-category ratings, unless you still left unitary ratings as an option for reviewers.

I think the scoring system is fine as it is, and any added complexity would simply discourage people from voting at all. As for things like importance/notability, without an understanding of the IF world in general you're never going to know whether a game is important or notable; we all have different ideas of what constitutes quality/craft/polish; and creativity/innovation? A game might be creative and innovative without being any good, or could lack kind of creativity and innovation but still be a great game.

Okay, this has me curious: what game are you talking about?

_________________
Before this I was thinking I would consider it a 6/10, but now? It's a solid 5.