musings on scoring

As I began playing/reading, I thought a bit about how scoring might work.

Is it a simple average of votes or is there any weighting applied?

Surely there are some judges that strongly prefer choice or parser games only, and vote only for those. Therefore their votes will not be in the context of those unplayed games. Similarly, many judges will not try all entries. Accordingly, their votes will not be in the context of the unplayed entries.

One approach to correct for this would be to apply a weighting factor that is a function of total number of entries rated, the diversity of those entries, and the variance of one’s scores. The highest factor would be for the judge who rates all entries with a normal distribution centered on 5 (the ideal judge).

For that matter, why are choice and parser games not judged separately? It is counter-intuitive to me that they are compared side by side. In terms of the “interactivity” of the fiction, the expectations a reader has, and what is possible with the medium, there is a vast chasm between these two. Yes, there are some interesting bridges between the two sides, but in my experience most flavors of games are comfortably in one camp or the other.

Games’ scores are averaged by the number of votes they receive. If you find a game you cannot play for technical or personal reasons you should not give it a one only because of that - the rules state you should not vote on games that you do not play. I have been given a hate-one score in the past because the reviewer didn’t like the format my game was in, and wouldn’t play it, and it’s completely uncool.

Games’ scores also do not affect each other. If one game gets one vote of 7 and another gets ten votes of 7, they both rank 7. (I believe in a tie like that, they would consider the game with more votes as ahead of a game with fewer.)

So essentially, judge each game on its own merits, and not against the others. I do know some judges will give initial scores then go back and rebalance a bit, say if they found they gave out too many votes of 6 so that the spread makes more sense overall.

If you prefer one type of game to another, just play what you like and leave others alone. In the past I believe obvious troll voting (giving every choice game a 1 for example) has been disqualified, so only give real votes for the games that you want to play. There are so many games. Don’t feel forced to judge something you would rather not.

Scoring is a simple average of valid votes for each game. Yes, it’s the simplest possible system, but my feeling is that making it more complicated would do more to amplify arguments than it would to improve the comp results. (Arguments about voting systems can go on forever.)

The rules suggest (ifcomp.org/about/judging) that you use the full range from 1 to 10. That encourages, without mandating, a common mean and deviation.

It is the concensus of the people involved that both kinds of games are IF enough to be in IFComp. Therefore, they can be compared to each other as IF.

Also there’s the games that wouldn’t fall into either category, like AlethiCorp.

Choice voting vs. parser voting — really? People should be able to vote for whatever they want to, for whatever reason — that’s whole f-ing point of voting. However, the voter should be informed. If you’re voting on games you didn’t play, then you broke the rules.

As far as making separate categories: Separate but equal — we tried that in my country for over five decades. While I’m certainly not trying to equate this silly comp to the civil rights movement; I’m just saying that anyone who thinks that separating groups is helpful is almost always wrong.

SkinnyMike, while I don’t disagree that IFComp should not separate choice-based games and parser-based games into different categories, I have trouble with your metaphor.

It does not follow that, because all people should have the same rights and share those rights collectively as human beings, all IF games should be judged in the same category.

Separating groups can sometimes be helpful. We could just say that IF is media, but so are other things. Maybe the organizers should let movies be entered to the Comp. Or we could say that media is a thing, but so are dinosaurs. Maybe the organizers should allow dinosaurs to be entered into the Comp.

I realize that I am taking your logic to the extreme, and I do not mean to offend, but I couldn’t help but to point out that separation of categories in a competition can be a good thing sometimes and is in no way similar to the metaphor that you put forth in your post.

Hey Matthew,

I appreciate your comments on my post. Although it was (I thought obviously) intended to be hyperbolic, I realize it probably went too far. My basic point — which I apparently failed to get across — was that we already have a system of separation: it’s called voting. With the number of works we have this year, the fact is that some people will choose to play games created in a form or even a system that they prefer, while ignoring the others. Here in the States, we elect presidents in a similar way. (Great. I just did it again, didn’t I?)

Anyway, I’m not in any way offended by you voicing your opinion; actually, I’m just happy that someone commented on my post. (I was going to write that I seem to be persona non grata around here now, but that would sound paranoid, right?) It seems that we actually agree on the topic at hand, but you weren’t particularly fond of the way I put it (which I think is more of an allegory than a metaphor — but we can discuss that later). So for now, we can agree to agree in different ways.

Thanks for the dialog,

On separating choice & parser, I don’t think it’s necessary, yet I also don’t see how it’s a terrible or offensive idea.

AlethiCorp feels like a choice game to me - the unusual interface and the fact that at some point you have to enter a name/string (IIRC) don’t really change that.

I agree there games which blur the parser-choice separation, perhaps by using both systems in different parts (though I don’t consider parser games with conversation menus an example of this), but don’t think AlethiCorp was one.

By some definition all videogames could be ‘interactive fiction’. It’s fiction you interactw ith.
Sure, the amount of textual input & output is usually far smaller than what we generally call IF, but that’s a matter of quantity rather than quality.
Yet I don’t think many here would like FPS or pure strategy games in the IfComp.

There are specific requirements to be a presidential candidate though. Of course some are very controversial, such as being a ‘natural-born’ citizen.
That and there can only be one president of the US, while a hypothetical IF with separate parser & choice categories would have two 1st place winners.

As for separate but equal, isn’t it one thing to apply it to society as whole, another to competitions? The main problem being that one of the groups gets a much worse treatment (so it’s not actually equal).

That wouldn’t necessarily happen for a game competition with separate categories where everyone has access to the exact same prize pool.

I remember that in the past parser-only donations to the prize pool were offered (they wanted to encourage this kind of game), but only accepted after a choice-only equivalent was provided (by someone else, IIRC) too.

Of course, if parser & choice were completely separate then each prize would need to be duplicated which is simply not feasible.

TL,DR: Maybe having parser & choice categories wouldn’t be so terrible or controversial if we could ensure the same rewards for author, but we can’t.

Leaving aside this, one thing that I’d like to bring up (again) on voting is that the example voting guidelines suggest trying to be somewhat objective - judge a game compared to its importance to IF as a whole and whether you would recommend it or not - but it is considered perfectly acceptable to vote subjectively and ‘normalize’ voting by giving 10 to the game you consider the best, even if you do not believe that “People interested in the form will be talking about and studying this game for years to come.”

I think it might be a good thing to mention this on the site, because many people might not realize and want to vote that way (to make their votes matter more both on games they like & dislike), but think they might be breaking the rules or doing something unproper.

There was a ParserComp last year, and I suspect there will be another one soon. As far as I know there hasn’t been a comp specifically for choice/hypertext games (closest I can think of is the Windhammer Prize for gamebooks, and the Twiny Jam which wasn’t a comp), but I wouldn’t be surprised if someone decided to run one at some point. Having separate comps seems to make more sense than a comp with categories, because they’d have more unified audiences. And given these outlets, having one big categoryless comp makes sense to me too, like a World Series of IF (there’s probably a better metaphor but I am blanking…)