intfiction.org

The Interactive Fiction Community Forum
It is currently Fri Nov 24, 2017 7:58 am

All times are UTC - 6 hours [ DST ]




Post new topic Reply to topic  [ 40 posts ]  Go to page 1, 2, 3, 4  Next
Author Message
PostPosted: Wed Mar 30, 2016 7:07 am 
Offline

Joined: Wed Mar 30, 2016 6:23 am
Posts: 29
Hello :)

I'm a computer science and artificial intelligence student and I'm interested in creating an AI system capable of good natural language understanding and representation. What my advisor and I think is that IF could be a great source of data and learning material for such system. Why? IF stories or games usually include some sort of feedback to the player which is something that could possibly help the system learn useful representations of natural language. The dream scenario would be to let the system train on a set of IF games and then test it on games that it has never seen before and see it perform well (or at least human-like in some ways).

There are already two papers that do something similar [1][2], except they simply test their systems on the same games they have been trained on – which makes very little sense to me, as this makes the task virtually trivial.


Now I have a couple of questions about the availability of IF games with specific features which might help (they might or might not be necessary) us create the system described above:

1. Are there any games (or perhaps servers) with user input data available? More specifically, it would be extremely useful to know how exactly users play IF games (simply put, see their input and actions) or in other words – how well they play. Using this data, it might be possible to bootstrap the system to behave at least partially human-like. Even more importantly, it would be great to have such data for evaluation, so that we could see how the AI system's peformance compares to that of human players.

2. Some (perhaps most?) games have different endings. In that case, the endings can usually be associated with some kind of reward. The problem is that the reward is specified in the form of words, perhaps a couple of sentences. To evaluate the human or AI players, though, it would be very much needed to assign numeric values to the different endings in different games. Are there any (sets of) games that actually have their endings annotated with numbers (score)?

3. Are there any random games? The randomness can either come from: A. a random description of the same inner world state (e.g. 'You see a bird on the window' and 'There is a bird on the window'); or B. random transitions between the world states (one action from a given world state can lead to two or more different world states, based on some random distribution).

Ideally, I'd like to have a set of games with all the properties above but I realise such dataset (or in this case, such collection of IF games) may not be available at all. At any rate, any help or feedback is greatly appreciated!

As a final note, I do know about MUD games that even offer interaction with human players. The problem is that the system needs to have a very fast simulation of the environment (in the case of IF games, perhaps something like a simple HTML page with hyperlinks) which is something that the server-based MUD games don't really allow.

Thank you very much :)


Top
 Profile Send private message  
Reply with quote  
PostPosted: Wed Mar 30, 2016 11:38 am 
Offline

Joined: Fri Oct 18, 2013 10:13 am
Posts: 2470
Location: The Midwest
Adventure and the original Zork come to mind. They have a numerical scoring system and random elements through the wandering NPCs. I don't know how much input data is available, but the source code in a variety of languages is available on the Archive.

_________________
Daniel Stelzer


Top
 Profile Send private message  
Reply with quote  
PostPosted: Wed Mar 30, 2016 11:56 am 
Offline

Joined: Thu Oct 30, 2014 12:10 pm
Posts: 613
myke wrote:
1. Are there any games (or perhaps servers) with user input data available? More specifically, it would be extremely useful to know how exactly users play IF games (simply put, see their input and actions) or in other words – how well they play. Using this data, it might be possible to bootstrap the system to behave at least partially human-like. Even more importantly, it would be great to have such data for evaluation, so that we could see how the AI system's peformance compares to that of human players.


There are ClubFloyd transcripts for lots of games here: http://allthingsjacq.com/interactive_fi ... #clubfloyd


Top
 Profile Send private message  
Reply with quote  
PostPosted: Wed Mar 30, 2016 12:03 pm 
Offline

Joined: Tue Jul 28, 2015 1:05 pm
Posts: 983
Ifcomp saves users transcripts, with dozens of transcripts per game. Individual authors may have downloaded their transcripts, but I don't know if they are still kept by the organizers.

_________________
-My IFDB name is Mathbrush.

Anyone can make interactive fiction; if you've made a game and need a review on IFDB, let me know!


Top
 Profile Send private message  
Reply with quote  
PostPosted: Thu Mar 31, 2016 5:32 am 
Offline
User avatar

Joined: Sat Dec 22, 2007 6:00 pm
Posts: 687
Location: Western Australia
myke wrote:
1. Are there any games (or perhaps servers) with user input data available? More specifically, it would be extremely useful to know how exactly users play IF games (simply put, see their input and actions) or in other words – how well they play. Using this data, it might be possible to bootstrap the system to behave at least partially human-like. Even more importantly, it would be great to have such data for evaluation, so that we could see how the AI system's peformance compares to that of human players.

Aaron Reed has collected and analysed player transcripts for a couple of his games, Whom the Telling Changed and Blue Lacuna. Neither game is really standard parser IF, though.

_________________
Emily Boegheim


Top
 Profile Send private message  
Reply with quote  
PostPosted: Thu Mar 31, 2016 8:18 am 
Offline
User avatar

Joined: Thu Aug 20, 2015 4:35 pm
Posts: 94
myke wrote:
Hello :)

3. Are there any random games? The randomness can either come from: A. a random description of the same inner world state (e.g. 'You see a bird on the window' and 'There is a bird on the window'); or B. random transitions between the world states (one action from a given world state can lead to two or more different world states, based on some random distribution).


While I agree that the Zork games are excellent exemplars for playability and scoreability, I don't remember much randomness in them, other than the behaviour of the thief, grues etc. which aren't what I think you mean. However, more recent games (those written in Inform 7, for example) often have facilities for random speech.

It occurs to me that you might answer your objectives by learning an if creation language (not too difficult, I promise you) and building games ideally suited to the training function you envisage. Then you could test your trained automaton on a wider selection of games.

_________________
____________________________________________________
http://www.amazonsystems.co.uk/data/textadv.htm


Top
 Profile Send private message  
Reply with quote  
PostPosted: Wed Apr 13, 2016 11:02 am 
Offline

Joined: Wed Mar 30, 2016 6:23 am
Posts: 29
Thanks for the responses!

Draconis wrote:
Adventure and the original Zork come to mind. They have a numerical scoring system and random elements through the wandering NPCs. I don't know how much input data is available, but the source code in a variety of languages is available on the Archive.

Thank you – I found Zork on the Archive, and although it does have the 'I7 source available' tag, I really couldn't find an I7 source in the Downloads. I also found a sourceforge project from 2008 but the source file won't compile in Inform 7 (I tried fixing the bugs that probably were caused by the difference between I7 and previous versions but gave up after about half an hour – more issues were still popping up). Does anyone know about an I7-compatible version of Zork's source, please?

bg wrote:
There are ClubFloyd transcripts for lots of games here: http://allthingsjacq.com/interactive_fi ... #clubfloyd

craiglocke wrote:
Ifcomp saves users transcripts, with dozens of transcripts per game. Individual authors may have downloaded their transcripts, but I don't know if they are still kept by the organizers.

Thanks, this looks really promising :) Though the main problem is finding the 'correct' games and only then look for the user traces for those games. Do you perhaps know about any (concrete) games submitted to the IfComp that would fulfill my requirements (i.e. preferably large worlds with a scoring system and random elements (ideally random transitions between states or at least partially randomly generated world)? Not all of these are necessary, but the more, the better (the least needed requirement is probably the scoring system, since one could probably add that to the game fairly easily).

Emerald wrote:
Aaron Reed has collected and analysed player transcripts for a couple of his games, Whom the Telling Changed and Blue Lacuna. Neither game is really standard parser IF, though.

Thank you – this looks very interesting but I have yet to think about how these two games could fit into the chosen frame.

gil wrote:
While I agree that the Zork games are excellent exemplars for playability and scoreability, I don't remember much randomness in them, other than the behaviour of the thief, grues etc. which aren't what I think you mean. However, more recent games (those written in Inform 7, for example) often have facilities for random speech.

You're absolutely right about the randomness. I was thinking, though, that it might be possible to add a random arrangement of the game world to Zork (basically randomly generating the map while keeping the original transitions). Not sure if it's really possible, though, I'll have to look at I7 if I get a working version of Zork.
Also, could you recommend some of the random speech games, please? Or even better – is there any way to look for games with such features other than asking others or simply trying to play all of them? :) (it would be great if the Archive had something like a 'random' tag).

gil wrote:
It occurs to me that you might answer your objectives by learning an if creation language (not too difficult, I promise you) and building games ideally suited to the training function you envisage. Then you could test your trained automaton on a wider selection of games.

I agree this would be the best solution in an ideal world, but creating such dataset of games would take too much time. But of course, if I won't be able to find any suitable (set of) game(s), this is what I'll have to do :)


Top
 Profile Send private message  
Reply with quote  
PostPosted: Wed Apr 13, 2016 1:04 pm 
Offline

Joined: Thu Oct 30, 2014 12:10 pm
Posts: 613
myke wrote:
Thanks, this looks really promising :) Though the main problem is finding the 'correct' games and only then look for the user traces for those games. Do you perhaps know about any (concrete) games submitted to the IfComp that would fulfill my requirements (i.e. preferably large worlds with a scoring system and random elements (ideally random transitions between states or at least partially randomly generated world)? Not all of these are necessary, but the more, the better (the least needed requirement is probably the scoring system, since one could probably add that to the game fairly easily).


It's hard for me to think of any game that meets all of your requirements.

I don't think I've ever played an IF with numbered endings. I'm not sure if any exist. I'm not sure even unnumbered multiple endings are especially common in parser IF.

Untold Riches from IFComp 2015 has randomly generated text passages that appear throughout the story. And there's a ClubFloyd transcript for it. It's not that big a world though. And as far as I know there is only one ending. I don't remember if there's scoring.

Lobster Bucket is a small Shufflecomp game with a randomly generated map, but no transcripts that I know of.

The Dreamhold has a score, I think (a point for each mask?) and at least one random part of the map. But no transcripts that I know of.

If you want scoring, you might have better luck with older games. At one point I believe Inform, for instance, switched from making scoring on by default to making it off by default, because it wasn't as popular to use scoring anymore.

The only way I can think of for figuring out if a game has random elements without asking or playing is to search the source code for the word "random" (if it's in inform 7). In inform 7, the relevant phrases will be "at random" or "if a random chance of X in Y succeeds."

If you're planning on making changes to the game, does that mean you're limited to games with the source code public?


Top
 Profile Send private message  
Reply with quote  
PostPosted: Wed Apr 13, 2016 1:17 pm 
Offline

Joined: Thu Oct 30, 2014 12:10 pm
Posts: 613
Another idea: maybe look at the source code some of Andrew Schultz's games? I believe he sometimes publishes source code, and I believe some have been played by Club Floyd (for instance, Threediopolis), and I think he might like to use random cycling text? And his wordplay games might use scoring. I can't remember.

Oh, wait, I know. Check out Six by Wade Clarke. I think it has random elements, scoring, and a ClubFloyd transcript. And it was in IFComp 2011, so maybe there are IFComp transcripts. And the source is public.


Top
 Profile Send private message  
Reply with quote  
PostPosted: Wed Apr 13, 2016 1:43 pm 
Offline

Joined: Wed Mar 30, 2016 6:23 am
Posts: 29
Thank you very much, bg :)

It is not that important for the game to have multiple endings as long as there is another metric that can be used to evaluate player's performance (e.g. number of steps, occurence of 'good' states, reactions of NPCs, etc.). But of course, the existence (and 'quality') of multiple endings is one of the most natural metrics.

I'll look at the games you've suggested, greatly appreciated.

bg wrote:
If you're planning on making changes to the game, does that mean you're limited to games with the source code public?

It might or might not be that case – it depends on what changes would be needed. For example, only adding something like scoring or evaluation of player's performance could be done externally, without actually modifying the game.

The most important feature I'm looking for in the game(s) is probably the following: in order for the AI to be interesting, it should be able to generalise (for example, learn by playing one set of IF games and then succesfully play another set games it has never seen before). In the case of one game (so far it seems that it would be much easier to take one complex game, something similar to Zork), this would correspond to not being able to visit all possible states of the one game during learning – and then, during testing, whenever it would encounter a state it hasn't seen before, it should still work well.

So that's why random description of states or random states or random transitions between them are important – that way we'd be sure that the AI can't brute-force its way through the game by simply remembering it all. These features would make the game either literally or at least virtually (so big that it can't all be seen in reasonable time) infinite and one could then prove that the AI is actually learning something meaningful.


Top
 Profile Send private message  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 40 posts ]  Go to page 1, 2, 3, 4  Next

All times are UTC - 6 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group