Text-to-speech and dictation for interactive fiction?

Has anyone had any success or ideas about how one would go about using text-to-speech and then dictation for standard parser-based or multiple choice interactive fiction files?

This is built into the OS on iOS and MacOS. (I haven’t tested interpreters on MacOS, though.)

A couple years ago an entire interpreter was made for Android that does speech recognition and output for parser-based Glulx and Z-machine games. Name is Incant. Here is a video of it in action: youtube.com/watch?v=D6i7c7j … e=youtu.be - I don’t think it was ever released on the Play Store (please correct me if you find it), but it is fully open source with flexible licensing.

I have found the code to be high quality and it hasn’t gotten much attention. It supports graphics and sound output - but needs some work. This video is from mid 2014, Google has improved the speech in Android a lot - so it would likely be better if run today on a new Android 7 device.

1 Like

I’d like to bump this thread if that’s okay. What libraries/systems are available for control of an IF game by voice?

I’m more interested in speech-to-text than text-to-speech, but any information on either is welcome.

I ran across StoryHarp on ifwiki. It looks like at least one version of it offered voice input. (No personal experience with it.)

1 Like

I think on Mac it’s possible to make almost any text spoken.

In fact, I just highlighted this message, right clicked and chose Speech>Start Speaking. I didn’t know that was even there.

3 Likes

I think on Mac it’s possible to make almost any text spoken.

And to dictate into pretty much any editable text field, with Edit > Start Dictation. I haven’t used it enough to say how reliable it is, though.

2 Likes

MacOS (and iOS) have a more systematic accessibility mode called “VoiceOver”. You can turn this on in system settings. The idea is that VoiceOver makes every UI control discoverable and readable using audio cues. App developers are supposed to test to make sure VoiceOver works for their app.

I know there have been some surveys of VoiceOver for MacOS IF interpreters, but I don’t remember where the most current results are.

(I know how to navigate iOS VoiceOver, because I’ve tested my iOS IF apps with it. The MacOS version is still pretty much a mystery to me.)

1 Like

After seeing this topic (and having an interest in this kind of thing), I played around with the built-in Windows speech-to-text feature for an hour or so. I found it pretty terrible. Very poor recognition compared to my iOS phone and Android television. It also seems to attempt translating dictation into idiomatic English, which is less helpful than one might guess. There may be a way to configure this, but I did not see one. Maybe it could be trained to work with document creation but IF feels like a reach. By contrast, Google docs speech-to-text recognized common IF commands on the same equipment, but sadly it is internal to their docs.

I’m sure there are apps out there, but this really ought to work at the OS level. The apps I found weren’t cheap and were also largely concerned with document creation rather than general input.

1 Like