intfiction.org

The Interactive Fiction Community Forum
It is currently Sat Jan 19, 2019 5:05 pm

All times are UTC - 6 hours [ DST ]




Post new topic Reply to topic  [ 24 posts ]  Go to page Previous  1, 2, 3  Next
Author Message
PostPosted: Mon Nov 05, 2018 6:28 pm 
Offline

Joined: Sun Apr 18, 2010 3:58 pm
Posts: 851
I had no idea that Gargoyle had a TTS feature. It's not a traditional approach, but I do agree that it works better than macOS VoiceOver on any of the interpreters I tried.

The first problem I noticed was that if the first page of a game overflows the page, it will have a "-- more --" page break on screen, but the TTS engine will just keep reading on past that point. As a result, when it comes time for the player to type, input is ignored until you use the space bar to scroll down to the end. This would be very confusing to users who can't read the screen.

The next problem is that there's no way to go back and re-read the text. VoiceOver supports features allowing you to skip around the text, paragraph by paragraph, sentence by sentence, word by word. TTS engines often mispronounce words, and it helps to navigate to that word and have the engine spell the word aloud for you. None of that can work here, because the app is reading the text itself.

The next problem is that there's no preference to control the word rate of the spoken text. Many (most??) VI users configure the engine to talk much faster than the normal, default speaking rate. With practice listening to a particular engine, you can get good at understanding the engine at breathtakingly fast speeds, so speed control is important. (And if you miss a bit, you'll want to be able to rewind and replay.) Again, this all comes built-in with macOS VoiceOver.

_________________
At Choice of Games, we sell long-form choice-based interactive fiction games. We're looking for writers, paid in advance.


Top
 Profile Send private message  
Reply with quote  
PostPosted: Mon Nov 05, 2018 7:04 pm 
Offline

Joined: Wed Nov 12, 2014 5:20 pm
Posts: 215
At least on Mac, the text-to-speech in Gargoyle will honor the speed setting of speech in the system preferences. The other things you mention will be harder to fix.


Top
 Profile Send private message  
Reply with quote  
PostPosted: Mon Nov 05, 2018 7:17 pm 
Offline

Joined: Wed Nov 12, 2014 5:20 pm
Posts: 215
The Flexible Windows issue seems to be gone with this version of Spatterlight. I can't reproduce it anyway.


Top
 Profile Send private message  
Reply with quote  
PostPosted: Mon Nov 05, 2018 7:33 pm 
Offline
User avatar

Joined: Wed Oct 14, 2009 4:02 am
Posts: 2579
dfabulich wrote:
Lectrote is only slightly better in this sense: it uses an HTML renderer, so it can tell VoiceOver about individual paragraphs. When you type a command, you remain focused on the command area, and you have to navigate backward through the text to find the new text and read that. But at least on Lectrote it is possible to navigate paragraph by paragraph. (Lectrote does mark the transcript window as a live region, but macOS VoiceOver doesn't appear to notice.)


I wonder if all that would need to be done to improve this would be to stop GlkOte from manually focusing the input element (with an option, the current code is probably best for non screenreader users.)

GlkOte issue for further discussion on this.


Top
 Profile Send private message  
Reply with quote  
PostPosted: Mon Nov 05, 2018 9:57 pm 
Offline

Joined: Sun Apr 18, 2010 3:58 pm
Posts: 851
Angstsmurf wrote:
At least on Mac, the text-to-speech in Gargoyle will honor the speed setting of speech in the system preferences.


Interesting. I think that would be worth a comment in the garglk.ini file. (Note that it does not honor the VoiceOver speed setting; it honors the "Speech" speed setting, which is separate, and is capped at a pretty slow speed relative to what advanced VI users would prefer.)

_________________
At Choice of Games, we sell long-form choice-based interactive fiction games. We're looking for writers, paid in advance.


Top
 Profile Send private message  
Reply with quote  
PostPosted: Mon Nov 05, 2018 10:15 pm 
Offline

Joined: Wed Nov 12, 2014 5:20 pm
Posts: 215
Quote:
I don't think source code is available, but take a look at the Zoom interpreter by Andrew Hunter. Aside from command-line version of frotz, it's the best Z-Machine interpreter around.

I don't know if I'm missing something, but when testing Zoom 1.1.5 on OS X 10.7 with VoiceOver, it seems to have the same issues as Spatterlight: when the game window becomes first responder all text in it will be read all the way from the start, including any scrolled off-screen. It will also speak any entered commands, but none of the new text printed after pressing enter. At least not until navigating away from the window and then back to it again, when it will once again read the text all the way from the start.

However, Zoom has custom commands such as "Speak previous move" to compensate for this, but it still seems like a bit of a kludge.

Quote:
Apple has extensive documentation. I've never looked at the MacOS version of VoiceOver, but the iOS version was fairly straightforward.

Well, Apple has plenty of documentation for its latest APIs, but if you want to write something that runs on 10.7 you are out of luck. There was an all-new accessibility API introduced in 10.10, and Apple seems to have been busy erasing any traces of the old way of doing things. Even the in-app help of Xcode 4, which purports to be the "OS X 10.8 Doc Set," will only show the latest online docs. But I'll keep googling I guess.


Last edited by Angstsmurf on Mon Nov 05, 2018 10:20 pm, edited 1 time in total.

Top
 Profile Send private message  
Reply with quote  
PostPosted: Mon Nov 05, 2018 10:18 pm 
Offline

Joined: Sun Apr 18, 2010 3:58 pm
Posts: 851
Hey, uh, the scrollbar isn't working in Spatterlight 0.5.5b. I can't scroll back with trackpad, and I can't scroll forward either (but luckily spacebar does work).

_________________
At Choice of Games, we sell long-form choice-based interactive fiction games. We're looking for writers, paid in advance.


Top
 Profile Send private message  
Reply with quote  
PostPosted: Mon Nov 05, 2018 10:22 pm 
Offline

Joined: Sun Apr 18, 2010 3:58 pm
Posts: 851
FWIW, I have reason to think that there may not be any good online examples of making an IF-like application work with macOS VoiceOver. I ranted about this last year; I'll copy and paste the rant here for your convenience.

The traditional IF UI is a commonplace stdio application, but stdio is totally unheard of in modern UI development.

Rant: show
Consider this bog standard IF transcript:

Quote:
At End Of Road
You are standing at the end of a road before a small brick building. Around you is a forest. A small stream flows out of the building and down a gully.

>east

Inside Building
You are inside a building, a well house for a large spring.

There are some keys on the ground here.

There is tasty food here.

There is a shiny brass lamp nearby.

There is an empty bottle here.

>examine lamp
It is a shiny brass lamp. It is not currently lit.

>get lamp
Taken.


The feeling of play is conversational. You might imagine that the game is playing the role of a Dungeon Master in Dungeons and Dragons, where the player and game are taking turns speaking.

If I were coding a "conversational" game today, I'd probably take examples from chat-based apps like Slack, where there's a fixed input box at the bottom of the screen, and a transcript of messages in the main part of the window.

But that is not at all how IF games work (not how they have ever worked historically). Instead of having a separate "input box" into which you type commands, there's just one main window area, the "transcript." The game prints messages into the transcript, then prints a prompt symbol (the ">" symbol) and lets the user type directly into the transcript.

To extend my Dungeons and Dragons metaphor, you might imagine the Dungeon Master writing something on a piece of paper, then handing that same piece of paper over to the player, who would write something at the bottom and then hand the paper back to the Dungeon Master, taking turns writing on the page.

This can be confusing to screen readers. One single rectangle is acting as both output and input. Inform/Quixe has handled this by marking the entire transcript area with "aria-live=polite," meaning that any new text that appears in the window is automatically read aloud.

But the bigger problem has to do with the "focus" of the screen reader.

When it's time for the user to type, the last line is an invisible input box, where the user types. When the user hits enter, the input box is removed and replaced with text that the user typed. Then the game says some response message, and finally a new invisible input box appears at the bottom of the screen.

Removing the currently focused UI element is very bad for screen readers. The screen reader software tries to guess where to focus next. It might try to focus the user on the top of the document (the very first line at the very beginning of the game), or it might focus on the top of the screen (probably a random moment a few turns back).

We can try to focus on the new input box, but focusing that box may interrupt the live ARIA reading of the game's message.

An obvious thing to do here would be to abandon the idea that the user should type directly into the transcript, and instead offer the user a fixed box to type in, at the bottom of the screen. That would probably work, but it would break some existing games.

For example, a number of games like to provide the user with a custom prompt inline with the text. "Please enter the secret code: " That may not make sense when the input box appears at the bottom of the screen.

A few games even rely on the fact that the prompt is in the transcript to play games with the player. For example, "Taco Fiction" prints a fake prompt ">" into the transcript and then waits for the player to type any key. No matter what the user types, the game inserts a character of the author's choosing, "forcing" players to type what the author wants them to type, letter by letter. I guess it's more of a practical joke than a serious UI consideration, but needless to say that joke would not work with a fully separate input box.

(And what about menus?)


FWIW, I just tried VoiceOver with macOS Terminal, and that works fine, but the source isn't available, and I think it may have special-case treatment from Apple. I tried it with iTerm 2 (for which source is available) and, surprise surprise, it had the same kinds of bugs as other IF applications.

_________________
At Choice of Games, we sell long-form choice-based interactive fiction games. We're looking for writers, paid in advance.


Top
 Profile Send private message  
Reply with quote  
PostPosted: Mon Nov 05, 2018 10:41 pm 
Offline

Joined: Wed Nov 12, 2014 5:20 pm
Posts: 215
Quote:
Hey, uh, the scrollbar isn't working in Spatterlight 0.5.5b.

Heh. Now when did this happen? Time for 0.5.6b, I guess.


Top
 Profile Send private message  
Reply with quote  
PostPosted: Tue Nov 06, 2018 12:21 am 
Offline

Joined: Sun Apr 18, 2010 3:58 pm
Posts: 851
Better!

_________________
At Choice of Games, we sell long-form choice-based interactive fiction games. We're looking for writers, paid in advance.


Top
 Profile Send private message  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 24 posts ]  Go to page Previous  1, 2, 3  Next

All times are UTC - 6 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group