Ogg in Web Player

Will the initial release of the browser-gameplay system be capable of playing audio and displaying images, or is it text-only?

This is not an idle question. I’ve started adding music clips to my game, and I’m wondering whether those who play it in a browser will be able to hear them. (I’m using ogg files, if that makes a difference.)

I have to say, I’m really glad this capability exists! It’s going to add some nice atmosphere. I haven’t tried the unfinished game yet in CocoaTADS on the Mac, but in my tests last year, CocoaTADS was fine with audio. I don’t recall if I tested ogg files, though.

It is capable of playing sounds, but there’s no library support for it at the moment. You can’t use the HTML TADS tag, as you would have in older versions of TADS 3. Instead, you’ll have to use HTML/Javascript to add your own sound playback function, but I don’t imagine it would be that difficult for you. The System Manual has a bit more information.

Does that mean that Ogg Vorbis is supported in Firefox, Opera and Chrome but not in IE or Safari? (That’s what it seems like, unless I’m mistaken.)

I wish it had more information, but it doesn’t. It just says, in essence, “If you want to have sound, you’ll have to use Javascript.” Well, that’s okay – I’ve done a tiny bit of Javascript programming, and I can work out the details. But I have to know where to start. Where the heck do I put my Javascript code? Do I put it all in a double-quoted string, like this?

"< ... js code here ... >";

Or is some other approach required?

Sadly, as Trumgotist points out, some browsers will play mp3 but not ogg, while others will play ogg but not mp3. (Stupid, arrogant developers…) So my Javascript code will need to know which browser it’s running in. Javascript may provide a way of grabbing that information. I’ll look into it when the time comes. Another interesting question is whether I will be able to host the music files on my server rather than clutter up the TADS servers with them. Javascript can load files from anywhere on the Internet – but will T3 games be sandboxed?

Could you just do a manual test – have the game make a ping with an .ogg at the beginning, and ask if the player hears it?

Fortunately that’s not necessary as you can provide an ogg and an mp3 version of each audio file. The browsers skip incompatible formats until they find one they can play. If all else fails, you can have a Flash player fallback for older browsers. I can recommend the jPlayer library (jplayer.org/) that uses these techniques for a wide browser support. (Assuming TADS supports external JavaScript libraries.)

The drawback is of course that you have to make two versions of each audio file, which can be rather laborous if you have many of them.

It’s probably best to package the music files along with the game, so that a player who downloads the web version and uses it with a desktop interpreter can play the sounds without requiring an Internet connection.

T3 games are not sandboxed in the way you’re thinking - the T3 game acts as a web server, and can serve up content in essentially the same way. That content could be JS code that plays Ogg sounds.

The sandboxing happens at the client / player / browser level. If your JS code says “do something with content from a different server” then the browser may block that for security reasons. The workaround would be to serve the other content from T3, so the browser sees it as coming from the same server and allows the interaction.

I will try to put together a sound demo over the next few days.

Making two versions of each file requires no more than 30 extra seconds per file – it’s super-easy.

The problem with browsers skipping formats that they can’t play may prove to be (untested) that Chrome can play both. So if I just tell the game, “play this file AND this file,” people who are using Chrome might hear a doubled playback, resulting in phase cancellation ugliness.

This part of the project is several weeks in the future at this point – I’ve just started creating the music, and the game is not yet ready for testing.

No, Chrome handles that just fine. You’re not actually telling the browser to play two files, you’re telling it to play one sound with several source file options, like this:

[code]

[/code]

I’ve put together a basic sound support extension for WebUI, using the SoundManager 2 JS library.

You can find the project files here:
dev.tads.io/websound/dir?ci=tip

(Or download the zip archive.)

It uses HTML TADS sound tags, which are filtered from the output and dispatched as events to WebUI. Most of the attributes are unimplemented, the exceptions being “src” and “cancel”, both with caveats.

The makefile contains some resource shadowing trickery, where the stock main.htm and main.js are bundled and later replaced by the modified versions. I don’t know if this behavior is legal or well-defined, but it seems to work.

Sound files are currently required to be under webuires; this can probably be fixed but I couldn’t figure out how to get resources stored elsewhere to not return a 404. The last line of the makefile shows how to remap a sound file in a different folder into the webuires hierarchy.

For our purposes, the most significant logic is the soundPlay function in sound.js. This takes the attributes from a sound tag as a parameter and does stuff. It should be straightforward to extend it if the SM2 API supports what you want to do.

MP3 is probably your best bet as far as formats go. SM2 does support Ogg and can be set to favor it where native support is available, but to take advantage of this you’d need to specify multiple files in the sound tag, then modify soundPlay to set things up correctly.

Thanks for doing the extra work, Ben. I can’t help wishing I understood what you’re saying…

“…filtered from the output and dispatched as events to WebUI…”, “…resource shadowing trickery…”, “…currently required to be under webuires…”, “…last line of the makefile shows how to remap a sound file in a different folder into the webuires hierarchy…”, “…the most significant logic is the soundPlay function in sound.js. This takes the attributes from a sound tag as a parameter…”, “…specify multiple files in the sound tag, then modify soundPlay…”

I can get around pretty well in T3, but when I read descriptions like that, I can only echo Manuel in the Fawlty Towers episode where Basil has bet on a horse and doesn’t want Sibyl to find out: “I know nothing.”

Well, I am happy to unpack it.

The only TADS code is the OutputFilter defined in websound/websound.t. Basically all this does is scan output for tags. It rewrites them in an XML compliant way and then uses eventPage.sendEvent to send them to the browser.

On the browser side, I modified main.htm to load the new sound libraries (websound/sound.js and websound/soundmanager2.js). I also modified the main.js script to initialize the sound code (line 27) and to handle tags (line 115). Those were the only real changes needed in the stock code.

However, since main.htm and main.js are part of webui, they are already included as resources in the T3 output. One approach would be to copy the entire webuires folder from the base install into the game directory, and then modify the files locally. This is straightforward but requires all of those files to be distributed instead of just the modified ones. It has the drawback of making the change look much more invasive than it really is.

If you look at the Makefile.t3m, you’ll see that I used a different approach. I included the modified files in a separate directory (websound) and then told the compiler to name them something else when linking them in. This is normal; the unusual part is doing that when there is definitely another resource with that name already in the .t3 file. First the default webuires/main.htm gets added as webuires/main.htm, and then websound/main.htm gets added as webuires/main.htm. The old resource is still there, but it’s inaccessible because another resource took its name.

This seems to do what I want - the first main.htm gets superseded by the next one - but I don’t know if it’s guaranteed to work.

Something similar has to happen with audio files in order to get them to where the browser can find them. Normally if you had a file called res/audio.mp3, you could link it in and all would be well. But with WebUI the files need to be under the webuires structure or the browser can’t access them. So in the makefile, you can say “add this resource but use this path instead of the original one” - that’s what the last line does.

Then in your T3 code you can refer to “res/audio.mp3”, the JS code will look for “/webuires/res/audio.mp3”, and the compiler will make sure that is the resource path.

The idea is that authors can use the same sound tags to support audio in both adv3 and adv3web games. The only thing that has to change is the makefile, but that has to change in any case to include the networking features.

If you open up soundPlay() in sound.js the rest of it should be clear. It’s just straight JS code at that point. Right now it only does stuff with the src and cancel attributes. But you could add any attribute you like and handle it in a similar way. For example:

function soundPlay(tag) {
	var cancel = tag.getNamedItem("cancel");
	if (cancel) {
		soundManager.stopAll();
	}
	else {
		var mp3Src = tag.getNamedItem("src");
		var oggSrc = tag.getNamedItem("oggSrc");
		// some code that uses both the MP3 and OGG filenames to set up SM2
	}
}

Then your T3 game code could send a somewhat nonstandard sound tag that references both files.

<sound src="res/test.mp3" oggSrc="res/test.ogg" />

Whether that works depends entirely on how the soundPlay function handles that combination of attributes.

I’ve waited a week to respond to this thread, because I wanted to make a bit more progress with my project before I started worrying about distribution of a game that includes music. My game is now ready for preliminary testing, so please allow me to reply to your most recent post, Ben, by saying … I still don’t get it. You’re aiming way over my head.

I’m sure what you’re saying makes perfect sense, but I’m not a real programmer. I’m just a hobbyist and an IF author. I’m fairly conversant with TADS programming, but you can safely assume that I know nothing else. If you assume this, you will be right 95% of the time.

What I would need would be a step-by-step tutorial that tells me precisely what to do. As in, (a) put these files in these folders, (b) if you need to change item A, go to line X in file Z and edit it to read “blah blah blah,” © use this menu command in Workbench to compile the result, (d) upload the result to web server Q by logging in to web page P.

Beyond this, though, I’m wondering how the heck I can get a game beta-tested using the web server setup. Obviously, it can’t go onto an IFDB page! So what exactly would I do? I don’t have a server. I do have a website, but I wouldn’t even know what questions to ask of tech support in order to determine whether my site could serve as a playable repository for the beta version.

In order to get the game tested in time for the Spring Thing, I need to start looking at these issues now. I’m hoping the game will have graphics, but there will definitely be music, and it will all need to be thoroughly tested for online play, because the only Mac interpreter that will currently play 3.1 games is FrobTads, which won’t do music, graphics, or even basic font effects.

What’s my next step? How do I compile a version for web distribution, and how do I get it tested?

Just for kicks, I thought I’d try the Build Web Page Package command in the Build menu. Here’s the result:

That’s the limit of my expertise – choose a likely-looking menu command, try it, and observe the results. Maybe it’s the wrong command. Dunno. Suggestions welcome.

Can you post or PM me the contents of your project’s .t3m file? I will edit it to make the changes necessary to build a WebUI project. You can obscure any strings as long as structure remains the same.

Workbench is set up to abstract away the details of the Makefile process, which is normally useful but obscures some important choices here. The new project wizard gives you a choice between a web project and a regular one, but it’s relatively easy to switch - provided you haven’t made heavy use of incompatible features like banners.

This will be much clearer once you’ve built a WebUI version of your .t3 file. On Windows, you simply double-click it and the HTML TADS player will load it in a web window instead of the traditional one.

On Mac, I’ve been using Frob from the command line, which spits out a URL, which I paste into my browser. It’s clunky but it actually facilitates testing in different browsers.

Actual online play should be equivalent to local play, except somewhat slower. You can upload your file anywhere and then construct a link to the gs.tads.io network.

Like so: Play my game!

Your best bet will be to build two versions in parallel - web and regular - and make sure your testing covers both. It’s not crucial that testers run through it online (though it can’t hurt) but you definitely want to get some eyes on the web format.

For music, if you can send me a sample game that does broadly the same things with sound tags as your game does, I will beef up the websound implementation so that it can handle it.