audio in Twine or other CYOA

I have perhaps an odd technical question.

Suppose I have sound clips #1 and #2 of the exact same length.

I want to (a) play clip #1, then stop it before it’s done playing (or maybe it’s on loop), then (b) get the timestamp t of where it is in the clip (say 2.7 s into clip #1, modulo loop), and © start playing clip #2 starting at t within clip #2. I’ve looked at sqTwineSound but it doesn’t seem to have anything like this.

I’m not married to doing this at Twine, but I couldn’t see a general forum for asking this question–I’ve looked at Inform 7 and couldn’t see a way of doing this using Massimo Stella’s Multiple Sounds (and if Damusix is available I have been unable to find it). If Twine won’t do this, is there another IF platform I should be looking at?

I do not know a way to do this in Twine or Inform. (Doesn’t mean there isn’t one, but.)

It’s a long shot that this will work with the rest of what you’re trying to do, but the tools at interlude.fm/ are designed for branching audio/video content and some of their samples let you switch tracks to view the same content from a different viewpoint/with different visuals/etc. The Bob Dylan music video is a good example, and under ads, the Coca-Cola ad shows a story where you can shift camera perspective at any time. I think the effect here is probably close to what you’re describing.

Another long shot possibility is that the Unity plug-in Fungus would a) be useful and b) give you the flexibility to do something like this by dint of being attached to something as open-ended as Unity.

I think its traditional for sound engineers to handle this problem by starting both tracks at once, and then changing the volume of each track separately. So when you want to switch from track A to track B, you’d cut the volume of track A to zero, and set the volume of track B to maximum.

dfabulich’s method makes sense, and is probably feasible in all platforms.

Twine allows you to use arbitrary Javascript, so you should be able to find several websites which explain how to do that.

For Inform 7 there is a glk_schannel_play_multi() function which will start two sound channels at exactly the same time, but you would probably have to use I6 to use it. (But maybe there’s an extension to do it, I’m not sure what the state of the art for sound is.)

[rant]Gah. Technically that’s great, but it’s a nightmare to watch. That’s pretty much why I think authors and filmmakers should retain control of the narrative; the only way to see it properly is to see it twice without changing the camera angles at all. You’re missing out on half the story all the time. Whereas in a non-interactive there are “time-bending” conventions (two things don’t HAVE to be seen at the same time to be simultaneous; and even then, there’s such a thing as split-screen) and the filmmaker can ensure people see the whole story.

This is not totally unrelated to IF; this is also something I heartily dislike in some games. Give the player too much control; take away too much control from the author; get the player to tell the story rather than see it… well, some people love it. The Coca-Cola ad shows very neatly why I hate it.[/rant]

There’s an extension called Music.i7x which lets you fiddle with channels and volume and fades and crossfades and such, though it doesn’t currently have a way to start multiple sounds in sync. I’ll look into that.

This might not have anything to do with what you’re saying, but could you just record each effect you want as it’s own single track? Instead of using timestamps and hardware to simulate it?

I have a waterfall sound loop. I have a birds chirping sound loop. They may occur separately or together, but instead of making the computer play two tracks simultaneously, I record the waterfall and the birds together as a third track to use when they’re both occurring.

If that makes any sense. No? Not applicable? Back to my cubicle.

Have you looked into html’s audio handling?

http://www.w3schools.com/tags/ref_av_dom.asp

I don’t think there’s going to be any handy way of doing everything you want together with HTML/JS, given that there’s no way to reliably ensure that a source starts playing (and keeps playing) at the exact time you want.* You could set a really tight interval to check the current time of the first clip, but I don’t believe there’d be any way to guarantee that there wouldn’t be any sort of gap between finding the point in question in clip #1 and starting clip #2. You could maybe reliably do that if they’re local media files, but most web IF doesn’t rely upon the user downloading a whole folder of media items along with the HTML.

Honestly, I think what Hanon suggested is almost certainly the best way to ensure that the two “files” play at the exact right time relative to one another.

*I’d really love to be wrong on this!