[Web demo] 3OA + Stereo not playing together in Binaural

[crawflord]

I used FFMPEG to create an IAMF file from 3OA and stereo wav . Uploaded to the Web Demo and when binaural is selected it will only play the 3OA file. When the stereo is selected it will play back both together. In binaural mode are both supposed to play? In the demo of 3OA and stereo it seems like they do.

[jwcullen]

Can you try creating the file with headphones_rendering_mode=binaural for both the 3OA and stereo audio element?

See Encoding 3rd order Ambisonics and stereo wavs to IAMF.

By default, the headphones_rendering_mode is set to binaural for the ambisonics input, and stereo for the stereo input, with the assumption that the stereo input is a non-diegetic sound source. If both sound sources should be binauralized instead of downmixed to stereo, change both instances of headphones_rendering_mode to binaural.

[crawflord]

I see, I changed the rendering mode from stereo to binaural and it played back but the stereo file sounded very strange. The demo version with the music (non diegetic) did not so I was wondering if there was something wrong. The use case I was testing for would be a VR type environment where the 3OA would be downmixed to binaural but the stereo (non-diegetic) would not go through the binaural renderer but play back alongside it. A voice over or music on top of a 3OA file. Like how previously on youtube you could playback a 4ch FOA plus Stereo. Is this possible?

[jwcullen]

3OA downmixed to binaural, with non-diegetic stereo should be possible. Tagged as bug, will have to look into what is going wrong.

If you feel like trying a workaround, then you could try encoding the IAMF with iamf-tools instead of the longer FFmpeg command.

See 3OA and stereo templates here. They are preconfigured for binauralized 3OA + non-diegetic stereo.

[jwcullen]

This may have been because the demo uses libiamf, which had a related issue or bug AOMediaCodec/libiamf#141.

The demo hasn't been synced, so uses an older verison.