Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a SoundEffectProcessor Parameter to convert sounds to Mono channel. #1478

Open
nkast opened this issue Apr 12, 2024 · 3 comments
Open

Add a SoundEffectProcessor Parameter to convert sounds to Mono channel. #1478

nkast opened this issue Apr 12, 2024 · 3 comments
Labels
enhancement New feature or request

Comments

@nkast
Copy link

nkast commented Apr 12, 2024

  • 3D listeners require the soundFX to be mono.
  • On OpenAL we use a workaround to implement Panning through a 3D emitter.
    (we could probably slip the sound to 2 sound sources, but that is another matter).
  • XNA would convert sounds to Mono when Quality : Low.
    KNI doesn't do that, and probably we shouldn't.

For those reasons it would be nice to have an option in the Pipeline, instead of converting
the original asset.

@nkast nkast added the enhancement New feature or request label Apr 12, 2024
@damian-666

This comment was marked as off-topic.

@damian-666
Copy link

damian-666 commented May 7, 2024

I'm doing some research on the Hexagon processor that comes with Qualcomm and now ships on most Windows tablets, Surface, and OEM devices. I don't see the value in people putting stereo sound effects into their package, and maybe it should just be a warning. I don't see the use case for stereo with 3D effects.

You could do that, but just output a warning that your track is being mixed down to mono. Then, they can just do that with Audacity and put the track back in. I don't think they should compress it, especially lossy. I'm not sure if 16-bit audio is good enough. But somebody advocated bringing it up to 24-bit. The Hexagon processor might convert it to fixed point, and then you don't have to, but you need that headroom just for your calculations, and then you can put it back to 16-bit PCM integers.

In some packages, like middleware Onyx or TensorFlow Lite, there is just a switch that says, "optimize for Hexagon." But I think OpenAL should have bugs logged if it's not utilizing that resource. Because OpenAL Soft is open source with hundreds or at least 30 actual hotword kernels that are listening and detecting OS with callbacks that go to the CPU and use the same memory. This isn't anything you can do with GPU acceleration, and it's been around for 10 years. So there's a lot of work to be done, especially in middleware, and it's huge because these DSPs have been underused over a decade. So all these reverb things are very expensive to do on software.

We can see that in OpenAL Soft, with the snaps, crackles, pops, and the garbage collection kicking in and using a lot of watts, and not being able to apply more than one voice to a sound isn't good enough for what is in your document on how to do that kind of reverb that uses the actual angles of the rays that you cast, which is way better than what I proposed. Compress this and make it work on Hexagon, and hopefully, it will do the best it can. Hexagons can handle integer math and any type of vector. They can be extremely low power, and that means you can listen and have 30 kernels listening at once to your microphone.

@damian-666
Copy link

I updated the comment above. I've been doing more research. I'm trying to implement a hot word ,not a wake word, but a real hot word. Low latency machine learning thing and see if I can learn anything from that. Maybe even running 60 like 30 kernels at once, listening for up, up, left, left, right, right, down, down. All that stuff. I'll. I'll put it in the discussion.
https://www.quora.com/Is-16-bit-44-1-kHz-audio-good-enough

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants