Encode AudioBuffer with Opus (or other codec) in Browser

问题

I am trying to stream Audio via Websocket.

I can get an AudioBuffer from the Microphone (or other Source) via Web-Audio-Api and stream the RAW-Audio-Buffer, but i think this would not be very efficient. So i looked arround to encode the AudioBuffer somehow. - If the Opus-Codec would not be practicable, i am open to alternatives and thankful for any hints in the right direction.

I have tried to use the MediaRecorder (from MediaStreamRecording-API) but it seems not possible to stream with that API, instead of plain recording.

Here is the Part how i get the RAW-AudioBuffer:

const handleSuccess = function(stream) {
    const context = new AudioContext();
    const source = context.createMediaStreamSource(stream);
    const processor = context.createScriptProcessor(16384, 1, 1);

    source.connect(processor);
    processor.connect(context.destination);

    processor.onaudioprocess = function(e) {
    
      bufferLen = e.inputBuffer.length
        const inputBuffer = new Float32Array(bufferLen);
        e.inputBuffer.copyFromChannel(inputBuffer, 0);

        let data_to_send = inputBuffer


    
      //And send the Float32Array ...
    
    }

navigator.mediaDevices.getUserMedia({ audio: true, video: false })
      .then(handleSuccess);

So the Main Question is: How can i encode the AudioBuffer. (and Decode it at the Receiver) Is there an API or Library? Can i get the encoded Buffer from another API in the Browser?

回答1:

The Web Audio API has a MediaStreamDestination node that will expose a .stream MediaStream that you can then pass through the WebRTC API.

But if you are only dealing with a microphone input, then pass directly that MediaStream to WebRTC, no need for the Web Audio step.

---

Ps: for the ones that only want to encode to opus, then MediaRecorder is currently the only native way. It will incur a delay, will generate a webm file, not only the raw data, and will process the data no faster than real-time.

Only other options now are to write your own encoders and run it in WabAssembly.

Hopefully in a near future, we'll have access to the WebCodecs API which should solve this use case among others.

来源：https://stackoverflow.com/questions/61292834/encode-audiobuffer-with-opus-or-other-codec-in-browser