How can audio be extracted from a video in HTML5?

Question

How could an audio track be extracted from a video in HTML5 Javascript as the raw audio data? I.e. an array of samples?

I am completely new to the HTML5 Video API so an example would be great.

Answer 1

The Web Audio API is exactly what you want. In particular, you want to feed a MediaElementAudioSourceNode into an AnalyserNode. Unfortunately, the Web Audio API is only implemented in Chrome (somewhat implemented in FF), and even Chrome doesn't have full support for MediaElementAudioSourceNode yet.

var context = new webkitAudioContext();

// feed video into a MediaElementSourceNode, and feed that into AnalyserNode
// due to a bug in Chrome, this must run after onload
var videoElement = document.querySelector('myVideo');
var mediaSourceNode = context.createMediaElementSource(videoElement);
var analyserNode = context.createAnalyser();
mediaSourceNode.connect(analyserNode);
analyserNode.connect(context.destination);

videoElement.play();

// run this part on loop to sample the current audio position
sample = new Float32Array(analyser.frequencyBinCount);
analyser.getFloatFrequencyData(sample);