I have a js script that records audio from microphone and sends it via a websocket to a python REST API.
Part of js script (works in console):
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
var options = {mimeType: "audio/webm;codecs=opus", audioBitsPerSecond:16000};
mediaRecorder = new MediaRecorder(stream, options);
mediaRecorder.addEventListener("dataavailable", function(event) {
var reader = new FileReader();
reader.addEventListener("loadend", function() {
var int8View = new Int8Array(reader.result);
console.log(int8View);
});
reader.readAsArrayBuffer(event.data);
});
mediaRecorder.start(200);
If I understand correctly the int8array that this code outputs is opus-encoded audio.
How can I decode it into a raw waveform (something that soundfile.read would return, for example) using python?
I can use ffmpeg to decode opus, but it works on files, how can I turn this array into a proper .opus or .ogg file on python side (since I don't want to send files via a websocket)?
how can I turn this array into a proper .opus or .ogg file?
You're capturing a Webm file stream via audio/webm;codecs=opus, so that would need to be converted from Webm to Ogg Opus. Instead, you could simply download or send the Webm Opus stream as a file as-is:
const recordedBuffers = []
reader.addEventListener('loadend', () => {
recordedBuffers.push(reader.result)
})
function download() {
const a = document.createElement('a')
a.href = URL.createObjectURL(new Blob(recordedBuffers))
a.download = 'my-recorded-audio.webm'
a.click()
}
function saveToServer(url) {
return fetch(url, {
method: 'post',
body: new Blob(recordedBuffers),
headers: {
'content-type': 'application/octet-stream', // optional
'content-type': 'audio/webm', // optional
},
})
}