I'm currently developing an API which takes a videoTrack as input and return a processed videoTrack. I've managed to draw the processed video on a canvas and hoping use the canvas.captureStream() to capture the videoStream from it.
As it turns out that I can capture a non-blank stream only if I load the canvas into the DOM document.body.appendChild(myTempCanvas) and keep it displayed, the stream turns blank if I hide the canvas myTempCanvas.style.display="none".
So is there any way to capture the stream and keep the canvas hidden as well?
Example: If uncomment the line canvas.style.display = "none";, then the output_video turns blank as well.
<!DOCTYPE html>
<html lang="en">
<head>
<title>Demo</title>
</head>
<body>
<div class="container">
<video autoplay class="input_video" ></video>
<canvas class="output_canvas"></canvas>
<video autoplay class="output_video" ></video>
</div>
<script>
const video = document.getElementsByClassName("input_video")[0];
const canvas = document.getElementsByClassName("output_canvas")[0];
const output_video = document.getElementsByClassName("output_video")[0];
let imageCapture = null;
let tmpStream = null;
navigator.mediaDevices.getUserMedia({
video: true
})
.then((stream) => {
imageCapture = new ImageCapture(stream.getVideoTracks()[0]);
window.requestAnimationFrame(frameloop);
tmpStream = canvas.captureStream(20);
// canvas.style.display = "none";
output_video.srcObject = tmpStream;
})
function frameloop() {
imageCapture.grabFrame()
.then(imageBitmap => {
drawCanvas(canvas, imageBitmap);
window.requestAnimationFrame(frameloop);
})
}
/* Utils */
function drawCanvas(canvas, img) {
canvas.width = getComputedStyle(canvas).width.split('px')[0];
canvas.height = getComputedStyle(canvas).height.split('px')[0];
let ratio = Math.min(canvas.width / img.width, canvas.height / img.height);
let x = (canvas.width - img.width * ratio) / 2;
let y = (canvas.height - img.height * ratio) / 2;
canvas.getContext('2d').clearRect(0, 0, canvas.width, canvas.height);
canvas.getContext('2d').drawImage(img, 0, 0, img.width, img.height,
x, y, img.width * ratio, img.height * ratio);
}
</script>
</body>
</html>
</script>
</body>
</html>
In your drawCanvas function you are setting the canvas width and height to its computed CSS width and height.
When you hide the canvas through display:none, or when it's not appended in the document, these values will be "0px", because the canvas is not in the CSSOM.
A canvas with zero width or height can't produce any output.
Setting the canvas width or height is a very slow operation, in Chrome, this will allocate a full new image bitmap every time you do so, added to the resetting of all the canvas context's default values.
So never set the canvas width or height in all the frames, only when it actually did change, and don't set it to the computed size of your <canvas> element, set it to the size of your input image, and then set the CSS size accordingly.