I'm currently sending a canvas stream over WebRTC, using canvas.captureStream(). This works as intended, but my canvas is an overlay over another video and therefore has transparent pixels.

I'm aware that the usual H26x or VPx formats don't support transparency (see also Streaming video with transparent pixels using webrtc), so I decided to go with good old chroma-keying (i.e. 100% green == transparent).

To get there, I'm currently filling the canvas with transparent green at startup:

// already tried various composite ops here, none seem to work
// context.globalCompositeOperation = "destination-out";
context.fillStyle = "rgba(0,255,0,0)";
context.fillRect(0, 0, canvas.width, canvas.height);

This looks correct on the client side (the canvas is transparent, whatever I draw on top is not), but in the resulting stream that's going out over WebRTC, the background that's transparent in the browser is apparently just black, rather than green. I would expect the alpha value to just get dropped?

When I change the alpha value to anything other than 0 (e.g. 1 or 128), then the result isn't transparent anymore, but fully opaque bright green, both in the outgoing stream (good) and in the browser (not good).

I'd rather avoid having to manually do RGBA -> RGB conversion in Javascript for every frame on a hidden canvas, which is the only alternative I can think of right now. Other ideas very welcome :-)

EDIT: tested with both Chrome 96 and Firefox 94, on Ubuntu 20.04. For reference, here's the description of the compositing ops: https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API/Tutorial/Compositing/Example