I have a canvas that is being rendered onto a video via video.srcObject = canvas.captureSteam()
That video is then made the active picture-in-picture element using video.requestPictureInPicture()
I would like to add some interactivity to the floating window (for example, a clickable button rendered in the canvas).
Is there a way to get mouse events on the floating window? All I really need is the mouse position and mouse button down/up.
I have tried using addEventListener on the PictureInPictureWindow that is returned from requestPictureInPicture but it seems like the only event that it is able to pass is resize
So for my use case, this was for a video conferencing app. I only needed to mute audio and video and a button to hang up; turns out these are implemented in the MediaSession API.
Adding the following made audio mute/unmute, video mute/unmute, and a hang up button that shows up in the picture-in-picture window on hover:
navigator.mediaSession.setActionHandler('togglemicrophone', toggleAudioMuted); navigator.mediaSession.setActionHandler('togglecamera', toggleVideoMuted); navigator.mediaSession.setActionHandler('hangup', hangup); navigator.mediaSession.setMicrophoneActive and navigator.mediaSession.setCameraActive can also be used to sync the muted/unmuted state with the picture-in-picture window.
More details can be found here: https://w3c.github.io/mediasession/#the-mediasession-interface
And here: https://developer.mozilla.org/en-US/docs/Web/API/MediaSession