I'm trying to provide remote access of puppeteer browser to the client so that I can trace the user activities. Using Socket.io, capturing mouse events and taking screenshots of the browser, I was able to render the browser to the client like this:
const launchBrowser = async (socket) => {
try {
browser = await puppeteer.launch({defaultViewport: null, args: minimal_args});
browser.on("disconnected", () => {
clearInterval(interval);
});
let page = await browser.pages();
page = page[0];
console.log("connected")
await page.setViewport({
width: 800,
height: 600
});
await page.goto("https://example.com/", {waitUntil: "networkidle0", timeout: 0});
socket.on('MOUSE_MOVE', async ({x, y}) => {
await page.mouse.move(x, y);
});
socket.on('MOUSE_CLICK', async ({x, y, button}) => {
await page.mouse.click(x, y, {button: button});
});
socket.on('KEY_PRESS', async ({key}) => {
if (key === "ArrowUp") {
await page.mouse.wheel({deltaY: -100});
return
}
if (key === "ArrowDown") {
await page.mouse.wheel({deltaY: 100});
return
}
await page.keyboard.press(key);
});
socket.on('WHEEL', async ({x, y}) => {
await page.mouse.wheel({deltaX: x, deltaY: y});
});
//taking continuous screen shots and broadcasting it to the client
interval = setInterval(async () => {
try {
const ss = await page.screenshot({encoding: "base64", captureBeyondViewport: false});
socket.emit("receive-screen", ss);
} catch (e) {
console.log(e.message);
}
}, 1);
} catch (e) {
console.log(e);
}
}
But, this way of taking screenshots and broadcasting isn't smooth and doesn't provide a good user experience. I also tried recording the browser using puppeteer-stream and streaming it to the client but the video streams with around 3-4 seconds delay.
Is there any way to make the broadcasting of screenshots smooth or any other way to render the puppeteer browser to the client to provide remote access?