I am trying to crawl the contents of a website and it works most of the time (finishes in minutes) but sometimes it takes up to 8 hours. I've managed to pinpoint the issue and it is fired in the page.evaluate part. Looking at the website in headless mode=false it just loads infinitely (after a click). Also if I manually try "document.querySelector" on that page that is stuck in loading, it works for me.

The code is the following:

    console.log("Test");
    let value = await page.evaluate((sel) => {
        let element = document.querySelector(sel);
        return element? element.innerHTML: null;
    },selector);
    console.log("Test2");

What can I do to prevent it from running that long (I would try to setup some kind of timeout system for this case)? Or how could I track the time while the code is in this part? The code immediately after this part never runs (only after hours) probably because of the await.