Empresas
Empregos
  • Sobre nós
  • Soluções
    • Publicação de vagas
      Publique sua vaga e receba candidatos qualificados em 48h.
    • Avaliações de candidatos
      Mais de 500 testes técnicos e psicológicos, mais anti-fraude.
    • Headhunting
      Busca executiva personalizada do início ao fim.
    • Folha de Pagamento + EOR
      Dispersão de folha e EOR em mais de 15 países da LATAM.
  • Preços
  • Empregos

0

181
Visualizações
Puppeteer & Chromium using large CPU?

I have a nodejs script running on my Macbook 24/7, and every 20 minutes or so it runs Puppeteer to parse the contents of a website. Every two days, the fan gets very loud, and I see Chromium is using lots of CPU and RAM. Is there someway I can fix this?

This is the code I am running:

const options = {
                args: [
                    '--no-sandbox',
                    '--disable-setuid-sandbox',
                    '--disable-dev-shm-usage',
                    '--disable-accelerated-2d-canvas',
                    '--no-first-run',
                    '--no-zygote',
                    '--single-process', 
                    '--disable-gpu'
                ],
                headless: true
            }

const browser = await puppeteer.launch(options);

const page = await browser.newPage();

await page.setUserAgent('Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36')

await page.setViewport({ width: 1920, height: 1080 });


await page.setRequestInterception(true);
page.on('request', (req) => {
if(req.resourceType() === 'stylesheet' || req.resourceType() === 'font' || req.resourceType() === 'image'){
    req.abort();
 } else {
      req.continue();
  }
});

await page.goto(url, { waitUntil: 'networkidle0' });


// ... do stuff

await page.close();
await browser.close();

enter image description here

enter image description here

about 4 years ago · Juan Pablo Isaza
1 Respostas
Responde à pergunta

0

I have an idea to automatically kill and launch puppeteer every 20 minutes. Puppeteer browser (chromium) will automatically closed when scrape/parse job finished, and then after 20 minutes, it will relaunch fresh new browser, so it wont eat more resources. I'm using browser pool (cluster) for this method.


import cluster from 'cluster'
import * as os from 'os'
import * as process from 'process'
import * as puppeteer from 'puppeteer'

const launchOptions = {
    args: [
        '--no-sandbox',
        '--disable-setuid-sandbox',
        '--disable-dev-shm-usage',
        '--disable-accelerated-2d-canvas',
        '--no-first-run',
        '--no-zygote',
        '--single-process',
        '--disable-gpu'
    ],
    headless: true
}

let lastProcessUptime = []
let userAgentString = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36'
let viewportSize = { width: 1920, height: 1080 }
let CPUsNum = os.cpus().length
CPUsNum = 1 // Set this number as how many browser page do you want to launch simultaneously

const runThis = async (processID) => {
    const browser = await puppeteer.launch(launchOptions)
    const page = await browser.newPage()
    await page.setUserAgent(userAgentString)
    await page.setViewport(viewportSize)
    await page.setRequestInterception(true)
    page.on('request', (request) => {
        let blockedResources = ['stylesheet', 'font', 'image']
        if (blockedResources.includes(request.resourceType())) {
            await request.abort()
        } else {
            await request.continue()
        }
    });
    await page.goto(url, { waitUntil: 'networkidle0' })
    // ... and do stuff
    // ... then close the page and browser and also kill the process
    await page.close()
    await browser.close()
    lastProcessUptime.push({
        pid: process.pid,
        uptime: Math.round(process.uptime() * 1000)
    })
    process.kill(processID)
    process.exit(0)
}

if (!cluster.isWorker) {
    console.log(`Primary ${process.pid} is running`)

    // Fork workers.
    for (let i = 0; i < CPUsNum; i++) {
        cluster.fork()
    }

    cluster.on('exit', (worker, code, signal) => {
        console.log(`worker ${worker.process.pid} died`) /*with code: ${code} and signal ${signal}*/
        setTimeout(() => cluster.fork(), (20 * 60 * 1000) - lastProcessUptime[lastProcessUptime.findIndex(process => process.pid === worker.process.pid)].uptime) //lets recreate the worker that died
    })
} else {
    let processID = process.pid
    console.log(`Worker ${processID} started`)
    ;(async (processID: number) => {
        await runThis(processID)
    })(processID)
}


about 4 years ago · Juan Pablo Isaza Relatório
Responde à pergunta
Encontrar trabalhos remotos

Descubra a nova forma de encontrar um emprego!

melhores empregos
Principais categorias de trabalho
Empresas
Postar vaga Preços Comercial
Jurídico
Termos e Condições Política de privacidade
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Recomende algumas ofertas para mim
Preciso de ajuda