Empresas
Empregos
  • Sobre nós
  • Soluções
    • Publicação de vagas
      Publique sua vaga e receba candidatos qualificados em 48h.
    • Avaliações de candidatos
      Mais de 500 testes técnicos e psicológicos, mais anti-fraude.
    • Headhunting
      Busca executiva personalizada do início ao fim.
    • Folha de Pagamento + EOR
      Dispersão de folha e EOR em mais de 15 países da LATAM.
  • Preços
  • Empregos

0

96
Visualizações
Restart while loop when selector for next page is clicked

sorry but I am very new to coding and this is my first project, in which i chose to create a simple scraper for eBay using puppeteer. The program takes in the item to be searched and the desired amount of pages to scraped as a command line argument. However, due to the selector children li:nth-child increasing each time, i had to create an incremental while loop to through said n-th children from 2 to 60. This works for the first page, and outputs all items desired. However, due to having to scrape 2+ pages, the i value used in the loop must reset at the start of the next page to restart said loop. I believe the pagination is correct as in headless mode, the second, third, etc. pages are clicked through.

current loop

let currentPage = 1;

while (currentPage <= pagestoScrape) {
    let content = await page.evaluate(() => {
        let data = [];
        let childnumber = 2;
        let i = 2;

        while (i < 60) {
            let itemids = document.querySelectorAll(`#srp-river-results > ul > li:nth-child(${childnumber}) > div`);
            itemids.forEach((itemid) => {
                let PRODUCT_ID = `#srp-river-results > ul > li:nth-child(${childnumber}) > div > div.s-item__info.clearfix > a > h3`;
                let PRODUCT_PRICE = `#srp-river-results > ul > li:nth-child(${childnumber}) > div > div.s-item__info.clearfix > div.s-item__details.clearfix > div:nth-child(1) > span`;
                let item = itemid.querySelector(PRODUCT_ID).innerText;
                let price = itemid.querySelector(PRODUCT_PRICE).innerText;
                data.push({
                    item,
                    price,
                });
            });
            childnumber++;
            i++;
        }
        return data;
    });

    let urls = [];
    urls = urls.concat(content);
    if (currentPage < pagestoScrape) {
        await Promise.all([
            await page.click('#srp-river-results > ul > div.srp-river-answer.srp-river-answer--BASIC_PAGINATION_V2 > div.s-pagination > span > span > nav > a.pagination__next.icon-link'),
            await page.waitForSelector('#srp-river-results > ul > div.srp-river-answer.srp-river-answer--BASIC_PAGINATION_V2 > div.s-pagination > span > span > nav > a.pagination__next.icon-link'),
        ])
    }

    currentPage++;
    let jsonData = JSON.stringify(content, null, 1);
    fs.writeFileSync(`./output/output${item}.txt`, jsonData);
}

I have tried reavaluating i after the loop, during the promise and changing the loop to a for. I expected this to reset the loop and for the output to contain the results of each page. However, it results in the output of just 1 page, with no error.

about 4 years ago · Juan Pablo Isaza
Responde à pergunta
Encontrar trabalhos remotos

Descubra a nova forma de encontrar um emprego!

melhores empregos
Principais categorias de trabalho
Empresas
Postar vaga Preços Comercial
Jurídico
Termos e Condições Política de privacidade
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Recomende algumas ofertas para mim
Preciso de ajuda