Business
Jobs
  • About Us
  • Solutions
    • Job Postings
      Post your job and receive qualified candidates in 48h.
    • Candidate Assessments
      500+ technical and psychological tests, plus anti-fraud.
    • Headhunting
      Tailor-made executive search from start to finish.
    • Payroll + EOR
      Payroll dispersal and EOR across 15+ LATAM countries.
  • Pricing
  • Jobs

0

664
Views
Scrolling to the bottom of a div in puppeteer not working

So I'm trying to scrape all the concerts in the boxed off area in the picture below:

https://i.stack.imgur.com/7QIMM.jpg

The problem is the list only presents the first 10 options until you scroll down in that specific div to the bottom, and then it dynamically presents more until there are no more results. I tried following the link below's answer but couldn't get it to scroll down to present all the 'concerts':

How to scroll inside a div with Puppeteer?

Here's my basic code:

const browser = await puppeteerExtra.launch({ args: [                
    '--no-sandbox'                                                  
    ]});

async function functionName() {
    const page = await browser.newPage();
    await preparePageForTests(page);
    page.once('load', () => console.log('Page loaded!'));
    await page.goto(`https://www.google.com/search?q=concerts+near+poughkeepsie&client=safari&rls=en&uact=5&ibp=htl;events&rciv=evn&sa=X&fpstate=tldetail`);   

    const resultList = await page.waitForSelector(".odIJnf"); 
    const scrollableSection = await page.waitForSelector("#Q5Vznb");    //I think this is the div that contains all the concert items.
    const results = await page.$$(".odIJnf");  //this needs to be iterable to be used in the for loop

//this is where I'd like to scroll down the div all the way to the bottom

    for (let i = 0; i < results.length; i++) {
      const result = await (await results[i].getProperty('innerText')).jsonValue();
      console.log(result)
    }
}
over 4 years ago · Santiago Trujillo
1 answers
Answer question

0

Try this to scroll down on the list of concerts. You can keep looping until the number of results stops increasing, or you find the concert you are looking for:

await page.evaluate(()=>{
  document.querySelector("#Q5Vznb").scrollIntoView(false);
});
over 4 years ago · Santiago Trujillo Report
Answer question
Find remote jobs

Discover the new way to find a job!

Top jobs
Top job categories
Business
Post vacancy Pricing Sales
Legal
Terms and conditions Privacy policy
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Show me some job opportunities
There's an error!