Business
Jobs
  • About Us
  • Solutions
    • Job Postings
      Post your job and receive qualified candidates in 48h.
    • Candidate Assessments
      500+ technical and psychological tests, plus anti-fraud.
    • Headhunting
      Tailor-made executive search from start to finish.
    • Payroll + EOR
      Payroll dispersal and EOR across 15+ LATAM countries.
  • Pricing
  • Jobs

0

364
Views
unable to get ElementHandle when filtering puppeteer elements

I am trying to use Puppeteer to scrape data from https://pagespeed.web.dev - I need to be able to take screenshots of the results and while it would be much simpler to use Google's own API to get the raw data, I can't screenshot the actual results that way. The challenge I'm having is filtering out DOM elements while still retaining the ElementHandle nature of the objects. For example, getting the "This URL" / "Origin" buttons:

In a normal JS console, I would run this:

[...document.querySelectorAll('button')].filter(b => b.innerText === 'This URL')

This would give me an Array of DOM elements that I could then run click() on or whatever.

I have tried a number of ways to get Puppeteer to give me a usable ElementHandle object and they have all returned an array of objects, with the sole member of that object being an __incrementalDOMData object:

const browser = await puppeteer.launch()
const page = await browser.newPage()
await page.goto(`https://pagespeed.web.dev/report?url=${url}`)
await page.waitForSelector(homepageWaitSelector, { visible: true })

// here's where the fun starts
const buttons = await page.$$eval('button', buttons => buttons.filter(button => button.innerText === 'This URL'))
const buttons = await page.$$eval('button', buttons => buttons.map(b => b.innerText).filter(t => t === 'This URL'))

// This one seems to run because the list of elements returned has the right length, but I can never get a breakpoint to catch inside the `evaluate` method, nor does a `console.log` statement actually print.
const buttons = await page.evaluate(() => {
  const b = [...document.querySelectorAll('button')].filter(b => b.innerText === 'This URL')
  return b
})

All of those methods end up returning something like this:

[{
  __incrementalDOMData: {
    j: ['class', 'the-classes-used'],
    key: 'a key',
    v: true
}]

Because there are so many buttons, and because the class names are all random and reused, I can't just target the ones I want (I mean I suppose I could build a super precise selector), filter them, and then return not just the filtered data but the actual elements themselves. Is what I'm asking for even possible?

about 4 years ago · Juan Pablo Isaza
1 answers
Answer question

0

Thanks to ggorlen's comment above I simplified my approach and got something working without an overly complex selector or logic:

const buttonContainer = await page.$('div.R3HzDf.ucyZQe')

const thisUrlButton = await buttonContainer.$(':first-child')
await thisUrlButton.evaluate(b => b.click())
await page.screenshot({ path: 'homepage.png' })

const originButton = await buttonContainer.$(':last-child')
await originButton.evaluate(b => b.click())
await page.screenshot({ path: 'origin.png' })
about 4 years ago · Juan Pablo Isaza Report
Answer question
Find remote jobs

Discover the new way to find a job!

Top jobs
Top job categories
Business
Post vacancy Pricing Sales
Legal
Terms and conditions Privacy policy
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Show me some job opportunities
There's an error!