I'm trying to scrape (this) product page, specifically the modal that shows up when you click "View all bids".
The html structure is just a simple table, I'm trying to get every "Size" element. The problem is that whenever I run my code, it opens up the modal but only returns a few random shoe sizes that are not in order.
shoeSizeBids: [ '14', '11.5', '10.5', '11', '8.5', '11', '9', '9', '7', '13' ]
const bidsChartSel = '#market-summary > div.ask.ask-button-b > div.sale-size > div:nth-child(2)'; await Promise.all([page.click(bidsChartSel)]); // Get all the shoe size bids const shoeSizeBids= await page.evaluate(() => Array.from( document.querySelectorAll('tbody > tr > td:nth-child(1)'), (element) => element.textContent ) );
The sorting order comes from that page, i.e. the sizes are rendered in that order. To get them sorted properly, you'd need to:
That can be achieved with the following:
const uniqueSortedSizes = Array.from(new Set(shoeSizeBids)) .map(s => parseFloat(s, 10)) .sort((a, b) => a > b ? 1: a < b ? -1 : 0);
You are matching multiple HTML tables with the current selector (
tbody > tr > td:nth-child(1)). the one inside the modal uses:
.activity-table > tbody > tr > td:nth-child(1)
You can also use
page.$$eval as a puppeteer shorthand for
const shoeSizeBids = await page.$$eval('.activity-table > tbody > tr > td:nth-child(1)', elems => elems.map(el => el.innerText))