The web page that I am scraping has several DIV elements that have the CSS class name of 's-suggestion'.
I'm looking to get dataset values out of these elements.
I have the following line of code in my script (running on node.js):
const suggestions = await page.$$eval('.s-suggestion',(divs) => divs);
When I look at the value of suggestions.length it is 10.
Now the following code:
for (let n=0;n<suggestions.length;n++) {
const suggestion = suggestions[n];
console.log("n="+n)
// line below outputs value of: undefined
console.log(" - suggestion className: "+suggestion.className);
// line commented out below is approx. what I'd like to have:
// const sKeyword = suggestion.dataset.keyword
} // next n
The values returned for the className are all undefined .
I don't think these objects are DOM nodes!
BTW: These DIVs have the same className in question but not some unique id that I would know before-hand.
Unfortunately, page.evaluate() and alike ones can only transfer serializable values (roughly, the values JSON can handle). As await page.$$eval('.s-suggestion',(divs) => divs); returns a collection of DOM elements that are not serializable (they contain methods and circular references), each element in the collection is replaced with an empty object or undefined. You need to return either serializable value (for example, an array of texts or attributes) or use something like page.$$(selector) and ElementHandle API.
For your case, try:
const keywords = await page.$$eval(
'.s-suggestion',
divs => Array.from(divs, div => div.dataset.keyword),
);