0
190
ViewsHow to scrape part of a page node.js
I'm fairly new to coding and I don't understand why my code isn't working. It is suppose to output "Dominus Empyreus"
Though it only outputs []
Here is my current code:
const axios = require('axios');
const cheerio = require('cheerio');
const extractLinks = $ => [
...new Set(
$('.border-bottom item-name-container') // Select pagination links
.toArray() // Convert cheerio object to array
),
];
axios.get('https://www.roblox.com/catalog/21070012/Dominus-Empyreus').then(({ data }) => {
const $ = cheerio.load(data); // Initialize cheerio
const stuff = extractLinks($);
console.log(stuff);
// ['Dominus Empyreus']
});
Any help is appreciated! Thanks.
2 answers
Answer question0
This is the element you are trying to select.
<div class="border-bottom item-name-container">
You have written:
.border-bottom item-name-container
Which consists of:
- A class selector (for
class="border-bottom") - A descendant combinator
- A type selector (for
<item-name-container>)
What you need is:
- A class selector (for
class="border-bottom") - Another class selector (for
class="item-name-container") - No descendant combinator (because you are targeting two features of the same element, not one element that is a descendant of the other).
Such:
.border-bottom.item-name-container
0
An alternative solution is to use the full selector path to the element directly that you want to scrape from a web page as such:
const axios = require('axios');
const cheerio = require('cheerio');
const extractLinks = $ => [
...new Set(
$("#item-container > div.remove-panel.section-content.top-section > div.border-bottom.item-name-container > h1") // Select pagination links
.toArray() // Convert cheerio object to array
),
];
axios.get('https://www.roblox.com/catalog/21070012/Dominus-Empyreus').then(({ data }) => {
const $ = cheerio.load(data); // Initialize cheerio
const stuff = extractLinks($);
console.log(stuff[0].children[0].data);
// ['Dominus Empyreus']
});
A quick and simple way of obtaining a full selector path is using the F12 developer's console on your browser, right-click the element -> Copy -> Copy Selector.
In this case, the selector for this <h1> element is:
#item-container > div.remove-panel.section-content.top-section > div.border-bottom.item-name-container > h1
The one thing to note is that this solution will only work if the selector path is the exact same on every page you wish to scrape.