Have just been playing around with axios & cheerio. I was attempting to scrape table data from the world rugby rankings website. I would like to return the top 10 rows of the table. https://www.world.rugby/tournaments/rankings/mru
Presently I can only retrieve the first row of the table and I can't figure out why.
const axios = require('axios')
const cheerio = require('cheerio')
async function getWorldRankings() {
try {
const siteUrl = 'https://www.world.rugby/tournaments/rankings/mru'
const { data } = await axios({
method: "GET",
url: siteUrl,
})
const $ = cheerio.load(data)
const elemSelector = 'body > section > div.pageContent.flex-content > div:nth-child(2) > div.column.large-8 > div > section > section > div.fullRankingsContainer.large-7.columns > div > div > table > tbody > tr'
$(elemSelector).each((parentIndex, parentElem) => {
$(parentElem).children().each((childIndex, childElem) => {
console.log($(childElem).text());
})
})
} catch (err) {
console.error(err);
}
}
getWorldRankings()
Result:
>node index.jsx
Position
Teams
Points
For full context and credit I was playing around with this guide: https://www.youtube.com/watch?v=5YCuUCRS_Ks (I'm using the same code just different url's and css selectors - and I can retrieve table rows as intended with his example coinmarketcap.com and many other websites).
For the world rugby rankings site - even though the html is available in dev tools is the data being injected in some way that makes it unselectable? (I have no idea what I am talking about just throwing out a guess).
Thanks for any help.
node v16.4.2 "axios": "^0.22.0", "cheerio": "^1.0.0-rc.10",
The data for that table is being loaded later with AJAX and is not initially loaded with the page, you therefore cannot select it with Cheerio. The good news is you don't even need Cheerio for this. If you take a look at the network requests tab in your browser's development tools, you'll see the AJAX request being made uses the following URL to load JSON formatted data --the data you want-- into the page:
https://cmsapi.pulselive.com/rugby/rankings/mru?language=en&client=pulse