I essentially know why I'm stuck but not sure how to correct it or move forward. So this is a web scrapper to gather the Los Angeles Dodgers schedule and outcome of games. It should scrape the date of the game, who the visiting and home team are, and the relative scores for each team. I got it to point to the first row in the table but cannot figure out how to iterate over the table to collect every game.
import fs from "fs";
import cheerio from "cheerio";
import axios from "axios";
const url = "https://www.baseball-reference.com/teams/LAD/2022-schedule-scores.shtml";
let page
try {
page = fs.readFileSync("page.html", "utf8");
} catch(e) {
page = await axios(url).then((res) => res.data);
fs.writeFileSync("page.html", page, {});
}
const $ = cheerio.load(page);
const rows = $("#team_schedule > tbody > tr")
.map(function(i, row) {
const date = $("tr:nth-child(1) > td:nth-child(2)").text()
const away_team = $('tr:nth-child(1) > td:nth-child(4)').text()
const away_score = $('tr:nth-child(1) > td:nth-child(8)').text()
const home_team = $('tr:nth-child(1) > td:nth-child(6)').text()
const home_score = $('tr:nth-child(1) > td:nth-child(9)').text()
return { date, away_team, away_score, home_team, home_score };
}).get();
console.log(rows);
Do I need to use more arr.map() functions or findAll()? I am really kind of stuck and hit a wall on this one. Sorry, I am still very new at programing in general. Only been doing this for a few months.
EDIT 1: So the results of this code is as follows:
{ date: 'Friday, Apr 8', away_team: 'LAD', away_score: '5', home_team: 'COL', home_score: '3'}
{ date: 'Friday, Apr 8', away_team: 'LAD', away_score: '5', home_team: 'COL', home_score: '3'}
{ date: 'Friday, Apr 8', away_team: 'LAD', away_score: '5', home_team: 'COL', home_score: '3'}
Plus 159 more instances of this. I know using this:
$("tr.nth-child(1) > td:nth-child(2)")
Is incorrect as I am trying to fill the other 161 occurances with data from the rest of the table. So the output should look like:
{ date: 'Friday, Apr 8', away_team: 'LAD',away_score: '5',home_team: 'COL',home_score: '3'}
{ date: 'Saturday, Apr 9', away_team: 'LAD',away_score: '2',home_team: 'COL',home_score: '3'}
{ date: 'Sunday, Apr 10', away_team: 'LAD',away_score: '4',home_team: 'COL',home_score: '9'}
And so on...
One solution could be just to fill in the data in a seperate array:
import fs from "fs";
import cheerio from "cheerio";
import axios from "axios";
const url = "https://www.baseball-reference.com/teams/LAD/2022-schedule-scores.shtml";
let page
try {
page = fs.readFileSync("page.html", "utf8");
} catch(e) {
page = await axios(url).then((res) => res.data);
fs.writeFileSync("page.html", page, {});
}
const $ = cheerio.load(page);
const data = [];
const table = $("#team_schedule");
table.find("tbody tr").each((i,e)=>{
const $el = $(e);
//sort out title rows
if($el.attr('class') != "thead"){
data.push({
date: $el.find("td:nth-child(2)").text(),
away_team: $el.find('td:nth-child(4)').text(),
away_score: $el.find('td:nth-child(8)').text(),
home_team: $el.find('td:nth-child(6)').text(),
home_score: $el.find('td:nth-child(9)').text()
});
}
})
data.forEach(r=>console.log(r));