Empresas
Empregos
  • Sobre nós
  • Soluções
    • Publicação de vagas
      Publique sua vaga e receba candidatos qualificados em 48h.
    • Avaliações de candidatos
      Mais de 500 testes técnicos e psicológicos, mais anti-fraude.
    • Headhunting
      Busca executiva personalizada do início ao fim.
    • Folha de Pagamento + EOR
      Dispersão de folha e EOR em mais de 15 países da LATAM.
  • Preços
  • Empregos

0

184
Visualizações
What is the correct approach on making a For Loop with a Cheerio Object?

Simply put, I'm scraping data from a website and storing it in a database.

The relevant fields are links, names, prices and item condition.

The way I'm handling this right now is to iterate through each Element and pushing them into their respective lists. Then adding it to a database with a For Loop. So, for example:

var names= [];
$(".midbox .framebox .frameboxcells .displaybox .displayboxbottom .dt.bg0 .serptablecell2-adv .serptablebasestyle2 .valtitle.lovewrap.padr4 .underlinedlinks").each(function(){
            names.push($(this).text());
        });
...
for (x in names){
                var sql = "REPLACE INTO `item` (`link`, `title`, `price`, `date`, `item_condition`, `country`) VALUES (?)";
                var values = [links[x], names[x], prices[x], '', states[x], cc];
            
                con.query(sql, [values], function(err, result){
                    if (err) throw err;
                    });
            }

This is very naive, as it hopes all Elements exist and that they align perfectly, which as worked well so far, until I've noticed some listings on the website I'm scraping do not have an Item Condition element, so it gets skipped and the lists get desynced, resulting in the wrong values being paired up.

I understand the answer I'm looking for has to do with the .each function, but I'm not exactly sure how to go about it. I suppose I have to go the highest point, it being .midbox .framebox .frameboxcells .displaybox .displayboxbottom .dt.bg0 .serptablecell2-adv .serptablebasestyle2 and go from there. Adding a NULL value if it doesn't find an Element.

Below is the full (relevant) code:

const $ = c.load(response.data);

        $(".midbox .framebox .frameboxcells .displaybox .displayboxbottom .dt.bg0 .serptablecell2-adv .serptablebasestyle2 .splittable .splittablecell1 .padr2.bhserp-txt1.bhserp-new1").each(function(){
            var fixedStr = $(this).text().replace(/,|£|\$|\s|[(GBP)]|[(USD)]/g, '');
            prices.push(Number(fixedStr));
        });

        $(".midbox .framebox .frameboxcells .displaybox .displayboxbottom .dt.bg0 .serptablecell2-adv .serptablebasestyle2 .valtitle.lovewrap.padr4 .underlinedlinks").each(function(){
            names.push($(this).text());
        });

        $(".midbox .framebox .frameboxcells .displaybox .displayboxbottom .dt.bg0 .serptablecell2-adv .serptablebasestyle2 .splittable .splittablecell1.bhserp-txt1 .padl1.labinfo").each(function(){
            if ($(this)){
                states.push($(this).text());
            }
            else{
                console.log("Mistake here, pick me up!"); // I understand what I'm doing here does not make sense and is wrong as I've stated, but since that's what made me realize what I needed to do, I'm leaving it.
                states.push("None");
            }
        });

        $(".midbox .framebox .frameboxcells .displaybox .displayboxbottom .dt.bg0 .serptablecell2-adv .serptablebasestyle2 .valtitle.lovewrap.padr4 .underlinedlinks").each(function(){
            var tempLink = $(this).attr('href');
            var fixedLinks = tempLink.split("=");
            var fixedLinks = fixedLinks[1].split("&");
            links.push("https://www.ebay.co.uk/itm/" + fixedLinks[0]);
        });
...
con.connect(function(err){
            if (err) throw err;
            console.log("Connected!");
            for (x in names){
                var sql = "REPLACE INTO `item` (`link`, `title`, `price`, `date`, `item_condition`, `country`) VALUES (?)";
                var values = [links[x], names[x], prices[x], '', states[x], cc];
            
                con.query(sql, [values], function(err, result){
                    if (err) throw err;
                    });
            }
        });
over 4 years ago · Santiago Trujillo
2 Respostas
Responde à pergunta

0

You should iterate the elements. If you try to get prices separately from links you will have a bad experience. Something like:

for(let div of $('.product').get()){
  let item = {
    link: $(div).find('a').attr('href')
    price: $(div).find('.price').text(),
  }
  // insert item into the db
}
over 4 years ago · Santiago Trujillo Relatório

0

pguardiario's answer worked perfectly, I'll leave here the code I ended up with for future reference:

for(let div of $('.midbox .framebox .frameboxcells .displaybox .displayboxbottom .dt.bg0 .serptablecell2-adv .serptablebasestyle2').get()){
        
        var tempLink = $(div).find('.underlinedlinks').attr('href');
        var fixedLinks = tempLink.split("=");
        var fixedLinks = fixedLinks[1].split("&");

        var fixedStr = $(div).find('.padr2.bhserp-txt1.bhserp-new1').text().replace(/,|£|\$|\s|[(GBP)]|[(USD)]/g, '');
        
        let item = {
            link: "https://www.ebay.co.uk/itm/" + fixedLinks[0],
            name: $(div).find('.valtitle.lovewrap.padr4 .underlinedlinks').text(),
            price: Number(fixedStr),
            condition: $(div).find('.padl1.labinfo').text()

        }
}
over 4 years ago · Santiago Trujillo Relatório
Responde à pergunta
Encontrar trabalhos remotos

Descubra a nova forma de encontrar um emprego!

melhores empregos
Principais categorias de trabalho
Empresas
Postar vaga Preços Comercial
Jurídico
Termos e Condições Política de privacidade
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Recomende algumas ofertas para mim
Preciso de ajuda