Business
Jobs
  • About Us
  • Solutions
    • Job Postings
      Post your job and receive qualified candidates in 48h.
    • Candidate Assessments
      500+ technical and psychological tests, plus anti-fraud.
    • Headhunting
      Tailor-made executive search from start to finish.
    • Payroll + EOR
      Payroll dispersal and EOR across 15+ LATAM countries.
  • Pricing
  • Jobs

0

76
Views
FS.Readable Stream saltando filas al importar a noSQL usando esquemas Mongoose

Estoy intentando importar un CSV usando mi modal mangosta e independientemente de su tamaño, estoy importando las primeras 2 filas y luego cada dos filas.

 const fs = require('mz/fs'); const { parse } = require('@fast-csv/parse'); const streamToIterator = require('stream-to-iterator'); mongoose.Promise = global.Promise; mongoose.set('debug', true); const Product = require('./schemas/Product'); mongoose.Promise = global.Promise; const options = { useNewUrlParser: true, useUnifiedTopology: true, }; const database = mongoose .connect( process.env.DATABASE_URL, options ) .then((db) => (async function () { console.log('Connected to database.'); try { await Promise.all( Object.entries(db.models).map(([k, m]) => m.deleteMany()) ); let headers = Object.keys(Product.schema.paths).filter( (k) => ['_id', '__v'].indexOf(k) === -1 ); if (await fs.exists('./database.csv')) { let stream = fs .createReadStream('./database.csv') .pipe(parse({ headers })); const iterator = await streamToIterator(stream).init(); let buffer = [], counter = 0; for (let docPromise of iterator) { let doc = await docPromise; buffer.push(doc); counter++; if (counter > 10000) { await Product.insertMany(buffer); buffer = []; counter = 0; } } if (counter > 0) { await Product.insertMany(buffer); buffer = []; counter = 0; } } } catch (e) { console.error(e); } })() ) .catch((err) => console.error('Error connecting to database:', err)); module.exports = database;

Cuando miro mi variable doc, ya está en un estado incorrecto (cada dos filas) y cuando estoy leyendo la secuencia, ya está en un estado incorrecto, ¿así que asumo que está ocurriendo por ahí?

about 4 years ago · Santiago Gelvez
1 answers
Answer question

0

Lo que terminé haciendo para resolver esto fue simplemente convertir el CSV en JSON e importarlo normalmente. No es ideal o realmente aborda el problema subyacente, pero mi base de datos tiene lo que necesita.

about 4 years ago · Santiago Gelvez Report
Answer question
Find remote jobs

Discover the new way to find a job!

Top jobs
Top job categories
Business
Post vacancy Pricing Sales
Legal
Terms and conditions Privacy policy
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Show me some job opportunities
There's an error!