Empresas
Empregos
  • Sobre nós
  • Soluções
    • Publicação de vagas
      Publique sua vaga e receba candidatos qualificados em 48h.
    • Avaliações de candidatos
      Mais de 500 testes técnicos e psicológicos, mais anti-fraude.
    • Headhunting
      Busca executiva personalizada do início ao fim.
    • Folha de Pagamento + EOR
      Dispersão de folha e EOR em mais de 15 países da LATAM.
  • Preços
  • Empregos

0

224
Visualizações
Best way to save large data to SQL server using Nodejs

I have a buffer data which is approximately 320MB. I am trying to save it into SQL server table from node js.

What is the best way I can do this? When I try to insert the buffer data directly I am getting Cannot create a string longer than 0x1fffffe8 characters.

What can I do to save it?

This is my buffer data

<Buffer 53 6f 75 72 63 65 53 63 68 65 6d 61 2c 43 6f 75 6e 74 72 79 4e 61 6d 65 2c 49 44 2c 41 4d 54 5f 45 58 43 4c 5f 54 41 58 2c 41 4d 54 5f 49 4e 43 4c 5f ... 340629006 more bytes>

I am trying to save this data to database using sequelize as below

const DBMODEL = require("../../../models/SaveBufferData")

module.exports = async (req, res) => {

try{
var bufdata = req.file.buffer;

 var fdata = fs.writeFile('files/uploaded_files/' + req.body.fileName, bufdata, "utf8", function (err) {
          console.log('errrrr',err)
          stream = fs.createReadStream('files/uploaded_files/' + req.body.fileName);
          
        // Read and display the file data on console
        stream.on('data', function (chunk) {
            console.log(chunk);
        });
        stream.on('end', function(){
          console.log('stream ended')
      })
      
      stream.on('open', function(){
          console.log('File opened')
      });
        })
var response = await DBMODEL.SaveBufferData(req.body.fileId, bufdata);

} catch (error) {
   console.log('errorerror',error)
   res.status(500).json(ResponseManager(false, error.message));
 }
 

I ended up with the below error

node:buffer:669
slice: (buf, start, end) => buf.hexSlice(start, end),
                                ^

Error: Cannot create a string longer than 0x1fffffe8 characters
    at Object.slice (node:buffer:669:37)
    at Buffer.toString (node:buffer:811:14)
    at BLOB._stringify (C:\IWA-BACKEND\PRIMS-local\PRIMS - Iwa\BACKEND\node_modules\sequelize\dist\lib\data-types.js:423:23)
    at BLOB.stringify (C:\IWA-BACKEND\PRIMS-local\PRIMS - Iwa\BACKEND\node_modules\sequelize\dist\lib\data-types.js:22:19)
    at escape (C:\IWA-BACKEND\PRIMS-local\PRIMS - Iwa\BACKEND\node_modules\sequelize\dist\lib\sql-string.js:40:48)
    at C:\IWA-BACKEND\PRIMS-local\PRIMS - Iwa\BACKEND\node_modules\sequelize\dist\lib\sql-string.js:101:14
    at String.replace (<anonymous>)
    at Object.formatNamedParameters (C:\IWA-BACKEND\PRIMS-local\PRIMS - Iwa\BACKEND\node_modules\sequelize\dist\lib\sql-string.js:96:14)
    at Object.formatNamedParameters (C:\IWA-BACKEND\PRIMS-local\PRIMS - Iwa\BACKEND\node_modules\sequelize\dist\lib\utils.js:112:20)
    at Sequelize.query (C:\IWA-BACKEND\PRIMS-local\PRIMS - Iwa\BACKEND\node_modules\sequelize\dist\lib\sequelize.js:283:21) {
  code: 'ERR_STRING_TOO_LONG'
}
about 4 years ago · Juan Pablo Isaza
1 Respostas
Responde à pergunta

0

Looking at the call-stack, this is what I diagnose is happening :

  1. You are passing a file as a buffer (from multer I guess) to the sequelize node module
  2. sequelize node module is calling Blob.stringify

step 2) is where you are probably hitting node string length upper limits

Try the following to test this hypothesis

-> try this endpoint with a file of a smaller size and keep incrementing the size of the said file ,

if this endpoint succeeds for smaller file sizes and fails after a certain file size is reached , we are can mention with high probability that the file to buffer to string conversion is the issue (as in we are hitting the ceiling of allowed size of string by either the node process or the system memory)

about 4 years ago · Juan Pablo Isaza Relatório
Responde à pergunta
Encontrar trabalhos remotos

Descubra a nova forma de encontrar um emprego!

melhores empregos
Principais categorias de trabalho
Empresas
Postar vaga Preços Comercial
Jurídico
Termos e Condições Política de privacidade
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Recomende algumas ofertas para mim
Preciso de ajuda