Empresas
Empregos
  • Sobre nós
  • Soluções
    • Publicação de vagas
      Publique sua vaga e receba candidatos qualificados em 48h.
    • Avaliações de candidatos
      Mais de 500 testes técnicos e psicológicos, mais anti-fraude.
    • Headhunting
      Busca executiva personalizada do início ao fim.
    • Folha de Pagamento + EOR
      Dispersão de folha e EOR em mais de 15 países da LATAM.
  • Preços
  • Empregos

0

488
Visualizações
NodejS Read/Write Binary Data to/from file

Alright, so I have a string with binary data ("111011 10001 etc"), and I'm trying to save it to a file to them read it on another file using streams, the issue is that, the stream is cutting the data off (final binary number in chunk is cut off)enter image description here

This is how I'm sending the data to the file (reading a file, encoding it with the golombRice encoder and storing it in a file, using chunks of data)

  const writer = fs.createWriteStream(
    `./encodedAndDecoded/encoded${filename}`,
    {
      encoding: "binary",
    }
  );
  const reader = fs.createReadStream(`./silesia/${filename}`, {
    encoding: "base64",
  });

  await new Promise((resolve, reject) => {
    reader.on("data", (chunk) => {
      writer.write(Buffer.from(golombRiceEncoding(chunk, 3)));
    });
    reader.on("end", () => {
      writer.end();
      resolve();
    });
  });

This is how I'm reading it (reading the encoded file, decoding it with the golombRice encoder and storing it in a file, using chunks of data, the issue with this is that the chunks don't have the full binary data because the stream cuts it)

  const writer = fs.createWriteStream(`./encodedAndDecoded/decoded${filename}`);

  const reader = fs.createReadStream(`./encodedAndDecoded/encoded${filename}`, {
    encoding: "binary",
  });

  await new Promise((resolve, reject) => {
    reader.on("data", (chunk) => {
      writer.write(Buffer.from(golombRiceDecoding(chunk, 3), "base64"));
    });
    reader.on("end", () => {
      writer.end();
      resolve();
    });
  });

Wonder if there's a way to make it read the data using streams, but without cutting a binary number? I don't mind if it reads x at a time, the issue is when it cuts a binary number, invalidating the data when decoding. Thank you

about 4 years ago · Juan Pablo Isaza
1 Respostas
Responde à pergunta

0

Your particular compression scheme that you're trying to decode looks like it's a variable byte scheme. So, if a chunk boundary doesn't perfectly line up with the size of a variable byte piece of compression, your decompression library will fail.

To decode something like this on the fly, you have to build logic right into the decoder that recognizes when it only has a partial piece of data and then buffers that to combine it with the next chunk of data that arrives. Because the compression is variable byte, you can't known where proper boundaries are without being part of the decompression logic.

Or, you can give up doing on-the-fly decoding and buffer all the compressed data into memory and then decompress it all at once when you have all the data (then you don't have any chunk boundaries to worry about).

about 4 years ago · Juan Pablo Isaza Relatório
Responde à pergunta
Encontrar trabalhos remotos

Descubra a nova forma de encontrar um emprego!

melhores empregos
Principais categorias de trabalho
Empresas
Postar vaga Preços Comercial
Jurídico
Termos e Condições Política de privacidade
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Recomende algumas ofertas para mim
Preciso de ajuda