Empresas
Empleos
  • Sobre nosotros
  • Soluciones
    • Publicación de vacantes
      Publica tu vacante y recibe candidatos calificados en 48h.
    • Evaluación de candidatos
      500+ pruebas técnicas y psicológicas, más anti-fraude.
    • Headhunting
      Búsqueda ejecutiva a la medida de principio a fin.
    • Nómina + EOR
      Dispersión de nómina y EOR en más de 15 países de LATAM.
  • Precios
  • Empleos

0

272
Vistas
node.js - How to adjust the the stdout buffer size (aka highWaterMark) of child_process.spawn()?

I'm attempting to adjust the buffer size (highWaterMark) for the stdout from a child_process.spawn() call, and I'm having a hard time doing this successfully. Below is some code that illustrates what I'm looking for. I'm trying to perform some benchmarking with various stdout buffer sizes, but until I can actually adjust the buffer size, I won't be able to do so. Any advice would be appreciated, thanks!

// I'm using the `stdout` from a child_process.spawn() call to feed into another stream, and I'm having a hard time increasing the buffer size on stdout.
// The code below illustrates the issue:
const child_process = require('child_process');
const stream = require('stream');
const fs = require('fs');

const compressStream = child_process.spawn("cat", ["testfile_32MB"], {writableHighWaterMark: 1024*1024, highWaterMark: 1024*1024});  // <-- child_process.spawn does not have highWaterMark or writableHighWaterMark options, so including them has no effect, but I'm including it just as an illustration of what I want.
compressStream.stdout.writableLength = 1024*1024; // <-- This does not help.
compressStream.stdout._writableState.writableLength = 1024*1024; // <-- This does not help.
compressStream.stdout._writableState.writableHighWaterMark = 1024*1024; // <-- This does not help.
compressStream.stdout.writableHighWaterMark = 1024*1024; // <-- This does not help.
compressStream.stdout._readableState.highWaterMark = 1024*1024; // <-- This does not help.
compressStream.stdout._writableState.highWaterMark = 1024*1024; // <-- This does not help.
compressStream.stdout.on("data", (data) => {
  console.log("data length [from spawn()]:", data.length); // <-- data.length will always be <= 64KB! How can we get it higher?
});

const readStream = fs.createReadStream("testfile_32MB", {highWaterMark: 1024*1024});
readStream.on("data", (data) => {
  console.log("data length [from createReadStream()]:", data.length); // <-- This prints values <= 1MB, which is what I wish I had for child_process.spawn/stdout...
});

Note, you'll need a file called "testfile_32MB" in place to run this example code, and it should be at least a few megabytes in size.

about 4 years ago · Juan Pablo Isaza
1 Respuestas
Responde la pregunta

0

Alas, you cannot do this. highWaterMark is a mechanism that is used to signal the writer to stop because the reader is not reading and its buffer is full.

A Socket will send you the data as soon as it is available. It will always use an internal hard-coded 64k buffer. Increasing the highWaterMark will make it capable of holding more data, but it will still send you 64Kb chunks - except that you will get bursts of several calls per event loop rotation.

You don't have access to the Stream creation options when using spawn - you can eventually pass a preconstructed named pipe stream - but this won't solve your problem.

about 4 years ago · Juan Pablo Isaza Denunciar
Responde la pregunta
Encuentra empleos remotos

¡Descubre la nueva forma de encontrar empleo!

Top de empleos
Top categorías de empleo
Empresas
Publicar vacante Precios Comercial
Legal
Términos y condiciones Política de privacidad
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Recomiéndame algunas ofertas
Necesito ayuda