Question

0

205

Vistas

What is the best practice of streaming JSON objects in Nodejs, if it is bigger than 10M and split it into multiple streams?

I have a JSON object, it is a big object. I need to send (write) it as a Kafka message and it takes a lot of time.

I need a mechanism to stream the JSON object and if it is bigger than 10M, split it into multiple stream messages and then send it.

Please help me to find a best practice.

Thank you.

about 4 years ago · Juan Pablo Isaza

1 Respuestas

Responde la pregunta

0

Responde la pregunta

Encuentra empleos remotos

Answer 1 · 2022-06-30T03:31:26.556Z

If it is possible to do decompress in the client. I would do :

compress 10MB json file. It will reduce to about 115KB
encode base64 the file. It will add the size to 160KB

160KB is a lot smaller than 10MB and should be easy to streamed. It's similar with what netflix does ( compression ) that's why their movie stream so fast in our TV

0

205

What is the best practice of streaming JSON objects in Nodejs, if it is bigger than 10M and split it into multiple streams?

1 Respuestas

0

Encuentra empleos remotos

Andres GPT