i'm implementing a Transform stream which is piped by mongoose cursor (which works like readable stream, (or maybe it's real readable stream))
const myTransform = new Transform({
readableObjectMode: true,
writableObjectMode: true,
transform(chunk: Document, encoding, callback) {
this.push(chunk?.toObject() ?? null); // <- transforms data
callback(); // <- calls callback
},
});
MyMongooseModelWithHugeData.find()
.cursor()
.pipe(myTransform)
.pipe(someWriteStream)
Yup. My current code works fine.
But I found out the callback inside transform implementation, receives second parameter (which looks like classic nodejs async callback style obviously).
So, I changed my code like below, and found out that it also works fine..
const myTransform = new Transform({
readableObjectMode: true,
writableObjectMode: true,
transform(chunk: Document, encoding, callback) {
callback(null, chunk?.toObject() ?? null); // <- transforms data and calls callback.
},
});
And I felt somthing awkward. I've search some blogs about creating Transform stream. And all articles that I found was telling same that I have to use this.push() and call callback().
But none of them did even mention about the second parameter of callback().
So, my question is:
this.push() and callback() when passing transformed value?this.push instead of callback ?from nodejs documantation
It is possible that no output is generated from any given chunk of input data.
The callback function must be called only when the current chunk is completely
consumed. The first argument passed to the callback must be an Error object
if an error occurred while processing the input or null otherwise. If a second
argument is passed to the callback, it will be forwarded on to the
transform.push() method.
In other words, the following are equivalent:
transform.prototype._transform = function(data, encoding, callback) {
this.push(data);
callback();
};
transform.prototype._transform = function(data, encoding, callback) {
callback(null, data);
};