I have a strange problem. I have recently switched from XHR to fetch in an attempt to implement streaming downloaded content directly into filesystem. I noticed a considerable slowdown between writing a response.blob to stream and piping response.body directly to it. Here is a first version, which is fast:
exports.httpDownloadImpl = url => headers => body => writableStream => left => right => {
return async function () {
try {
const response = await fetch(url, { method: 'POST', headers, body });
const blob = await response.blob();
await writableStream.write(blob);
await writableStream.close();
return right({});
}
catch(err) {
return left(err.message);
}
}
}
This completes in a few seconds or so for an 11MB file. The latter version takes up to 3 mins to complete:
exports.httpDownloadImpl = url => headers => body => writableStream => left => right => {
return async function () {
try {
const response = await fetch(url, { method: 'POST', headers, body });
await response.body.pipeTo(writableStream, { preventAbort: true, preventCancel: true, preventClose: true });
await writableStream.close();
return right({});
}
catch(err) {
return left(err.message);
}
}
}
I suspect that the latter version may be slower, as we are interacting with a filesystem many times instead of keeping the response in RAM, but not that slower. Do you have any idea what can cause this?
Edit: I also noticed in devtools that sometimes it works just fine, pipes the file to completion, sometimes the request "hangs" after a second (i.e. payload size and response time don't go up) sometimes after a few seconds... It's quite non-deterministic actually. And when I close chrome in the meantime, the server logs show that the connection was closed by the client, so the connection is really open, just hangs for a few mins and then suddenly reports completion.