In my NodeJS application, I have an axios response that returns a stream. I'd like to upload this stream to my S3 bucket, but I also need to know the ContentType to prevent S3's default assignment of application/octet-stream.
So far I've developed a function that uploads a stream using Stream.PassThrough:
streamUpload(key: string, acl: string) {
const pass = new stream.PassThrough();
const upload = new Upload({
client: this.s3Client,
params: {
Key: key,
Body: pass,
Bucket: `${process.env.BUCKET}`,
ACL: acl,
ContentType: "" // <--- How can I find this from a stream without storing the file in memory??
},
})
return {
writeStream: pass,
upload,
};
}
As a result, I can use this function to pipe the stream data from my axios response to the PassThrough and then await using the upload.done() while the file is uploading like this:
// data is a stream!
const { data } = await axios.post('http://watermark:4006', {}, {
responseType: "stream",
});
const { writeStream: uploadWS, upload } = s3API.streamUpload(audioEnding, 'public-read');
// pipe the stream to my `PassThrough`
// Can I add a custom Transform to extract the content-type from the stream before
// passing it to my uploadWriteStream?
pipeline(data, uploadWS, (e) => {
if (!!e) {
console.log(e)
}
})
// The S3 library allows you to wait while data is being uploaded
await upload.done()
I discovered a library called file-type that allows retrieving the file type via stream using fileTypeFromStream, however I'm not sure how to add this to my pipeline to extract the file type before running the next step of my pipeline because it is a function that you must pass a stream argument that must be awaited before receiving the file type.
Would love to hear any suggestions!