I'm trying to serve videos and images from a private S3 bucket. I can download the files from the bucket and when I download them directly via the web interface they work fine, but when I use the SDK to server them using express I get a corrupted file. Currently, I use this code to fetch the file from S3:
export async function getObjectAsString(key: string): Promise<S3File> {
const accessKeyId = process.env.AWS_KEY_ID as string;
const secretAccessKey = process.env.AWS_SECRET_KEY as string;
const bucketName = process.env.AWS_BUCKET_NAME as string;
const client = new S3Client({
region: 'eu-west-1',
credentials: {
accessKeyId: accessKeyId,
secretAccessKey: secretAccessKey,
},
});
const response = await client.send(
new GetObjectCommand({
Bucket: bucketName,
Key: key,
}),
);
// The code like below should really be provided as nice interfaces by the SDK itself.
return new Promise((resolve, reject) => {
if (!response.Body) {
reject('No Body on response.');
} else {
const chunks: Uint8Array[] = [];
const bodyStream = response.Body! as Readable;
bodyStream.once("error", (error) => reject(error));
bodyStream.on('data', (chunk) => chunks.push(Buffer.from(chunk)));
bodyStream.on('end', () =>
resolve({
file: Buffer.concat(chunks).toString('utf-8'),
type: response.ContentType!,
length: response.ContentLength!,
}),
);
}
});
}
It's a slightly modified version of this: https://github.com/aws/aws-sdk-js-v3/issues/1877#issuecomment-1129709683
Then I use the result of that function in this express endpoint:
router.get('/media/:id', async (req, res) => {
const file = await getObjectAsString(req.params.id);
if (!file) {
return res.status(404).send({error: 'File not found'});
}
writeFileSync('./img.jpeg', file.file); // I use this to debug, this will later be removed
res.setHeader('Content-Type', 'image/jpeg').send(file.file);
});
As you can see I also tried just writing the output to a file, but that file also got corrupted, so my guess is that I'm not fetching the file correctly. Also, the S3File type looks like this:
type S3File = {
file: string,
type: string
length: number
}