I am new to AWS. I am creating a Scheduled lambda function using NodeJS. There I need to get all the objects in a folder and iterate through each, to check the last modified date and I want to delete old objects. But I am getting empty array in ListObject's Response Contents.
var s3 = new aws.S3();
exports.handler = (event, context, callback) => {
var params = {
Bucket: 'bucket_name',
Delimiter: '/',
Prefix: 'folder_name',
MaxKeys:100000
};
s3.listObjects(params, function(err, data) {
if (err) console.log(err, err.stack); // an error occurred
else
{
var folderLength = data.Contents.length;
data.Contents.forEach(function(metadata) {
console.log('Key: ' + metadata.Key);
});
}
});
callback(null, 'Finished');
};
But in the response i am getting Contents: []. Can anyone tell me whats wrong?
When you call ListObjects() with Delimiter: '/', there are two arrays of interest inside data: Contents and CommonPrefixes.
Contents is the list of objects beginning with the Prefix value you specified up to but not including the next Delimiter, and CommonPrefixes is a list of the prefixes common to all the objects with the prefix of Prefix, up to and including the next occurrence of Delimiter (which is / by convention).
S3 doesn't actually have folders -- just objects that share common prefixes, delimited with /.
When you create a folder foo in the S3 console, an empty object with a key of foo/ is actually created. However, if you upload an object with key foo/bar.html using the API, the console will -- for convenience -- still display this as a file bar.html inside a folder named foo, even though in this latter case, no foo/ object was explicitly created. Significantly, in neither case is the object actually "in" a meaningful "folder" container. It's only presented that way. S3 has no actual folders -- only objects and buckets.
If you want all the objects "in a folder," what you're really saying is that you want the objects sharing a common key prefix.
But when S3 provides a list of all objects sharing the common key prefix "folder_name" up to the next / (as noted above) then there aren't any.
In your original code, Contents is empty, but you would have found that data.CommonPrefixes in the response contained a single value, folder_name/, because all of the objects whose keys begin with folder_name up to the next / actually share the common prefix folder_name/.
There's no point in making two requests with the first one being redundant and unnecessary (and, in fact, doing so will give you incorrect results if more than one folder has a name beginning with "folder_name") so when you want a list of objects "in a folder," the correct approach is to set the Prefix to folder_name + '/'.