I want to get a list of all the objects(or max 1000) located at foo-bucket/foo-dir , e.g.
Bucket: foo-bucket
Directory: foo-dir
I see Bucket.objects boto3 API use filtering(Bucket.objects.filter)
I am wondering what it does internally? Does it list all the objects and filter? Or does AWS provide an API that takes filter arguments and return the filtered results.
The S3 API supports a prefix on the ListObjects request that will filter the responses to include only those objects whose name matches the prefix. So:
import boto3
s3 = boto3.resource('s3')
bucket = s3.Bucket('mybucket')
for obj in bucket.objects.filter(Prefix='foo/bar/fie/baz/'):
# do something with obj here
Would only return objects whose name begins with the prefix foo/bar/fie/baz/.