I have the following code to try to remove a directory from S3:
s3 = boto.connect_s3(
aws_access_key_id=settings.AWS_ACCESS_KEY_ID,
aws_secret_access_key=settings.AWS_SECRET_ACCESS_KEY,
)
bucket = s3.lookup(self.bucket_name)
if not bucket:
return True
key = Key(bucket, "test/%s/%s" % (
self.account_id,
self.id
))
key.delete()
When the code runs, it doesn't fail. But the directory isn't removed. If I add a single file thats in the directory, then it gets removed but not the directory.
I am using Python 2.7 with Boto 2 and can't figure this out
Directories do not actually exist in Amazon S3.
Amazon S3 is an object store, not a filesystem. All objects are stored in a flat structure, but a filename ('Key') can contain slashes to act like a directory.
Therefore, s3://my-bucket/foo/bar.txt is not in the foo directory. Rather, the Key is foo/bar.txt.
This has the interesting property that you can add a file to S3 in a directory that doesn't exist and the directories magically 'appear' to be created:
aws s3 cp foo s3://my-bucket/dir1/dir2/foo
Deleting that object makes the directories magically disappear too, since they really didn't exist.
However, this then raises the question of how directories are 'created' in the AWS Management Console when a user clicks the Create Folder button? The answer is that the console creates a zero-length object to 'hold the place' and make the directory appear to be there:
$ aws s3 ls s3://my-bucket/dir
2017-03-16 22:53:53 0
Here is some sample code that deletes all buckets, and objects within those buckets, where the bucket name ends with -cheese. (Please note, this code doesn't work well with Versioned buckets.)
#!/usr/bin/env python
from boto.s3.connection import S3Connection
conn = S3Connection()
buckets = conn.get_all_buckets()
for b in buckets:
if b.name.endswith("-cheese"):
print "Processing bucket", b.name
objects = b.list()
for o in objects:
print "Deleting object", o.key
b.delete_key(o.key)
print "Deleting bucket", b.name
conn.delete_bucket(b.name)