Business
Jobs
  • About Us
  • Solutions
    • Job Postings
      Post your job and receive qualified candidates in 48h.
    • Candidate Assessments
      500+ technical and psychological tests, plus anti-fraud.
    • Headhunting
      Tailor-made executive search from start to finish.
    • Payroll + EOR
      Payroll dispersal and EOR across 15+ LATAM countries.
  • Pricing
  • Jobs

0

184
Views
Download specific folder and all of its files recursively from S3 bitbucket using boto

I know how to download files recursively using aws cli

aws s3 cp {s3Directory} {localDirectory} --recursive

But i want to use Python Boto and below is my code. I want to download folders and all of its files ('Daily/2017/02/15/') from bucket 'news-live'. When i execute below command, all folders created but there is no file downloaded.

def main():
DOWNLOAD_LOCATION_PATH = "/Users/test/"
BUCKET_NAME = 'news-live'
conn  = boto.connect_s3(AWS_ACCESS_KEY_ID, AWS_ACCESS_SECRET_KEY)
bucket = conn.get_bucket(BUCKET_NAME)

#goto through the list of files
bucket_list = bucket.list('Daily/2017/02/15/')

for l in bucket_list:
    key_string = str(l.key)
    print("bucket key", l.key)
    s3_path = DOWNLOAD_LOCATION_PATH + key_string
    try:
        print ("Current File is ", s3_path)
        l.get_contents_to_filename(s3_path)
    except (OSError,S3ResponseError) as e:
        pass
        # check if the file has been downloaded locally  
        if not os.path.exists(s3_path):
            try:
                os.makedirs(s3_path)
            except OSError as exc:
                # let guard againts race conditions
                import errno
                if exc.errno != errno.EEXIST:
                    raise

I don't know how to get this work.

over 4 years ago · Santiago Trujillo
Answer question
Find remote jobs

Discover the new way to find a job!

Top jobs
Top job categories
Business
Post vacancy Pricing Sales
Legal
Terms and conditions Privacy policy
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Show me some job opportunities
There's an error!