Business
Jobs
  • About Us
  • Solutions
    • Job Postings
      Post your job and receive qualified candidates in 48h.
    • Candidate Assessments
      500+ technical and psychological tests, plus anti-fraud.
    • Headhunting
      Tailor-made executive search from start to finish.
    • Payroll + EOR
      Payroll dispersal and EOR across 15+ LATAM countries.
  • Pricing
  • Jobs

0

305
Views
AWS Batch vs AWS CodeBuild

I am very new in AWS and when I was searching something to download a code from GitHub (a python project), run it, and save the output in s3 the first service that I found was CodeBuild.

So I implement this kind of workflow using CodeBuild.

But now I have seen that AWS have a service called AWS Batch and I am wondering if I should migrate my arquitecture to AWS Batch.

Can you explain which one - AWS CodeBuild or AWS Batch - is more suitable with my case? When use AWS Batch instead of AWS CodeBuild?

Thank very much.

over 4 years ago · Santiago Trujillo
2 answers
Answer question

0

TLDR Summary: AWS Codebuild is the nicer choice for simple jobs.


My (reverse) experience...

I needed to run a simple job that pulls data from external api, read/write to external database, and generate a CSV report.

The job takes ~1 hour to run, so AWS Lambda is out of the picture.

After some googling, I found AWS Batch and decided to give Creating a Simple “Fetch & Run” AWS Batch Job a try.

The required steps to this "simple" job working:

  • Build a Docker image with the fetch & run script
  • Create an Amazon ECR repository for the image
  • Push the built image to ECR
  • Create a simple job script and upload it to S3
  • Create an IAM role to be used by jobs to access S3
  • Configure a compute environment
  • Create a job queue
  • Create a job definition that uses the built image
  • Submit and run a job that execute the job script from S3

After spending the time to create all these resources, it did not work out of the box. I found myself debugging random things I shouldn't have to debug such as:

  • Dockerfile
  • entrypoint script
  • ECS cluster
  • EC2 instance and autoscaling group

After failing to find simple practical examples, and realizing the amount of effort required, I decided to explore other solutions.

I stumbled onto Using AWS CodeBuild to execute administrative tasks and this post.

I've used AWS Codebuild in the past for CI/CD pipeline, and thought "what the hell, lets give it a try". In a shorter amount of time, I was able to get a "codebuild job" running on a cloudwatch scheduler and codebuild slack notifications added, with less effort:

  • Connect build project to your source code
  • Select a runtime environment
  • Create IAM role
  • Create a buildspec.yml and add runtime commands

One major advantage is that CodeBuild runs tasks in a full-blown Linux environment.

Drawbacks:

  • Max execution time of 8 hours

AWS Codebuild was much easier to get working for my simple job.

Sorry for the long post, just wanted to share my experience with these 2 services.

over 4 years ago · Santiago Trujillo Report

0

AWS Batch is used highly parallel computations, e.g., processing large number of images at the same time:

AWS Batch enables you to run batch computing workloads on the AWS Cloud. Batch computing is a common way for developers, scientists, and engineers to access large amounts of compute resources, and AWS Batch removes the undifferentiated heavy lifting of configuring and managing the required infrastructure, similar to traditional batch computing software.

Thus its not suited for what you are trying to use it. CodeBuild is better choice, based on your description.

over 4 years ago · Santiago Trujillo Report
Answer question
Find remote jobs

Discover the new way to find a job!

Top jobs
Top job categories
Business
Post vacancy Pricing Sales
Legal
Terms and conditions Privacy policy
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Show me some job opportunities
There's an error!