Empresas
Empregos
  • Sobre nós
  • Soluções
    • Publicação de vagas
      Publique sua vaga e receba candidatos qualificados em 48h.
    • Avaliações de candidatos
      Mais de 500 testes técnicos e psicológicos, mais anti-fraude.
    • Headhunting
      Busca executiva personalizada do início ao fim.
    • Folha de Pagamento + EOR
      Dispersão de folha e EOR em mais de 15 países da LATAM.
  • Preços
  • Empregos

0

330
Visualizações
Maximum number of concurrent tasks in 1 DPU in AWS Glue

A standard DPU in AWS Glue comes with 4 vCPU and 2 executors. I am confused about the maximum number of concurrent tasks that can be run in parallel with this configuration. Is it 4 or 8 on a single DPU with 4vcpu and 2 executors?

over 4 years ago · Santiago Trujillo
1 Respostas
Responde à pergunta

0

I had a similar discussion with the AWS Glue support team about this, I'll share with you what they told me about Glue Configuration. Take in example the Standard and the G1.X configuration.

Standard DPU Configuration:

  • 1 DPU reserved for MasterNode
  • 1 executor reserved for Driver/ApplicationMaster
  • Each DPU is configured with 2 executors
  • Each executor is configured with 5.5 GB memory
  • Each executor is configured with 4 cores

G.1X WorkerType Configuration:

  • 1 DPU added for MasterNode
  • 1 DPU reserved for Driver/ApplicationMaster
  • Each worker is configured with 1 executor
  • Each executor is configured with 10 GB memory
  • Each executor is configured with 8 cores

If we have for example a Job with Standard Configuration with 21 DPU means that we have:

  • 1 DPU reserved for Master
  • 20 DPU x 2 = 40 executors
  • 40 executors - 1 Driver/AM = 39 executors

Which we then end up with a total amount of 156 cores. Meaning, your job has 156 slots for execution. If for example you read files from S3 that means that you will be able to accept 156 input files in parallel.

Hope it helps.

over 4 years ago · Santiago Trujillo Relatório
Responde à pergunta
Encontrar trabalhos remotos

Descubra a nova forma de encontrar um emprego!

melhores empregos
Principais categorias de trabalho
Empresas
Postar vaga Preços Comercial
Jurídico
Termos e Condições Política de privacidade
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Recomende algumas ofertas para mim
Preciso de ajuda