Empresas
Empregos
  • Sobre nós
  • Soluções
    • Publicação de vagas
      Publique sua vaga e receba candidatos qualificados em 48h.
    • Avaliações de candidatos
      Mais de 500 testes técnicos e psicológicos, mais anti-fraude.
    • Headhunting
      Busca executiva personalizada do início ao fim.
    • Folha de Pagamento + EOR
      Dispersão de folha e EOR em mais de 15 países da LATAM.
  • Preços
  • Empregos

0

389
Visualizações
Python multiprocessing within Flask request with Gunicorn + Nginx

I want to build a service that will be able to handle:

  • a low volume of requests
  • a high compute cost for each request
  • but where the high compute cost can be parallelized.

My understanding of a pre-fork server is that something like the following happens:

  1. server starts
  2. Gunicorn creates multiple OS processes, also called workers, ready to accept requests
  3. request comes in. Nginx forwards to Gunicorn. Gunicorn sends to one of the workers.

What I want to understand is what happens if, in my Flask code, when handling the request, I have this:

from multiprocessing import pool as ProcessPool
with ProcessPool(4) as pool:
    pool.map(some_expensive_function, some_data)

In particular:

  1. Will additional OS processes be started? Will the speedup be what I expect? (I.e., similar to if I ran the ProcessPool outside of a Flask production context?) If Gunicorn created 4 web workers, will there now be 7 OS processes running? 9? Is there a risk of making too many? Does Gunicorn assume that each worker will not fork or does it not care?
  2. If a web-worker dies or is killed after starting the ProcessPool, will it be closed by the context manager properly?
  3. Is this a sane thing to do? What are the alternatives?
over 4 years ago · Santiago Trujillo
Responde à pergunta
Encontrar trabalhos remotos

Descubra a nova forma de encontrar um emprego!

melhores empregos
Principais categorias de trabalho
Empresas
Postar vaga Preços Comercial
Jurídico
Termos e Condições Política de privacidade
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Recomende algumas ofertas para mim
Preciso de ajuda