Business
Jobs
  • About Us
  • Solutions
    • Job Postings
      Post your job and receive qualified candidates in 48h.
    • Candidate Assessments
      500+ technical and psychological tests, plus anti-fraud.
    • Headhunting
      Tailor-made executive search from start to finish.
    • Payroll + EOR
      Payroll dispersal and EOR across 15+ LATAM countries.
  • Pricing
  • Jobs

0

384
Views
Python multiprocessing within Flask request with Gunicorn + Nginx

I want to build a service that will be able to handle:

  • a low volume of requests
  • a high compute cost for each request
  • but where the high compute cost can be parallelized.

My understanding of a pre-fork server is that something like the following happens:

  1. server starts
  2. Gunicorn creates multiple OS processes, also called workers, ready to accept requests
  3. request comes in. Nginx forwards to Gunicorn. Gunicorn sends to one of the workers.

What I want to understand is what happens if, in my Flask code, when handling the request, I have this:

from multiprocessing import pool as ProcessPool
with ProcessPool(4) as pool:
    pool.map(some_expensive_function, some_data)

In particular:

  1. Will additional OS processes be started? Will the speedup be what I expect? (I.e., similar to if I ran the ProcessPool outside of a Flask production context?) If Gunicorn created 4 web workers, will there now be 7 OS processes running? 9? Is there a risk of making too many? Does Gunicorn assume that each worker will not fork or does it not care?
  2. If a web-worker dies or is killed after starting the ProcessPool, will it be closed by the context manager properly?
  3. Is this a sane thing to do? What are the alternatives?
over 4 years ago · Santiago Trujillo
Answer question
Find remote jobs

Discover the new way to find a job!

Top jobs
Top job categories
Business
Post vacancy Pricing Sales
Legal
Terms and conditions Privacy policy
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Show me some job opportunities
There's an error!