Empresas
Empleos
  • Sobre nosotros
  • Soluciones
    • Publicación de vacantes
      Publica tu vacante y recibe candidatos calificados en 48h.
    • Evaluación de candidatos
      500+ pruebas técnicas y psicológicas, más anti-fraude.
    • Headhunting
      Búsqueda ejecutiva a la medida de principio a fin.
    • Nómina + EOR
      Dispersión de nómina y EOR en más de 15 países de LATAM.
  • Precios
  • Empleos

0

207
Vistas
How come multiprocessing.managers.BaseManager is this slow on linux?

I'm trying to diagnose a performance problem with the server created by multiprocessing.managers.BaseManager. The problem boils down to this:

server.py:

import multiprocessing
import multiprocessing.managers

auth_key = b'12345678901234567890'
port = 32001

shared_values = {}

def _shared_values():
    global shared_values
    return shared_values

class SharedStateManager(multiprocessing.managers.BaseManager): pass

SharedStateManager.register('SharedValues', callable=_shared_values)

multiprocessing.current_process().authkey = auth_key
server = SharedStateManager(address=('', port), authkey=auth_key)

server.get_server().serve_forever()

client.py:

import time
import multiprocessing
import multiprocessing.managers

auth_key = b'12345678901234567890'
port = 32001

class SharedStateManager(multiprocessing.managers.BaseManager): pass

SharedStateManager.register('SharedValues')

multiprocessing.current_process().authkey = auth_key
ssm = SharedStateManager(address=('', port), authkey=auth_key)

cl1 = time.time_ns()
ssm.connect()
cl2 = time.time_ns()
shared_values = ssm.SharedValues()
cl3 = time.time_ns()
print('2-1: {} ms'.format((cl2 - cl1) / (10 ** 6)))
print('3-2: {} ms'.format((cl3 - cl2) / (10 ** 6)))

When I run the client code on a Debian buster instance running on Intel 8th-gen 2.3GHz i5 with 4 cores, I get:

2-1: 42.124515 ms
3-2: 132.311978 ms

Which is very slow. The python version is 3.7.3 although I tried different version with docker containers and got similar results.

But if I run the same code on a Macbook (macos 11.6) that has a similar CPU, I get:

2-1: 1.950359 ms
3-2: 3.264376 ms

What might be the cause of such dramatic response time difference?

Edit: The target machine isn't broken or anything. It's running a couple of QEMU virtual machines plus a few docker containers. So far the network latency did not happen on for instance, a sanic based HTTP server or other containerized socket servers. Copying a file over a network share will max out the LAN at about 100MB/s. So it's happening to Python's multiprocessing server in particular.

Edit: I also found this question which sort of exhibited the same behavior regarding a Python multiprocessing program running much faster on mac than linux.

Edit: Solved! I thought I was running Debian bullseye but it turned out I was running buster. After exhausting all other options I gave release upgrade a try. And the latency went away after the upgrade.

over 4 years ago · Santiago Trujillo
Responde la pregunta
Encuentra empleos remotos

¡Descubre la nueva forma de encontrar empleo!

Top de empleos
Top categorías de empleo
Empresas
Publicar vacante Precios Comercial
Legal
Términos y condiciones Política de privacidad
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Recomiéndame algunas ofertas
Necesito ayuda