I am using Annoy library that is using mmap() to load some multi-Gb files into RAM memory. The goal of using mmap() is to load the file only once in memory even when different processes need it.
Using docker, I plan to scale with multiple containers executing the same script on the same host. But the multi-Gb file should be loaded only once in RAM (which is why we use mmap()), otherwise my server will explode.
The multi-Gb file is located in a volume mounted on my containers.
But I still need to find a way to share RAM between containers so that I get the advantages of mmap().
I found this article about using the --ipc tag in docker, but I don't know if it applies to my case and how to implement it. Any help welcome.
--ipc is a red herring. For local volumes no action is required, if it is the same file then memory will be shared. I suspect the same for remote volumes, but can't confirm that remote volumes will not multiply mounted.