Empresas
Empleos
  • Sobre nosotros
  • Soluciones
    • Publicación de vacantes
      Publica tu vacante y recibe candidatos calificados en 48h.
    • Evaluación de candidatos
      500+ pruebas técnicas y psicológicas, más anti-fraude.
    • Headhunting
      Búsqueda ejecutiva a la medida de principio a fin.
    • Nómina + EOR
      Dispersión de nómina y EOR en más de 15 países de LATAM.
  • Precios
  • Empleos

0

148
Vistas
Comparing _ids in two collections in different databases with 4million documents mongosh

I am using getSibling to achieve this and I am unsure if this is the only way to attempt to not use $nin with writing mongodb shell script. I start out with this

//This runs for a 8-10 mins for 4Million docs
db = db.getSiblingDB("a")
akeys = db.getCollection("colla").aggregate([
    {
        $group: {
            _id: "$_id"
        }
    }
]).toArray()


akeysarray = akeys.map(item => item._id)
['abc','a','d','erdfs',.......'anvdf']

I switch databases and I do the same

db = db.getSiblingDB("b")
bkeys = db.getCollection("collb").aggregate([
    {
        $group: {
            _id: "$_id"
        }
    }
]).toArray()
akeysarray = akeys.map(item => item._id)
['abc','a',.......'anvdf']

Then I loop through with vanilla JavaScript because - it is the only way I can accomplish this in mongosh, but this is my problem area. It takes 5000 elements a minute and it will run at 16 hours or so over night.


difference = [];
arrays = []
size = 250000
// I have to splice the list of comparing arrays by 250000
// because it is very slow
while (bkeysarray.length > 0)
    arrays.push(bkeysarray.splice(0, size))

difference = []


for (var idxo in arrays) {
    for (var idx in arrays[i]) {
        print("checking array set" + idxo + " with index " + idx + " and value " + arrays[idxo][idx])
        var value = arrays[idxo][idx];

        if (value == akeysarray[idxo]) {
            continue;
        }

        if (akeysarray.indexOf(value) == -1) {
            difference.push(value);
        }
    }
}

This will return 28K elements in an array and I want to use this in an $in statement to use the indexing for finding.

Please let me know if there is another way that can achieve a cross database _id comparison that is more performant than this that can only run in mongosh due to environment limitations with the client. I am aware it can be done with a driver in less than few seconds - but I can only run this script in mongosh.

about 4 years ago · Juan Pablo Isaza
Responde la pregunta
Encuentra empleos remotos

¡Descubre la nueva forma de encontrar empleo!

Top de empleos
Top categorías de empleo
Empresas
Publicar vacante Precios Comercial
Legal
Términos y condiciones Política de privacidad
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Recomiéndame algunas ofertas
Necesito ayuda