I use cartesian product to generate a very large amount of permutations which are then passed to some function. I created a generator
function* generateCartesianProduct(input) {
yield ...
}
const items = generateCartesianProduct(['a', 'b', 'c'], ['e, f, g'], ...)
in my case, items is size of 10m+.
I wish to pass this to node worker threads using node-worker-threads-pool library.
const pool = new StaticPool({
size: 8,
task: filePathToMyWorker
});
so I try to pass items to worker and process them
for (const item of items) {
(async () => {
try {
const output = await pool.exec(value, 1000);
// do something with output
} catch (e) {
console.error(e);
}
})();
}
This does not work, because it seems that it wants to first execute for-loop and then pass data to pool. But my wish is to iterate through items and pass them to worked pool, so I can iterate and concurrently execute my worker.
Any ideas how to make this work?