I am trying to anaylze more than 100k objects in an array to find patterns. I built some nest for loop to run over all the array items.
// const allPairs = [...] array with more than 100k objects
const patterns = [];
for (let i = -5; i < 5; i++) {
for (let j = -5; j < 5; j++) {
for (let k = -5; k < 5; k++) {
for (let l = -20; l < 20; l++) {
patterns.push({
pairs: allPairs.filter((pair) => {
return (
pair.varA > pair.value * i &&
pair.varB > pair.value * j &&
pair.varC > pair.value * k &&
pair.average > l
);
}),
name: `Pattern A ${l}-${i}-${j}-${k}`,
});
patterns.push({
pairs: allPairs.filter((pair) => {
return (
pair.varA < pair.value * i &&
pair.varB < pair.value * j &&
pair.varC < pair.value * k &&
pair.average < l
);
}),
name: `Pattern B ${l}-${i}-${j}-${k}`,
});
}
}
}
}
Basically I'm trying filter my array in order to find which pattern has the more objects.
This is always generating.
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
I did try to increase node memory export NODE_OPTIONS="--max-old-space-size=5120" but it fails as well.
Objects look like this
{
id: 111339,
symbol: 'HELLO',
tested: true,
successful: false,
average: 0.010298,
value: 1.4,
varA: 0.51118,
varB: -0.1545,
varC: -0.1547,
varD: -0.3592,
varE: 0.51118,
varF: -0.1545,
varG: -0.1547,
varH: -0.3592,
varI: 0.51118,
varJ: -0.1545,
createdAt: 2022-06-10T15:30:09.170Z,
updatedAt: 2022-06-10T16:29:08.324Z
}
Any recommendations to make this more efficient?