I have an arrow function which get two arrays as parameters then on certain condition i push items from second array to first array
The problem is that i have big amount of data that i should to push and res.send cant wait while loop is over and send me error about memory JavaScript heap out of memory
How i can wait while my loop is over and make loop to work faster? I just try with async/await, promiseAll, for await
let mergeById = async(arr1, arr2) => {
arr1.map(async(item) => {
item.properties = [];
await arr2.map(async(prop) => {
if (item.item_id == prop.item_id) {
await item.properties.push(prop);
}
})
})
return arr1
}
async someFunction(req, res) {
arr1 = dataItems(); //6000 items
arr2 = dataProps(); //60000 items
let result = await mergeById(arr1, arr2);
res.json({
data: result
})
}You have map inside map, therefore for each item in array1 it will iterate over whole array 2. You have encountered O(n^2) complexity and as you can see - even with 66 000 items (which is not that much for computer on its own), you are encountering huge impact. Because you have to do 6 000 * 60 000 = 360,000,000 iterations
There is a solution. You have to do some preprocessing - create new Map() for array2, iterate over all items and save item_id as a key with value true. Something like array2Map.set(prop.item_id, prop)
Then change your cycle to check your map
arr1.map(async(item) => {
item.properties = [];
const prop = array2Map.get(item.item_id);
if (prop) {
item.properties.push(prop)
}
Now you have O(n * log n) complexity (well its almost O(n) because of pseudo-linear complexity of hashmaps, but thats different story, not that important now). And thats good enough for your use-case.
Also there is another problem with usage of the await and async. Just remove all this usage from all the code you have presented. It does not do anything useful, but actually breaks few things.
.map function runs synchronously no matter if you put async function as a parameter. It will just create an array of promises. Unless you know exactly what you are doing, my tip is that for array functions (map, foreEach, reduce etc.) never ever use async function as parameter - as it will mostly not work as you expect.