I am processing a heavy array of objects on a node app, and at some point I want to remove two properties from all objects in the array.
At the same time I am measuring the node memory impact on rss (Residente Set Size).
What I am finding is that just the act of deleting them consumes a lot of memory.
Example, it's actually a lot bigger and with lots of objects. File size of json is 200MB.
[
{
keep: 'foo',
prop1: 'remove',
prop2: 'remove'
},...
]
This consumes the most from 500MB goes to 1000MB
const clean = original.map((obj) => {
delete obj.prop1
delete obj.prop2
return obj
})
This also consumes a lot, also around 1000MB
original.forEach((obj) => {
delete obj.prop1
delete obj.prop2
})
This comsumes the least, goes to around 650MB
const clean = original.map(({ prop1, prop2, ...obj }) => obj)
But if I do not delete them at all then it does not consume anymore than the original 500MB. What is going on? Should not removing properties make the memory lighter?
You can use either of the .map() calls you provided (they are faster than .forEach()).
When you've completed the deletion, simply set the original variable to null. This will allow the JS engine to mark this variable for Garbage Collection and it will be removed from memory. You won't see the result immediately as we have no idea when GC will run and cannot control it. But the memory will be reclaimed very quickly.
Like this:
const clean = original.map((obj) => {
delete obj.prop1
delete obj.prop2
return obj
})
original = null;
const clean = original.map(({ prop1, prop2, ...obj }) => obj);
original = null;
The issue here is that we as web devs don't have any control over memory consumption other than what I have proposed. Internally the V8 engine decides how things are put together in memory and all memory management issues are relegated to it.