I am benchmarking some simple javascript code and I see some unexpected memory usage in the following example:
function startTimer() {
const time = process.hrtime();
return time;
}
function endTimer(time) {
function roundTo(decimalPlaces, numberToRound) {
return +(Math.round(numberToRound + `e+${decimalPlaces}`) + `e-${decimalPlaces}`);
}
const diff = process.hrtime(time);
const NS_PER_SEC = 1e9;
const result = (diff[0] * NS_PER_SEC + diff[1]); // Result in Nanoseconds
const elapsed = result * 0.0000010;
return roundTo(6, elapsed); // Result in milliseconds
}
SIZE = 10000000
start = startTimer()
let a = Array(SIZE).fill(1)
elapsed = endTimer(start)
const proc = process.memoryUsage()
console.log("rss (MB): ", proc.rss/1000/1000)
console.log("heapTotal (MB): ", proc.heapTotal/1000/1000)
console.log("heapUsed (MB): ", proc.heapUsed/1000/1000)
console.log("elapsed (ms): ", elapsed)
When SIZE = 10MB I get:
[ThinkPad-E14-Gen-2:/home/jj/Documents/repos/tests_javascript]% node --max-old-space-size=4192 example.js
SIZE (MB): 10
rss (MB): 113.87
heapTotal (MB): 84.19
heapUsed (MB): 83.29
elapsed (ms): 47.44
When SIZE = 100MB I get 66x more rss memory and 500x longer execution time:
[ThinkPad-E14-Gen-2:/home/jj/Documents/repos/tests_javascript]% node --max-old-space-size=4192 example.js
SIZE (MB): 100
rss (MB): 7529.71
heapTotal (MB): 4083.26
heapUsed (MB): 4008.59
elapsed (ms): 23135.14
Why is this happening? How do I go about benchmarking where time/memory allocations are going?
You may find this usefull:
The performance penalty may be due the holey array type produced by the Array(SIZE) constructor.
After reading the above articles, you may find it interesting to compare the difference in memory and time between filling 10_000_00 and (say) 50_000_000arrays with this ways:
let a = Array(SIZE).fill(1)
let a = Array.from({ length: SIZE }).fill(1)
let a = [1]; for (let i = 0; i < SIZE; i++) a.push(1);
let a = [1]; for (let i = 0; i < SIZE; i++) a[i] = 1;
let a = new Uint8Array(SIZE).fill(1) // Wow!
Javascript engine optimizes on repeated use of functions. Anyway, here is another way of doing it(not fast enough but proves the just in time compiler / optimizer part):
Try producing the filled array only once, then stringify it, then use JSON.parse on string whenever you need.
let n=1000;
let arr=[];
for(let i=0;i<n;i++) arr.push(1);
const constant=JSON.stringify(arr);
for(let i=0; i<10;i++){
let t = Date.now();
let newArr = JSON.parse(constant);
console.log(Date.now()-t);
}
40ms for 1m array, 450ms for 10m array.