I recently learned about the cluster module in node, and wanted to give it a spin. I have a super basic express app that look like this.
const express = require('express');
const app = express();
const cluster = require('cluster');
const os = require('os');
const numCores = os.cpus().length;
app.get('/', (req, res) => {
res.status(200).json({ success: true })
});
if (cluster.isMaster) {
for (let i = 0; i < numCores; i++) {
cluster.fork();
}
cluster.on('exit', (worker, code, signal) => {
console.log(`worker ${worker.process.pid} died`);
console.log("Let's fork another worker!");
cluster.fork();
});
} else {
app.listen(1337, () => console.log('server is running on port 1337'));
}
I am also using the Artillery load testing tool to load test this basic node app.
The problem is that whether I use the cluster module or not, by the time my load test is done running, I have the same amount of failed requests.
This is the output of my test
Scenarios launched: 30000 Scenarios completed: 23145 Requests completed: 23145 Mean response/sec: 426.14 Response time (msec): min: 0 max: 449 median: 1 p95: 4 p99: 35 Scenario counts: simple request: 30000 (100%) Codes: 200: 23145 Errors: ETIMEDOUT: 6855
Why do I have 6855 ETIMEDOUT errors even when I am using the cluster module?
This is the exact same amount of errors I have when not using the cluster module.
You probably just need to define handler inside the child:
const express = require('express');
const cluster = require('cluster');
const os = require('os');
const numCores = os.cpus().length;
if (cluster.isMaster) {
for (let i = 0; i < numCores; i++) {
cluster.fork();
}
cluster.on('exit', (worker, code, signal) => {
console.log(`worker ${worker.process.pid} died`);
console.log("Let's fork another worker!");
cluster.fork();
});
} else {
const app = express();
app.get('/', (req, res) => {
let result = -1;
console.time();
for (let i = 0; i < 2 ** 28; i += 1) {
result += i;
}
console.timeEnd();
res.status(200).json({ result });
});
app.listen(1337, () => console.log('server is running on port 1337'));
}
I would put the const app = express() inside as well.
I think defining handler before you fork cause the problem.
SEE: Using cluster in an Expressjs app
I think the situation is your server implementation's bottleneck is not the processing power, it's somewhere else like IO or memory.
If you add some logic in the handler (see the script updated above), when there is only 1 worker, the CPU utilization will max out at about 25% if you have a quad core machine.
I don't have Artillery so I use the following script to test locally.
const fetch = require('node-fetch');
const CONCURRENT_REQUEST = 8;
let numberOfRequest = 100 - CONCURRENT_REQUEST;
let failed = 0;
const ping = () => (
fetch('http://localhost:1337')
.then(r => r.json())
.then(({ success }) => {
if (!success) {
throw new Error('oops');
}
})
.catch((e) => {
// console.error(`index ${i}`, e);
failed += 1;
})
.then(() => {
numberOfRequest -= 1;
if (numberOfRequest > 0) {
return ping();
}
})
);
const requests = new Array(CONCURRENT_REQUEST).fill(0).map((_, i) => (
ping()
));
Promise.all(requests).then(() => console.log('done. failed: ', failed));
With only 1 worker:
Minutes : 1
Seconds : 1
Milliseconds : 199
TotalDays : 0.000708324185185185
TotalHours : 0.0169997804444444
TotalMinutes : 1.01998682666667
TotalSeconds : 61.1992096
TotalMilliseconds : 61199.2096
With 4 workers:
Seconds : 17
Milliseconds : 249
Ticks : 172490478
TotalDays : 0.000199641756944444
TotalHours : 0.00479140216666667
TotalMinutes : 0.28748413
TotalSeconds : 17.2490478
TotalMilliseconds : 17249.0478