NOA Labs

Performance & Clustering

Using the cluster module for multi-core scaling, profiling basics, and common performance bottlenecks.

Node's single-thread limit, and what clustering does about it

Async Node.js covered how Node handles many concurrent I/O-bound operations well on a single thread — but that single thread is still just one thread, meaning it can only use one CPU core at a time. A Node process running on an 8-core machine leaves seven cores completely idle unless something explicitly puts them to work. The built-in cluster module does exactly that: it forks multiple copies of your process — one per CPU core, typically — with a primary process load-balancing incoming connections across them.

The cluster module

Javascript
import cluster from 'cluster';
import os from 'os';
import express from 'express';

if (cluster.isPrimary) {
  const cpuCount = os.cpus().length;
  console.log(`Primary ${process.pid} starting ${cpuCount} workers`);

  for (let i = 0; i < cpuCount; i++) {
    cluster.fork();
  }

  // A worker crashing shouldn't quietly reduce your capacity forever — replace it
  cluster.on('exit', (worker, code, signal) => {
    console.log(`Worker ${worker.process.pid} died (${signal || code}) — forking a replacement`);
    cluster.fork();
  });

} else {
  // This code runs once per worker — an ordinary Express app, unaware it's one of several
  const app = express();

  app.get('/', (req, res) => {
    res.json({ handledBy: process.pid });
  });

  app.listen(3000, () => console.log(`Worker ${process.pid} listening`));
}

Every worker is a completely separate OS process with its own memory and its own V8 instance — nothing is shared between them automatically. The primary process listens on port 3000 and hands off each incoming connection to one of the workers (round-robin on most platforms), which is why every worker can independently call app.listen(3000, ...) without a "port already in use" error — the primary owns the actual socket.

Bash
curl http://localhost:3000/
# {"handledBy":52104}
curl http://localhost:3000/
# {"handledBy":52117}

Repeated requests get handled by different worker PIDs, confirming the load is genuinely spread across processes — and therefore across CPU cores — rather than queuing up behind one busy thread.

What clustering does and doesn't fix

Clustering multiplies throughput for CPU-bound work and adds resilience (one worker crashing doesn't take the whole server down), but it does not make any single request faster, and it does not give workers shared in-memory state:

Single process Clustered (N workers)
CPU cores used 1 Up to N
Requests handled concurrently under CPU load Limited by the one thread Spread across N processes
In-memory cache/state Naturally shared everywhere Not shared — each worker has its own copy
One process crashes Whole server is down Primary can fork a fresh replacement; others keep serving
A single request's own latency Unaffected by clustering either way Unaffected by clustering either way

That "not shared" row matters in practice: an in-memory rate limiter, session store, or cache built as a plain JavaScript object works fine in a single process and silently breaks under clustering, because each worker enforces its limit (or serves its cache) independently, with no visibility into what the others are doing. Shared state across workers needs an external store — Redis is the standard choice — not a bigger Map in process memory.

Profiling basics

Before optimizing anything, measure where time is actually going — Node's built-in --prof flag records a low-overhead CPU profile with no extra dependencies:

Bash
node --prof server.js
# ... exercise the app under load, then stop it ...

node --prof-process isolate-0x*-v8.log > profile.txt

profile.txt breaks down time spent by function, showing whether the hot path is your own code, a dependency, or V8's garbage collector. For a friendlier, flame-graph view of the same kind of data, clinic.js (a popular third-party tool) wraps this into an interactive HTML report:

Bash
npx clinic flame -- node server.js
# exercise the app under load, then stop it — an HTML flame graph opens automatically

process.hrtime.bigint() is the simplest tool of all for a quick, targeted measurement of one specific function without any external tooling:

Javascript
const start = process.hrtime.bigint();
expensiveOperation();
const durationMs = Number(process.hrtime.bigint() - start) / 1_000_000;
console.log(`expensiveOperation took ${durationMs.toFixed(2)}ms`);

Common bottlenecks

Symptom Likely cause Fix
Whole server feels sluggish under any load, even for unrelated requests CPU-bound synchronous code blocking the event loop (heavy computation, a huge JSON.parse, a pathological regex) Break the work up, move it to a worker thread, or fix the regex
Memory usage climbs steadily and never comes back down A memory leak — most often a cache or listener list that grows forever with nothing ever removing old entries Cap cache size, remove listeners when no longer needed, profile heap snapshots
Latency scales badly as request volume grows, even though each query is individually fast The "N+1" problem — one query per item in a loop instead of one batched query Batch the queries, or use a data-loader pattern to coalesce them
One slow, unrelated route seems to slow down every other route too Synchronous or CPU-heavy work in that one route blocking the single thread for everyone Same as the first row — nothing in Node is free from this unless it's genuinely async

The unifying theme: because Node runs your JavaScript on one thread, any single piece of synchronous, CPU-heavy code — not I/O, which is handed off to libuv — blocks that thread for every other request in flight, not just the one that triggered it. Clustering helps by spreading load across processes, but a single worker can still be brought to its knees by one CPU-bound request if the underlying code itself isn't fixed.

Common mistakes

  • Building an in-memory cache, session store, or rate limiter as a plain object or Map and then clustering the app, discovering that each worker enforces its own separate copy instead of the shared behavior expected — move genuinely shared state to Redis or another external store before clustering.
  • Assuming clustering makes an individual slow request faster — it only lets more requests be handled concurrently across cores; a single CPU-bound request is exactly as slow on a clustered worker as it was unclustered.
  • Optimizing code based on a guess about what's slow instead of profiling first — --prof, clinic.js, or targeted hrtime measurements routinely reveal the actual bottleneck is somewhere unexpected (serialization, a synchronous regex, GC pressure), not the code someone assumed was the problem.
  • Not handling a worker's 'exit' event in the primary process — a crashed worker quietly reduces total capacity forever instead of being replaced, until eventually every worker has died one by one.

Interview questions

Q: Why does Node.js need the cluster module to use more than one CPU core? Node runs JavaScript on a single main thread, so one process can only ever use one CPU core no matter how much I/O concurrency it handles well. The cluster module forks multiple independent OS processes — typically one per core — with a primary process load-balancing incoming connections across them, letting the application actually use every core on the machine.

Q: If you cluster a Node app, does an in-memory cache still work the same way? No — clustering creates completely separate OS processes, each with its own memory, so an in-memory cache, session store, or counter is no longer shared across workers; each one keeps its own independent copy with no visibility into the others. Any state that genuinely needs to be shared across workers has to live in an external store like Redis instead of a plain in-process object.