Scaling Node.js Past 10M Requests/Day: What Actually Matters

Scaling Node.js, the boring version

Most "scaling Node.js" articles are written by people who scaled one service to one million requests per day and assume it generalizes. We've operated several systems past 10 million requests per day, and the lessons aren't glamorous.

This is the playbook we wish we'd had four years ago.

Stop arguing about microservices

The single most expensive mistake we've watched teams make is splitting into microservices before they had to. A monolith with clean module boundaries scales further than most engineers expect, and it's an order of magnitude cheaper to operate.

Our rule: stay monolithic until one of these is true.

• A specific path has different scaling characteristics from the rest (e.g. a heavy ML inference endpoint).
• A specific path has different deployment cadence (e.g. compliance-bound code that must ship monthly while the rest ships daily).
• The team is past 25 engineers and merge contention is real.

If none of those is true, splitting services adds operational cost — service discovery, distributed tracing, deployment orchestration, network failure modes — without buying anything you couldn't get from a clean monorepo.

The actual bottleneck is the database

In every Node.js scaling project we've done, the bottleneck eventually moved to the database. CPU and memory on Node.js services are usually fine because Node is good at I/O concurrency. The problems are:

• Unbounded connection pools melting Postgres
• Single hot index becoming a write bottleneck
• N+1 queries in ORMs that look innocent in code review
• Long-running transactions holding locks during business logic
• Read traffic on the primary instead of a read replica

Practical first moves:

1. PgBouncer or equivalent in front of Postgres, transaction pooling mode, connection cap based on actual database CPU not Node.js intuition.
2. Read replicas with explicit routing. Don't rely on the ORM to decide. Have two clients in code: db.read and db.write. Force every developer to pick.
3. Query timing in production. Pino + a slow-query logger. You want a Slack alert when any query crosses 200ms.
4. EXPLAIN ANALYZE in CI for new queries. We add this as a PR check on schema-touching changes.

Caching, in three layers

Caching saves you in the order: CDN, in-memory, Redis. Skip any layer and you'll regret it.

CDN. Anything that can be cached at the edge should be. This includes API responses for unauthenticated reads. Don't be clever — Cache-Control headers and a CDN that respects them solve 60% of read traffic for free.

In-memory. A small LRU cache (we use lru-cache) in front of Redis catches the long tail of repeated requests within a single process. Sounds trivial; saves 30–50% of Redis traffic in our profiles.

Redis. For shared state and computed values that survive across processes. Use it intentionally — every key needs an owner, a TTL, and a documented invalidation strategy. Untracked Redis keys are how production caches end up serving 6-month-old data on Black Friday.

Async everything that can be async

Synchronous request paths should do the minimum: validate, persist, respond. Anything else — emails, webhook fires, image processing, third-party API calls, search indexing — goes on a queue.

We use BullMQ on Redis for almost everything. Two reasons:

1. Failure isolation. A flaky third-party API can't take down your request path if the call happens in a worker.
2. Backpressure. When traffic spikes, the queue absorbs the burst and workers drain it at sustainable rate. Synchronous code under spike just dies.

Concrete example: a checkout flow we rebuilt for a client used to fire 6 third-party calls inline (tax, fraud, fulfillment, email, SMS, analytics). Median checkout took 1.4 seconds and the p99 was 8 seconds when one provider got slow. After moving everything except tax and fraud to BullMQ workers, median dropped to 220ms and p99 stayed under 600ms.

Observability before clever architecture

The instinct at scale is to design clever systems. The right move is to instrument the system you have until you can see exactly where time is spent.

We standardize on:

• OpenTelemetry tracing, exported to Honeycomb or Tempo. Every request has a trace ID. Every external call is a span.
• Structured logs, JSON format, with the trace ID. Pino is fine.
• RED metrics per endpoint — Rate, Errors, Duration. Grafana dashboards. Alerts on p99, not p50.
• Synthetic checks from outside your network, running every minute against critical paths.

You will not predict your bottlenecks correctly. You will measure them. The team that ships observability first wins the next six months of scaling work.

The pitfalls we've watched repeatedly

A short list of things we've seen take down production:

• Logging too much, then taking down the log pipeline, then having no logs.
• Forgetting to set Node's --max-old-space-size. Default is 1.5GB. Your container has 8GB. Math.
• Writing to a database in a Promise.all loop without bounding concurrency. Connection pool exhaustion at the worst moment.
• Health checks that pass when the service is actually broken. Test the dependency chain, not just the process.
• Trusting a third-party SDK to set timeouts. Always wrap with your own.

The unsexy truth

At 10M requests per day, the difference between a system that works and one that doesn't is rarely architectural genius. It's a team that watches their dashboards, fixes slow queries the week they appear, runs incident reviews honestly, and resists the urge to refactor when they should be instrumenting.

Get that culture right and Node.js will hold up further than the talks at conferences suggest. This is exactly the kind of work our backend and cloud & DevOps teams take on for growth-stage products.