Scaling Simmetric Chat to Multiple Instances with Redis and pg-boss

One instance is easy. Two are a design decision.

Running a single Simmetric Chat instance is deliberately boring: one server process, one Postgres, and every piece of state that would otherwise be shared lives comfortably in memory. The moment you put a second instance behind a load balancer, all of that in-process state becomes a correctness problem. Rate-limit counters drift apart, so the effective limit multiplies by the instance count. A token revoked on instance A is still accepted on instance B. A streaming user connected to instance B never sees events emitted on instance A. None of these failures announce themselves - they surface later, as "the limit doesn't work" or "logout doesn't log me out everywhere" bug reports.

Simmetric Chat treats horizontal scale as a first-class deployment mode. The multi-instance work, completed across the Redis layer milestone and shipped through v0.22, externalizes exactly those pieces of state so that any instance can serve any request. This article walks through the topology, the five Redis consumers, the pg-boss job queue, the keys that must match across a fleet, and the graceful-degradation behavior that keeps a single-instance deployment fully supported.

The topology: identical, stateless instances

The deployment shape is deliberately unremarkable. N identical server processes sit behind any HTTP load balancer; a shared Postgres holds app data, pgvector, and the pgboss job schema; a shared Redis holds the cross-instance coordination state:

       Load balancer (any HTTP LB - no sticky sessions needed)
           │
   ┌───────┼───────┐
Server 1  Server 2  …  Server N     identical processes, :3000
   └───────┼───────┘
           │
   Shared Postgres          Shared Redis
   app data + pgvector      rate limits, revocation list,
   + pgboss schema          SSE relay, locks, caches

Every instance runs the same code and is stateless from the load balancer's perspective. There is no server-side session map to pin a client to one instance: authentication is a stateless JWT, signed HS256 with a shared JWT_SECRET and validated on every request. Anything that would otherwise live in process memory is externalized to a named home:

• Rate-limit counters → Redis (shared window across the fleet); without Redis, each instance counts independently
• JWT revocation (`jti` list) → Redis; without Redis, revocation no-ops and tokens stay valid until their own expiry
• SSE fan-out → Redis pub/sub relay; without Redis, events reach only clients on the emitting instance
• Distributed locks (reaper schedulers) → Redis redlock; without Redis the lock is a no-op
• Auth and config caches → Redis; without Redis, every read falls through to a database query
• Background jobs → pg-boss, backed by the same shared Postgres

Sticky sessions are not required

The load balancer needs no session affinity for REST traffic. The only long-lived connections are SSE streams, and even those do not require pinning: a client holds a streaming connection to whichever instance accepted it, but events emitted on any other instance still arrive, because each instance publishes SSE events to a Redis channel and every instance re-emits that channel's messages to its own connected clients. The relay is verified end-to-end in a two-instance simulation, including the origin-skip behavior that prevents an instance from re-receiving its own events.

If your balancer offers least-connections routing, prefer it for the SSE path - long-lived connections should be spread by open-connection count rather than round-robin. That is an efficiency choice, not a correctness requirement. Any HTTP load balancer works.

The five Redis consumers

REDIS_URL is optional by design, and the single-instance fallbacks are fully supported behavior, not accidents. Multi-instance, however, requires every instance to point at the same Redis instance (or cluster or sentinel), because five subsystems otherwise silently diverge:

• Rate-limit stores. With a shared Redis store, a 200-requests-per-minute global limit means 200 across the whole deployment. Without it, every instance keeps its own counter and the effective limit becomes N times the configured one - an accident waiting for a spike.
• JWT revocation. The revocation check consults a blacklist that must be global. Without Redis, a logged-out or administratively revoked token keeps working on the other instances until its expiry; "log out everywhere" quietly becomes "log out here."
• SSE fan-out. The cross-instance relay lives on Redis pub/sub. Without it, a user whose stream landed on instance B never sees events emitted on instance A - the failure shows up as mysteriously stalled live updates.
• Distributed locks. Reaper-style maintenance schedulers acquire a cross-instance mutex through redlock so two instances never run the same cleanup concurrently. Without Redis the lock wrapper is a no-op - which is one of the reasons the scheduled jobs themselves moved to pg-boss, whose dedup does not depend on a lock.
• Auth and config caches. System settings and role caches are shared through Redis. Without it, there is no cache: every read is a fresh Postgres query, which shows up as read amplification exactly when traffic grows.

# root .env - identical on every instance
REDIS_URL=redis://redis-host:6379
# TLS variant:
# REDIS_URL=rediss://redis-host:6379

The clients speak standard Redis URLs through ioredis. A production boot with REDIS_URL unset logs a loud warning; treat that warning as the deployment telling you it is running in single-instance mode, whatever the instance count says.

pg-boss: the job queue with no new infrastructure

Background work in Simmetric Chat - archive consistency checks, synthesis reaping, vector cleanup, MCP health checks, MCP reaping, chat-message retention, and upload-draft cleanup - runs on pg-boss, a Postgres-backed job queue. Seven setInterval-based schedulers were migrated to pg-boss scheduled jobs. pg-boss auto-creates its own pgboss schema inside the database you already run for app data, so scaling out adds no new infrastructure component and keeps the whole stack air-gap friendly.

The property that matters most for multi-instance deployments is distributed job dedup: when several instances all reach a scheduled fire time, exactly one instance picks up the job and the others skip on contention. The queue, not a lock wrapper, is the dedup mechanism - one less thing to misconfigure.

Two latency-sensitive pollers intentionally did not migrate. The OCR pipeline reaper and the synthesis pipeline reaper stay on 10-second in-process intervals, because pg-boss cron's minimum granularity is one minute and would make pipeline pickup six times slower. Their multi-instance behavior is documented and safe: both are idempotent claim-then-dispatch cycles that dedupe at the database level with atomic claims, so concurrent instances cannot double-claim the same pending job, and the collector absorbs any redundant dispatch as a no-op.

Graceful shutdown is part of the contract. In-flight jobs drain under a pg-boss grace timeout before the database connection closes, and the whole shutdown sequence runs under an outer time cap. If Postgres is unavailable at boot or goes down mid-run, pg-boss jobs are skipped with a logged error - the server does not crash-loop, and the queue heals when Postgres returns. A queue outage must never take the chat API offline.

Keys that must match across the fleet

Two secrets deserve explicit attention when you grow from one instance to many:

• `JWT_SECRET` - the symmetric signing key for stateless session tokens. The same value on every instance is precisely what makes any-instance-can-serve-any-request true.
• `ENCRYPTION_KEY` - the AES-256-GCM at-rest encryption key (32 bytes), required in production, where the server fails loud at boot if it is missing. Every instance must carry the same value, because a blob encrypted by one instance must decrypt on all of them.

Keeping the two keys separate is itself a multi-instance safety property. Rotating JWT_SECRET is cheap - it just re-issues tokens - while rotating ENCRYPTION_KEY requires a full data re-encryption runbook. The old behavior of deriving the encryption key from the JWT secret coupled the two rotations together; in a fleet, that coupling is a fleet-wide brick waiting for a routine secret rotation.

# generate once, distribute to every instance
openssl rand -hex 32     # JWT_SECRET
openssl rand -base64 32  # ENCRYPTION_KEY

The Postgres connection pool is the practical scaling limiter. Every instance opens a bounded application pool plus a separate pg-boss pool, so sizing the database pool and Postgres max_connections against the instance count is the operator's responsibility. There is no hard upper bound on instance count; start with two and scale as load requires.

What breaks when a dependency is missing

Graceful degradation is designed behavior, worth knowing before capacity planning rather than after an incident:

• Redis absent, one instance. Everything works. SSE simply has nowhere to relay, rate limits are per-process, caches are per-process. This is the supported single-instance default.
• Redis absent, N instances. The five divergences above appear silently - multiplied rate limits, per-instance revocation, broken cross-instance streaming, no-op locks, and hot database reads. The production boot warning is your cue.
• Postgres unavailable. The server boots and serves requests that do not need the queue; the seven pg-boss schedulers skip their runs and retry later; requests that do need the queue fail gracefully with logged errors. A transient outage at boot does not crash-loop the process.

The design stance is consistent: optional dependencies degrade to documented behavior, and a dependency outage degrades a capability rather than the platform.

From one instance to two

Growing the deployment is configuration work, not code work:

1. Point every instance at the same DATABASE_URL, JWT_SECRET, COLLECTOR_SECRET, and ENCRYPTION_KEY.
2. Set REDIS_URL on every instance, pointing at a shared Redis (single instance, cluster, or sentinel).
3. Restart and confirm the boot log shows no single-instance warning.
4. Put the instances behind your load balancer and route health checks at GET /api/health, which reports database, collector, and disk status.

One more multi-instance detail worth knowing: API-key authentication is an O(1) keyed-HMAC lookup, not a bcrypt loop. That change was made precisely so API-key verification stays cheap enough to run on every request across every instance, with no shared cache and no per-instance CPU spikes. It is the kind of quiet decision that makes horizontal scale boring - which is exactly how it should be.

Read the full guide in the docs →