Redis Cluster and Scaling

The limits of a single instance#

The introduction page explained that Redis runs its core command processing on a single thread — extremely fast, but bounded: one instance is limited to one machine's RAM for the entire dataset, and to roughly what a single core can process. A single, well-sized Redis instance is genuinely enough for most applications — but once the dataset no longer fits in one machine's memory, or write throughput exceeds what one instance can process, scaling further needs either replication (for availability and read scaling) or sharding (for splitting the dataset itself across machines).

Replication for high availability#

Redis replication works conceptually like the MySQL replication covered elsewhere in this track: a replica continuously receives and replays the primary's write stream, staying (asynchronously) in sync.

# On the replica
REPLICAOF primary-host 6379

# Revert a replica back to being a standalone primary
REPLICAOF NO ONE

A replica alone gives read scaling (route read traffic across several replicas) and a warm standby copy of the data, but it doesn't handle failover automatically — if the primary dies, something still has to notice and promote a replica to take its place.

Redis Sentinel: automated failover#

Sentinel is a separate set of Redis processes that monitor a primary and its replicas, and automatically promote a replica to primary if the current primary becomes unreachable — without an operator manually intervening at 3 AM. Sentinel also acts as a discovery layer: clients ask Sentinel "who's the current primary?" rather than hardcoding a single server's address, so a promotion doesn't require reconfiguring every client by hand.

Sentinel 1 ---\
Sentinel 2 ----+---> monitor: Primary, Replica A, Replica B
Sentinel 3 ---/            |
                            X  (primary fails)
                            |
                            v
                     Sentinels agree Replica A should be promoted
                     Replica A becomes the new primary
                     Replica B is reconfigured to replicate from Replica A

Running an odd number of Sentinels (typically 3) matters for the same reason distributed systems generally prefer odd counts: promoting a replica requires a majority vote among Sentinels, and an odd number avoids ties.

Replication plus Sentinel solves availability and read scaling — every replica still holds a full copy of the entire dataset. It does nothing for a dataset that's simply too large, or writes too frequent, for one primary to hold or process at all. That's what Cluster is for.

Redis Cluster: sharding the keyspace#

Redis Cluster splits the dataset itself across multiple nodes, so each node holds only a portion of the total keys rather than a full copy. The keyspace is divided into 16,384 hash slots; every key is deterministically assigned to exactly one slot via CRC16(key) mod 16384, and each node in the cluster owns some subset of those slots.

Node A: slots 0     - 5460   (owns ~1/3 of the keyspace)
Node B: slots 5461  - 10922
Node C: slots 10923 - 16383

A client can send a command to any node in the cluster; if that node doesn't own the relevant key's slot, it responds with a redirect telling the client which node actually does (cluster-aware client libraries handle this automatically, transparently to application code).

Hash tags: keeping related keys on the same slot#

A single-key command works fine no matter which slot it lands in, but a multi-key operation (a transaction, or a command touching several keys at once) requires every key involved to live on the same slot — otherwise Redis Cluster rejects it outright. Hash tags — a {...} portion of a key — force the hashing to consider only what's inside the braces, letting related keys be pinned to the same slot deliberately:

SET user:{42}:profile  "..."
SET user:{42}:sessions "..."
# Both keys hash based on just "42", so they're guaranteed to land on the same slot
# — a multi-key command touching both is now valid in cluster mode.

Replication inside a Cluster#

Each shard in a Redis Cluster is typically itself a small primary-replica set — Node A above might really be "Node A primary + one or two Node A replicas" — combining sharding (for horizontal scale) with per-shard replication (for HA), so a single node failing doesn't lose that shard's slice of the keyspace entirely, just triggers a promotion within that shard the same way Sentinel would for a non-clustered setup.

Choosing the right architecture#

The practical guidance: start with a single instance (optionally with a replica or two for safety) for the overwhelming majority of applications, add Sentinel once automated failover genuinely matters operationally, and reach for Cluster only once there's a real, measured need to scale beyond what one node's RAM and single-threaded throughput can handle — not as a default architecture chosen up front "to be safe."

Common mistakes#

• Reaching for Redis Cluster before actually needing horizontal scale, taking on real operational complexity (hash tags, redirect-aware clients, per-shard replication) for a dataset that would have fit comfortably on one well-sized instance.
• Running a multi-key transaction or command across keys that don't share a hash tag, and having it fail in cluster mode — this works fine on a single instance and only surfaces once cluster mode is introduced, making it an easy thing to miss until production.
• Confusing replication (every replica holds a full copy of the same data, for HA and read scaling) with Cluster sharding (each node holds a different slice of the data, for horizontal scale) — they solve different problems and one doesn't substitute for the other.
• Running an even number of Sentinels, risking a tied vote during a failover decision instead of a clean majority.