Sharding in MongoDB

Why shard#

The introduction page mentioned that horizontal write scaling is a first-class, built-in MongoDB feature — this page is that feature. Sharding splits a collection's documents across multiple servers (shards), so the dataset's total size and total write throughput are no longer bounded by what one server (or one replica set) can hold and process. It's the tool for once a single, well-indexed, well-modeled replica set is genuinely no longer enough — not a default architecture to start with.

The moving parts of a sharded cluster#

A sharded MongoDB deployment has three kinds of components working together:

• Shards — each one holds a portion of the total data, and each is typically its own replica set, so an individual shard also has its own high-availability failover independent of sharding itself.
• Config servers — store the cluster's metadata: which ranges of data live on which shard.
• mongos routers — the entry point the application actually connects to. A mongos doesn't store data itself; it consults the config servers and routes each query to whichever shard(s) actually hold the relevant data.

Application
     |
     v
  mongos router  <---- consults config servers for cluster metadata
     |
     +----> Shard A (a replica set)
     +----> Shard B (a replica set)
     +----> Shard C (a replica set)

The shard key#

Sharding a collection requires choosing a shard key — one or more fields whose value determines which shard a document lives on:

sh.enableSharding("shop");

sh.shardCollection("shop.orders", { userId: 1 });

MongoDB divides the shard key's overall range into chunks, distributes those chunks across the available shards, and automatically migrates chunks between shards over time (the balancer) to keep the data distribution roughly even as the collection grows.

Choosing a good shard key#

This is the actual design problem sharding presents, and getting it wrong doesn't cause an error — it causes a cluster that technically works but scales badly, which is a much harder problem to notice early.

High cardinality and even distribution#

A shard key needs enough distinct values to actually spread data across every shard. A boolean flag, or any field with only a handful of possible values, can put an entire shard's worth of data behind just one or two of those values — most shards sitting nearly idle while one or two absorb everything.

Avoiding monotonically increasing keys#

A field that only ever increases — an auto-generated ObjectId, a plain incrementing counter, or a timestamp — is a common but risky choice as the sole shard key. Since new values are always at the high end of the range, every new document lands in whichever chunk currently owns the highest range of values, all on the same shard — a "hot chunk" that absorbs 100% of new writes regardless of how many total shards exist, defeating the entire point of sharding for write scaling.

// Risky as a sole shard key: every new order's timestamp is higher than the last,
// so every new write lands on whichever shard owns the current high end of the range
sh.shardCollection("shop.orders", { createdAt: 1 });

Query isolation: targeted vs. scatter-gather#

A query that includes the shard key's value can be routed by mongos directly to the one (or few) shard(s) that could possibly hold a match — a targeted query. A query that omits the shard key entirely has to be sent to every shard, with mongos merging the results — a scatter-gather query, far more expensive as the cluster grows, since every shard does work regardless of how relevant its portion of the data actually is.

// Targeted: mongos knows exactly which shard owns userId 42
db.orders.find({ userId: 42 });

// Scatter-gather: no shard key in the filter, every shard must be queried
db.orders.find({ status: "pending" });

This is why userId is a reasonable shard key choice for an orders collection specifically because most real queries against it ("this user's orders") already filter on userId — the shard key choice and the collection's actual query patterns need to be considered together, not independently.

Hashed shard keys#

When the best natural candidate for a shard key is monotonically increasing (like _id or a timestamp), a hashed shard key trades away range-query locality for guaranteed-even write distribution, by hashing the key's value before deciding which chunk it belongs to:

sh.shardCollection("shop.events", { _id: "hashed" });

Writes are now spread evenly regardless of the underlying field's ordering, since a hash scrambles any monotonic pattern — but a query needing a range of the original values (createdAt between two dates, for instance) can no longer be routed to a contiguous set of shards, since consecutive original values now hash to essentially random chunks; it becomes a scatter-gather query across the whole cluster instead.

Compound shard keys#

A compound shard key can combine a coarser, lower-cardinality field with a high-cardinality one, aiming for both reasonable distribution and some query targeting:

sh.shardCollection("shop.orders", { region: 1, userId: 1 });

Common mistakes#

• Choosing a monotonically increasing field (a timestamp, an auto-incrementing id) as the sole shard key, concentrating all new writes onto a single "hot" shard regardless of how many shards the cluster actually has.
• Picking a low-cardinality shard key that can't meaningfully spread data — a handful of distinct values means a handful of possible chunks, no matter how many shards exist to receive them.
• Designing the shard key without considering the collection's actual query patterns, ending up with routine scatter-gather queries across every shard for what should be a common, cheap lookup.
• Reaching for sharding as a default "at scale" architecture rather than a tool adopted once a genuinely measured need — replica set capacity, working-set size, or write throughput — actually requires it; the added operational complexity (choosing and living with a shard key, config servers, balancer behavior) is real and ongoing.