System Design Basics

What is system design?#

System design is the process of defining the architecture, components, data flow and trade-offs of a software system so it meets both its functional requirements ("what it does") and its non-functional requirements ("how well it does it, under load, over time, and when things fail").

Unlike a single-function coding problem with one correct answer, system design questions have no single right answer — every choice is a trade-off, and articulating those trade-offs is what interviewers (and real engineering decisions) actually care about.

Scalability#

Scalability is a system's ability to handle growing load by adding resources.

• Vertical scaling (scale up) — adding more CPU/RAM to a single machine. Simple, but has a hard ceiling and a single point of failure.
• Horizontal scaling (scale out) — adding more machines and distributing load across them. Effectively unlimited ceiling, but requires the system to be designed for distribution (stateless services, shared/replicated data, load balancing).

Most large-scale systems (Netflix, Uber, Amazon) rely overwhelmingly on horizontal scaling — vertical scaling alone can't support hundreds of millions of users.

Availability#

Availability is the percentage of time a system is operational and able to serve requests. It's usually expressed in "nines":

Higher availability costs exponentially more engineering effort (redundancy, failover automation, multi-region deployment) — so the right target depends on what the business actually needs, not "as high as possible."

Reliability#

Reliability is the probability a system performs its intended function correctly over a given period, without failure. A system can be available (responding to requests) but not reliable (returning wrong or corrupted data) — the two are related but distinct.

Latency vs throughput#

• Latency — how long a single request takes, end to end (often measured at p50/p95/p99 percentiles, since averages hide bad tail behaviour).
• Throughput — how many requests the system can process per unit of time.

These two often trade off against each other: batching requests can raise throughput while increasing individual latency, for example.

Estimating scale (back-of-the-envelope)#

A crucial system design skill: converting a vague business scenario into rough numbers that justify architectural choices.

Example: a URL shortener with 100M new URLs/month and a 100:1 read:write ratio

Writes:  100,000,000 / (30 * 24 * 3600) ≈ 39 writes/sec
Reads:   39 * 100 ≈ 3,900 reads/sec

Storage (5 years): 100M * 12 * 5 = 6B records
  at ~500 bytes/record ≈ 3 TB total

These estimates directly justify decisions later in this track — e.g., a 100:1 read:write ratio strongly suggests aggressive caching in front of the database.

The vocabulary you'll build on#

The rest of this track builds directly on these fundamentals:

• CAP theorem & PACELC — the fundamental trade-offs in distributed data systems.
• Load balancing, reverse proxies & caching — how traffic is distributed and reads are made fast.
• Database scaling — sharding, replication, and their trade-offs.
• Message queues & event-driven architecture — decoupling services in time.
• Microservices vs monolith — when splitting a system into services actually pays off.

Common mistakes (in interviews and in practice)#

• Jumping straight to technology names ("we'll use Kafka and Redis") before establishing requirements and rough scale.
• Ignoring the read/write ratio — it changes almost every downstream decision (caching, replication strategy, database choice).
• Treating "more microservices" as inherently better — splitting a system has a real operational cost (see the Microservices vs Monolith page later in this track).

Interview questions#

Q: What's the difference between scalability and performance? Performance is how fast a system responds under a given load. Scalability is how well that performance holds up as load increases — a system can be fast at low load but not scalable if performance collapses as traffic grows.

Q: Why do engineers use percentiles (p95, p99) instead of average latency? Averages hide tail behaviour — a system where 99% of requests take 10ms but 1% take 10 seconds has a great average yet a terrible experience for 1-in-100 users. Percentiles surface exactly that tail.