System design fundamentals are the decisions that determine what happens when traffic spikes, a database is behind, a queue is delayed, or a request times out. Tools change quickly, but these problems do not. This guide uses an online ticketing platform as a running example: public event pages receive bursts of traffic, checkout must not double-charge customers, inventory must not be oversold, and ticket delivery must be resilient.
System design fundamentals begin with failure
A distributed system rarely has one simple answer to “did it work?”. A payment request can reach a provider while its response is lost. A database replica may answer, but with an old value. One service can be healthy while unable to obtain a database connection. A timeout means the caller lacks information, not that the operation definitely failed.
Start by writing invariants and acceptable compromises. “A successful payment is charged at most once” and “confirmed inventory is never oversold” are invariants. “An event description may take a minute to update globally” can be an acceptable compromise. This distinction is more useful than beginning with a cloud product.
The request path: CDN, proxy and load balancer
Public traffic should not travel directly from a browser to one application process. A common path is browser, CDN, reverse proxy or API gateway, load balancer, application instance, then the services and data stores it needs.
A CDN caches shared content near users. It fits scripts, stylesheets, event images and cautiously cached public pages. It is not a safe default for personalised checkout or account responses. A reverse proxy provides a controlled public boundary for TLS, routing, authentication checks, compression and request limits. A load balancer distributes requests across healthy instances using approaches such as round robin, least connections or weighted routing.
Health checks must represent useful readiness. A process that returns HTTP 200 while it cannot use its database is not ready for a checkout request. AWS recommends combining CDNs, API gateways, reverse proxies and load balancing with a resilient workload behind them. AWS Well-Architected guidance
Rate limits protect capacity and fairness
Rate limiting controls how much work a client, account, token or tenant can cause. It protects against abuse, accidental loops and noisy neighbours. For a ticket launch, it stops one aggressive refresher from consuming capacity needed by everyone else.
| Approach | Idea | Useful for |
|---|---|---|
| Fixed window | Count requests in a time block. | Simple internal limits. |
| Sliding window | Smooth counts over time. | Fair public APIs. |
| Token bucket | Tokens refill and requests spend them. | Controlled bursts. |
| Concurrency limit | Limit work in flight. | Expensive exports or model calls. |
Return a useful 429 Too Many Requests response with Retry-After. Scope limits deliberately. IP-only limits can penalise a shared office network, while account-only limits are unavailable before sign-in. Edge IP limits and application token limits often complement one another.
REST, RPC and idempotent operations
REST works naturally for resource lifecycle: GET /events/evt_123 or DELETE /cart-items/item_789. RPC is clearer for an action with meaningful domain intent: POST /payments/authorize, POST /orders/ord_456/cancel, or POST /tickets/ticket_001/transfer. Use resources where the domain has stable resources, and explicit commands where the action has independent validation, permissions or audit requirements.
Idempotency makes retries safe. HTTP considers GET, PUT and DELETE idempotent in their intended server effect. RFC 7231 A payment-creation POST is different: if the customer clicks Pay, the charge succeeds and the browser times out, a second click must return the original result rather than charge twice.
POST /payments
Idempotency-Key: 5f3e65db-2e2f-4e8d-9e55-15d46a3eb4b9
{ "orderId": "ord_123", "amount": 12500, "currency": "GBP" }
The server records the actor, key, request fingerprint and final response. Equivalent retries return the stored response. Reusing the key for a different payload should fail. The IETF Idempotency-Key draft describes this contract.
System design fundamentals for queues and events
Synchronous calls are for answers needed now. Checkout must know whether payment is authorised before confirming the order. Asynchronous work is for activity that can happen later, needs independent retries, or should not make a non-critical dependency a checkout failure.
Checkout API -> payment authorisation -> immediate result
Order placed -> durable event -> ticket issuer
-> confirmation email
-> analytics pipeline
A queue normally gives one item of work to one worker. Pub-sub distributes an event to independent subscribers. An order.placed event may feed ticket issuance, email, fraud checks and analytics. Consumer groups scale a subscriber: each message goes to one member of a group, while separate groups receive their own copy. Kafka’s documentation describes this queue-like behaviour within a group.
At-least-once delivery means consumers must tolerate duplicates. Store processed event IDs, make writes idempotent, or enforce the business invariant with a database constraint. “Exactly once” is usually an end-to-end business design, not just a broker setting.
Transactions, caches and data ownership
ACID transactions protect related changes: atomicity means all intended changes commit or none do; consistency protects data rules; isolation controls concurrent visibility; durability keeps committed work. For inventory, do not merely read “one ticket remains” then write a new count. Two requests can both see the same value. Prefer a unique constraint, row-level control or atomic conditional update in the authoritative database.
The dual-write problem appears when an order commits but the process crashes before publishing OrderPlaced. A transactional outbox writes the business row and a pending event record in the same database transaction. A separate publisher reliably delivers the outbox record. Consumers still need idempotency, but the dangerous gap becomes observable and recoverable.
A cache is a performance optimisation, not the authority. With cache-aside, read the cache first, load the database on a miss, then store a bounded-lived copy. On a write, update the authoritative source then invalidate the relevant key. Redis cache-aside documentation describes this approach. Always define how stale a value may be and what happens if invalidation fails. A briefly old event description may be fine. Payment status and permissions often need a stricter read path.
Replication, CAP, partitioning and hashing
Replication creates copies for read capacity, recovery and availability. In leader-follower replication, the leader accepts writes and followers receive changes. With asynchronous replication, a customer can submit payment then read from a lagging follower and see an old answer. For critical read-after-write flows, read from the leader, return the newly created object directly, or route the user temporarily to a consistent read path.
CAP is most useful when stated precisely: during a network partition, a distributed system cannot guarantee both strong consistency and availability for every request. It is not a menu for permanently selecting any two letters. The business question is whether to reject or delay a request to protect an invariant, or to answer with potentially stale data.
Vertical scaling makes one machine larger. Horizontal scaling adds machines. Database partitioning divides a logical table into physical pieces, often by time or tenant. Sharding distributes data among independent database instances. Both help only when the partition key matches access patterns. PostgreSQL can prune irrelevant partitions, but also notes that partitioning has planning and execution overhead. PostgreSQL documentation
Consistent hashing helps a distributed cache or routing pool absorb membership changes. Nodes and keys map around a ring, so adding or removing a node moves only part of the keyspace. It does not fix a single hot key, and it does not choose a good shard key for you.
Retries, circuit breakers, locks and observability
Retry only transient failures and only operations that are safe to repeat. A sound policy defines retryable errors, per-attempt timeout, a total time budget, exponential backoff and jitter. Jitter prevents clients retrying in lockstep. AWS warns that immediate retries can create retry storms and recommends backoff, jitter and a cap. AWS retry guidance
A circuit breaker handles sustained failure. After repeated timeouts, it opens and fails quickly or uses a fallback. After a cool-down, a small number of test calls determines whether the dependency recovered. Distributed locks can coordinate scheduled work, but should not replace authoritative constraints. Locks expire, processes pause and networks partition. Redis recommends unique lock values, owner-checked release and fencing tokens where correctness is sensitive. Redis guidance
Finally, instrument the system. Logs record events. Metrics show rates, latency, errors and saturation. Traces follow a request across boundaries. OpenTelemetry treats logs, metrics and traces as the core signals. Correlate them with a trace ID, and never log secrets, payment data or unnecessary personal information.
A practical system design checklist
- Write business invariants before choosing infrastructure.
- Name the authoritative owner of every important datum.
- Define which reads may be stale and for how long.
- Make retryable mutations idempotent.
- Use durable asynchronous boundaries for non-immediate work.
- Use transactions, constraints and atomic updates for local correctness.
- Cache only with a freshness and invalidation contract.
- Design timeouts, retries, backoff, circuit breakers and limits together.
- Instrument critical paths before an incident makes them urgent.
- Scale from measured bottlenecks rather than anticipated complexity.
The best designs are not the ones with the most components. They are the ones whose trade-offs are explicit: what happens when a request duplicates, a replica is behind, a queue is delayed or a dependency fails. For a related perspective, read this site’s guide to Clean Architecture, SOLID and event-driven systems.