Networking for System Design:
From First Principles
When you connect two servers on a whiteboard, how do they actually talk to each other? Learn how data travels across the internet, how to choose the right communication protocol (HTTP/2, WebSockets, gRPC), and how to keep your systems fast and reliable.
How Networking Completes Your System Design Foundation
Let's look back at the journey we've taken so far:
💡 1. The Physical Reality: Lasers, Glass & The Speed of Light
At its lowest level, the internet is not magical software in the cloud. It is pulses of laser light shooting through undersea glass fiber-optic cables and electromagnetic radio waves between mobile phones and cell towers.
Distance between Sydney, Australia and Virginia, USA is ~15,000 km.
Therefore, the absolute fastest a packet can physically travel one-way is:
🎯 Why This Matters in System Design:
No software algorithm, no Rust/C++ rewrite, and no database index can make a packet travel faster than the speed of light in glass.
The Architectural Consequence: If your application server is in the US and your user is in Sydney, every single round-trip network handshake burns 150ms of your 200ms latency budget (from Module 1.3).
➔ Solution: This is why we place CDNs and Edge Reverse Proxies (Cloudflare/Fastly) physically near the user. Terminating connections locally in Sydney cuts handshake latency from 150ms down to 5ms!
📮 2. The Postal System of the Internet: IP Addresses & Ports
If millions of computers are simultaneously pulsing light down the same fiber-optic cables, how does a message sent from your phone reach the exact right application on a specific server in a datacenter?
Identifies the physical or virtual machine on the global network (e.g. 142.250.190.46). Routers inspect this IP to forward packets hop-by-hop across datacenters.
A single server runs dozens of programs. Ports direct traffic to the right software:
• :443 ➔ Web Server (HTTPS)
• :6379 ➔ Redis Cache
• :5432 ➔ PostgreSQL DB
Humans remember names (api.uber.com), not random numbers. DNS translates names into IP addresses in ~15ms before the first connection is made.
(Source IP, Source Port, Dest IP, Dest Port, Protocol).This exact 5-tuple uniquely identifies a connection among millions of concurrent users.
🤝 3. Transport & Reliability: TCP vs UDP (And The Socket Crisis)
Physical network cables are imperfect. Cellular towers drop radio packets when users enter elevators, and routers drop packets during datacenter traffic spikes.
The Fundamental Engineering Question: Do we care if a single byte is lost?
"Guarantee Every Single Byte": If a packet is dropped, TCP detects the missing sequence number and automatically retransmits it. Packets are assembled in exact chronological order.
- Requires a 3-Way Handshake before sending data.
- Enforces Congestion Control (slows down when network is congested).
- Used For: REST APIs, Financial Transactions, Database Queries, Web Pages.
"Fire and Forget": Sends raw packets without establishing a connection. If a packet is lost, it is permanently gone. Zero retransmission, zero waiting.
- 0 Handshake Latency: Data is transmitted immediately.
- No packet ordering, no head-of-line stalls.
- Used For: Live Video Calls (Zoom), Online Multiplayer Gaming, DNS Queries, HTTP/3 (QUIC).
The TCP 3-Way Handshake (Why Handshakes Cost 1 RTT)
Before client and server can exchange a single byte of HTTP data, they must synchronize sequence numbers to establish trust:
When an application server closes a TCP connection, the operating system holds the socket in a TIME_WAIT state for 60 seconds (2 Maximum Segment Lifetimes) to prevent delayed duplicate packets from corrupting new connections.
The Disaster Scenario: In Module 1.2, you scaled out 50 App Servers processing 50,000 database queries per second. If your app servers open and close a new TCP connection for every single database query, all 65,535 ephemeral ports are locked in `TIME_WAIT` within seconds!
Your servers immediately crash with EADDRNOTAVAIL (Cannot assign requested address) even though CPU utilization is only 5%!
✅ The Architecture Solution: Always use Persistent TCP Connection Pools and HTTP Keep-Alive. Sockets are kept open and reused forever, eliminating connection churn.
🌐 4. The Language of the Web: Evolution of HTTP (1.1 ➔ 2 ➔ 3)
Once a reliable TCP or UDP pipe is established between two machines, how do they format their messages?Over the last 25 years, HTTP evolved through 3 major eras to solve network bottlenecks:
How it worked: Human-readable plain text (`GET /index.html HTTP/1.1`).
The Problem (Head-of-Line Blocking): A TCP socket could only process 1 request at a time. If the first image was large, all other CSS and JS files queued up behind it.
The Workaround: Web browsers opened 6 parallel TCP connections per domain, causing heavy server memory bloat.
The Breakthrough (Multiplexing): Instead of 1 request per socket, HTTP/2 chops requests into small Binary Frames. You can send 100 simultaneous requests and receive 100 responses over 1 single TCP connection without blocking!
HPACK Compression: Compresses repetitive HTTP headers (cookies, user-agents), reducing payload sizes by 85%.
The Remaining Flaw: All streams still share 1 underlying TCP byte-stream. If Wi-Fi drops 1 packet, the OS pauses all streams until that packet is retransmitted.
The Innovation: Replaced TCP entirely with QUIC (built on top of UDP).
1. True Stream Independence: Each stream is independent. If Packet B drops, Stream A (HTML) and Stream C (Images) continue downloading at 100% speed with zero stalls.
2. Seamless Mobile Handover: TCP connections are bound to an IP address. When you leave your house and switch from Wi-Fi to 5G, your IP changes and TCP connections break. HTTP/3 uses a 64-bit Connection ID, allowing downloads and video streams to continue without interruption!
3. 0-RTT Handshakes: Returning clients resume sending encrypted data instantly with 0 round-trip handshake delay.
⚡ 5. Specialized Protocols: WebSockets, SSE & Microservice gRPC
Standard HTTP is based on client requests and server responses. But in modern system architectures, we need specialized communication patterns:
Two-Way Live Stream: Upgrades an HTTP connection to an open, persistent full-duplex TCP socket. Either client or server can push messages instantly with only 2 bytes of frame overhead.
One-Way Stream: Lightweight unidirectional text streaming over standard HTTP. Built-in browser auto-reconnect without custom WebSocket heartbeat complexity.
Binary RPC Engine: Uses Google Protocol Buffers over HTTP/2. Serializes data into compact binary bits, eliminating heavy JSON parsing and slashing internal latency to <0.5ms.
🗺️ 6. The Master Architecture Wire Map: What Goes Where?
Now let's connect all of this directly to the complete distributed architecture we designed across Track 1.
Click any wire or component below to see why that protocol is chosen at that exact architectural boundary:
Why HTTP/3 here? Mobile devices walk between cell towers and drop packets. HTTP/3 over UDP survives IP address changes (Wi-Fi ➔ 5G) without restarting downloads and reconnects in 0-RTT!
🔬 7. Interactive Lab: Experience TCP vs QUIC Head-of-Line Blocking
Click "Drop 1 Packet" to test how TCP Head-of-Line blocking impacts user download speeds!
🎯 8. Real-World System Design Interview Scenarios
You are designing the web interface for an AI chatbot (like ChatGPT). The server generates text word-by-word and must stream tokens to the user's browser in real time. Which protocol should you defend in your interview?
Your primary datacenter is in Virginia, USA. Users in Mumbai, India complain that loading your web app takes 600ms, even though your internal backend API response time is only 15ms. How do you fix this?
Inside your private datacenter, your checkout service calls the inventory, pricing, and fraud microservices millions of times per second. JSON parsing is consuming 40% of server CPU. What is the standard architectural fix?
Full Deep-Dive Locked
This production architecture deep-dive, interactive canvas labs, and 5-language code templates require an active Pro Pass ($1/mo or $49 lifetime).