Load Balancing
Using a "traffic manager" to distribute requests across multiple servers using algorithms like Round Robin or IP Hashing
Overview
A load balancer distributes incoming network traffic across multiple servers, ensuring no single server is overwhelmed. It's essential for horizontal scaling, high availability, and fault tolerance.
Load balancers can be hardware (F5, Citrix) or software (Nginx, HAProxy), and operate at different network layers (Layer 4 or Layer 7).
Key Concepts
Round Robin
Distributes requests evenly to each server in rotation. Simple but doesn't account for server load.
Least Connections
Routes to server with fewest active connections. Better for long-running requests.
IP Hashing
Routes based on client IP hash. Ensures same client always hits same server (sticky sessions).
Health Checks
Periodically ping servers to ensure they're healthy. Remove unhealthy servers from rotation.
How It Works
Client Request Flow:
- Client sends request to load balancer IP
- Load balancer applies algorithm (Round Robin, Least Connections, etc.)
- Chooses healthy server from pool
- Forwards request to chosen server
- Server processes and responds
- Load balancer forwards response back to client
Layer 4 (Transport): Routes based on IP/port (faster) Layer 7 (Application): Routes based on HTTP content/cookies (more flexible)
Use Cases
Distributing web traffic across multiple web servers
Database read replicas (route reads to replicas)
Blue-green deployments (switch traffic between versions)
Geographic routing (route to nearest data center)
A/B testing (route percentage of traffic to different versions)
Best Practices
Use health checks to remove unhealthy servers
Implement multiple load balancers for redundancy
Use Layer 7 for sophisticated routing (path-based, header-based)
Enable connection draining for graceful server shutdown
Monitor load balancer metrics (request rate, latency, errors)
Use SSL termination at load balancer to offload encryption
Configure appropriate timeout values
Use auto-scaling to add/remove servers based on load
Interview Tips
What Interviewers Look For
- •
Explain common algorithms: Round Robin, Least Connections, IP Hash
- •
Discuss Layer 4 vs Layer 7 load balancing
- •
Mention health checks and automatic failover
- •
Talk about session persistence/sticky sessions
- •
Explain how load balancers enable zero-downtime deployments
- •
Discuss DNS-based load balancing for geographic distribution
- •
Mention cloud load balancers: AWS ELB/ALB, Azure Load Balancer, GCP Load Balancing