Load Balancing
Enable scale out at all levels of the system, so you can just add more server boxes to handle more load.
There are two related issues
- Spreading the load evenly
- Routing the work to the correct server
To simplify this problem
- Ensure all web servers are identical (no server-specific state)? don’t retain client state on web servers (hard to avoid …)
- Randomly assign requests to servers (e.g., use an IP sprayer)
- Avoid sending requests to a failed server
We’ll deal with more complex scenarios later