Load balancing means spreading requests across a group of servers. A load balancer sits in front of the servers and decides which one handles each request.
It is how a site keeps working when one machine can't keep up, and keeps working when one of those machines falls over.
Why one server stops being enough
Every server has limits: CPU, memory, network bandwidth and the number of connections it can hold open at once. A small shop with a handful of visitors never gets near them. When traffic grows, requests start queueing, responses slow down, and eventually the server runs out of memory or starts timing out.
There are two ways out. You can buy a bigger machine (scaling up), or you can run copies of your app on several machines (scaling out). Vertical vs Horizontal Scaling compares the two. Scaling out lets you add capacity beyond one machine and, with spare capacity and failover, survive one machine dying, but it raises a new question straight away: when a visitor types your address, which server should they reach?
You can't hand every visitor a list of five server addresses and ask them to choose. Some servers will be busy, some idle, and one might have crashed. Something has to make that choice for them, on every request.
What a load balancer does
The load balancer becomes the one public address for your site. Your domain's DNS points at it, and the servers behind it (often called the backend pool) are usually not reachable from the internet directly.
For each request, it:
- accepts the connection from the visitor;
- picks a healthy server from the pool;
- forwards the request to that server;
- passes the server's response back to the visitor.
The visitor never knows which server answered, so you can add servers, take one out for maintenance or replace a broken one without anybody changing a bookmark.
Wondering how this was made? How we make these