Dynamic Feedback Load Balancing Scheduling
Contents
Introduction
Dynamic feedback load balancing scheduling algorithm considers the real-time loading and response time of each backend server, and adjusts the percentage of forwarded requests among the servers, in order to avoid some servers that may be overloaded still receive a lot of requests. This algorithm can help improve throughput of the whole system.
The working environment of this algorithm is illustrated in the following figure, there is a Monitor Daemon running on the load balancer, which is to monitor the availability and load information of each server. The Monitor Daemon can compute an aggregated load value based on all the load information for each server, and calculate a new server weight according to the aggregated load value. If the difference of new server weight and old server weight is more than the threshold, the Monitor Daemon adjusts the server weight with the new one in the IPVS inside the kernel. Usually, weighted round-robin scheduling or weighted least-connection scheduling is used for connection scheduling, because server weight can be adjusted in the weighted algorithms.
Connection Scheduling
When clients access network services, service time and computing resource consumption at servers for each request may vary a lot. It depends on many things, such as the type of service request, current network situation, and current resource consuption at server. Usually, heavy load requests may do computing-intensive query, database access, and serve long response data stream; light load requests may just access static HTML files or small image files.
The variance of request service time may lead to skew of server utilization, i.e. load imbalance among servers. For example, there are four web pages, A, B, C and D, in which D is time-consuming dynamic web page. When many people access those web pages, it is possible that all the requests for web page D are sent to the same server, which would cause this server overloaded, in the mean while the other servers are idle. Therefore, the overloaded servers may still receive new requests and put them into waiting queue, this would increase service time and lower the quality of service.
Dynamic Feedback Mechanism
Network services usually have a lot of small transcations and some long transactions, in which those long transaction may occupy high percentage of the whole work load. Therefore, we need to design a load balancing algorithm to avoid that long transactions might be always assigned to one or a small set of servers, and try to shape access burst into even distribution among servers.
We can use dynamic feedback mechanism to control new connection dispatching, to make load distribution more even among servers. For example, Weighted Round-Robin Scheduling is used to dispatch new connections inside the kernel in IPVS load balancer; there is a program called "Monitor Daemon" running in the user-space of load balancer, which is to monitor the availability and load information of each server periodically. The Monitor Daemon can compute the aggregated load value based on all the load information for each server and compute a new server weight according to the aggregated load value. If the aggregated load value is too high, the new weight is adjusted to a smaller value than the current one, so less new connections will be assigned to this server. If the aggregated load value is small, the new weight is adjusted to a higher value than the current one, so more new connections will be sent to this server. The Monitor Daemon only adjusts the server weight inside the kernel in IPVS when the server weight difference is larger than a threshold. The Monitor Daemon collects the load information of each server and adjusts server weight periodically. In short, this is a negative feedback mechanism for better utitlization of servers.