Performance and Tuning

2012-01-26T11:37:51Z

Dogonthesun: /* Connection Hash Table Size */

== Performance Measurement ==

=== Tools ===

* testlvs: simple throughput testing tool for LVS from Julian, see [http://www.ssi.bg/~ja/#testlvs the testlvs page]
* ab: apache benchmark
* httperf: a tool for measuring web server performance from HP, see [http://www.hpl.hp.com/research/linux/httperf/ the httperf homepage]
* Tsung: an open-source multi-protocol distributed load testing tool, see [http://tsung.erlang-projects.org/ Tsung project homepage]
* specweb: web server benchmark by spec.org, it is now [http://www.spec.org/web2005/ specweb2005].
* webbench: a licensed benchmark program that measures the performance of Web servers, developed by VeriTest.

== Performance Tuning ==

=== Connection Hash Table Size ===

The IPVS connection hash table uses the chaining scheme to handle
hash collisions. Using a big IPVS connection hash table will greatly
reduce conflicts when there are hundreds of thousands of connections
in the hash table.

Note the table size must be power of 2. The table size will be the
value of 2 to the your input number power. The number to choose is
from 8 to 20, the default number is 12, which means the table size
is 4096. Don't input the number too small, otherwise you will lose
performance on it. You can adapt the table size yourself, according
to your virtual server application. It is good to set the table size
not far less than the number of connections per second multiplying
average lasting time of connection in the table. For example, your
virtual server gets 200 connections per second, the connection lasts
for 200 seconds in average in the connection table, the table size
should be not far less than 200x200, it is good to set the table
size 32768 (2**15).

We can configure the size of IPVS connection hash table before compiling the Linux kernel. Here are the IPVS configurations in the 'make menuconfig' menu:

<pre>
Networking Options -->
Network packet filtering framework (Netfilter) --->
IP: Virtual Server Configuration -->
[*] IPv6 support for IPVS
[ ] IP virtual server debugging
(20) IPVS connection table size (the Nth power of 2)
*** IPVS transport protocol load balancing support ***
[*] TCP load balancing support
[*] UDP load balancing support
[*] ESP load balancing support
[*] AH load balancing support
*** IPVS scheduler ***
<M> round-robin scheduling
<M> weighted round-robin scheduling
<M> least-connection scheduling
<M> weighted least-connection scheduling
<M> locality-based least-connection scheduling
<M> locality-based least-connection with replication scheduling
<M> destination hashing scheduling
<M> source hashing scheduling
<M> shortest expected delay scheduling
<M> never queue scheduling
*** IPVS application helper ***
<M> FTP protocol helper
</pre>

=== Netfilter Connection Track ===

[[IPVS]] uses its own simple and fast connection tracking for performance reasons, instead of using netfilter connection tracking. So, if you don't use firewalling feature at [[load balancer]] and you need an extremely fast load balancer, do not load netfilter conntrack modules into you system, because there is no need to do double tracking. Note that [[LVS/NAT]] should work too without the conntrack modules.

Julian compared the performance of IPVS with ip_conntrack and without ip_conntrack. See http://archive.linuxvirtualserver.org/html/lvs-users/2001-12/msg00141.html

== Some Performance Data ==

While the [[LVS/DR]] cluster was pushing out 9.6Gbps traffic, the LVS [[load balancer]] was
doing a negilgable ammount of work, which seems to indicate that LVS
could push a great deal more traffic given sufficient real-servers and
end-users.

* http://archive.linuxvirtualserver.org/html/lvs-users/2005-11/msg00266.html

== External Links ==

* [http://www.kegel.com/c10k.html The C10K problem] written by Dan Kegel, good notes on how to configure operating systems and write code to support 10K clients on a single server

[[Category:LVS Handbook]]

Performance and Tuning

2012-01-26T11:36:45Z

Dogonthesun: /* Connection Hash Table Size */

== Performance Measurement ==

=== Tools ===

* testlvs: simple throughput testing tool for LVS from Julian, see [http://www.ssi.bg/~ja/#testlvs the testlvs page]
* ab: apache benchmark
* httperf: a tool for measuring web server performance from HP, see [http://www.hpl.hp.com/research/linux/httperf/ the httperf homepage]
* Tsung: an open-source multi-protocol distributed load testing tool, see [http://tsung.erlang-projects.org/ Tsung project homepage]
* specweb: web server benchmark by spec.org, it is now [http://www.spec.org/web2005/ specweb2005].
* webbench: a licensed benchmark program that measures the performance of Web servers, developed by VeriTest.

== Performance Tuning ==

=== Connection Hash Table Size ===

The IPVS connection hash table uses the chaining scheme to handle
hash collisions. Using a big IPVS connection hash table will greatly
reduce conflicts when there are hundreds of thousands of connections
in the hash table.

Note the table size must be power of 2. The table size will be the
value of 2 to the your input number power. The number to choose is
from 8 to 20, the default number is 12, which means the table size
is 4096. Don't input the number too small, otherwise you will lose
performance on it. You can adapt the table size yourself, according
to your virtual server application. It is good to set the table size
not far less than the number of connections per second multiplying
average lasting time of connection in the table. For example, your
virtual server gets 200 connections per second, the connection lasts
for 200 seconds in average in the connection table, the table size
should be not far less than 200x200, it is good to set the table
size 32768 (2**15).

We can configure the size of IPVS connection hash table before compiling the Linux kernel. Here are the IPVS configurations in the 'make menuconfig' menu:

<pre>
Networking Options -->
Network packet filtering framework (Netfilter) --->
IP: Virtual Server Configuration -->
<M> virtual server support (EXPERIMENTAL)
[ ] IP virtual server debugging
(12) IPVS connection table size (the Nth power of 2)
--- IPVS scheduler
<M> round-robin scheduling
<M> weighted round-robin scheduling
<M> least-connection scheduling scheduling
<M> weighted least-connection scheduling
<M> locality-based least-connection scheduling
<M> locality-based least-connection with replication scheduling
<M> destination hashing scheduling
<M> source hashing scheduling
<M> shortest expected delay scheduling
<M> never queue scheduling
--- IPVS application helper
<M> FTP protocol helper
</pre>

=== Netfilter Connection Track ===

[[IPVS]] uses its own simple and fast connection tracking for performance reasons, instead of using netfilter connection tracking. So, if you don't use firewalling feature at [[load balancer]] and you need an extremely fast load balancer, do not load netfilter conntrack modules into you system, because there is no need to do double tracking. Note that [[LVS/NAT]] should work too without the conntrack modules.

Julian compared the performance of IPVS with ip_conntrack and without ip_conntrack. See http://archive.linuxvirtualserver.org/html/lvs-users/2001-12/msg00141.html

== Some Performance Data ==

While the [[LVS/DR]] cluster was pushing out 9.6Gbps traffic, the LVS [[load balancer]] was
doing a negilgable ammount of work, which seems to indicate that LVS
could push a great deal more traffic given sufficient real-servers and
end-users.

* http://archive.linuxvirtualserver.org/html/lvs-users/2005-11/msg00266.html

== External Links ==

* [http://www.kegel.com/c10k.html The C10K problem] written by Dan Kegel, good notes on how to configure operating systems and write code to support 10K clients on a single server

[[Category:LVS Handbook]]

LVSKB - User contributions [en]

Performance and Tuning

Performance and Tuning