LVSKB - User contributions [en]

IPVS Wish List

2008-02-10T14:58:22Z

Maluyao: /* Feature List */

This page is to write all the wishes for [[IPVS]] development.

'''Remember''': you can ask for features as long as you want, but that won't make such features appear magically. :) Basically, you have these options:
* get your hands dirty, program and submit code to LVS
* convince other programmers that your wanted feature is valuable to them so that the programmers do it for you (and the rest of the community)
* put some bounty on a feature development and wait until someone wants to earn some money

== Feature List ==

* [[SCTP load balancing]] support
* connection synchronization improvement
* ip_gre tunnel

* [[IPV6 load balancing]] support

[[Category:IPVS]]

Building Helix Server Cluster using LVS

2007-01-01T14:26:06Z

Maluyao: /* VS/DR */

== Introduction ==

RealNetworks' streaming service uses the following protocols:

* PNM (TCP on port 7070, UDP from server -> player on ports 6970-7170)
* [[RTSP]] (TCP on port 554, similar UDP as above, but often on multiple ports)
* PNM was our original, but now mostly legacy, protocol used in our versions 1 through 5.
* There's also a HTTP/TCP-only fallback mode which is (usually) on port 8080.

== Architecture ==

== Configuration Example ==
=== VS/NAT ===
{|
|-
|| VIP/EXT IP || 192.168.9.244
|-
|| GATEWAY IP || 192.168.0.1
|-
||RS1 || 192.168.0.150
|-
||RS2 || 192.168.0.160
|}

Run the follwing script on LB Server:
ipvsadm -A -t 192.168.9.244:554 -s rr
ipvsadm -a -t 192.168.9.244:554 -r 192.168.0.150 -m
ipvsadm -a -t 192.168.9.244:554 -r 192.168.0.160 -m

Run
realplay rtsp://192.168.9.244/test.rm

for a test.

=== VS/DR ===
{|
|-
|| VIP/eth0:0 || 192.168.9.200/24
|-
|| LB/eth0 || 192.168.9.244
|-
||RS1 || 192.168.9.150 with lo0:0 192.168.9.244/32 noarp
|-
||RS2 || 192.168.9.160 with lo0:0 192.168.9.244/32 noarp
|}

Run the follwing script on LB Server:
ipvsadm -C
ipvsadm -A -t 192.168.9.200:554 -s rr
ipvsadm -a -t 192.168.9.200:554 -r 192.168.9.150 -g
ipvsadm -a -t 192.168.9.200:554 -r 192.168.9.160 -g

Run
realplay rtsp://192.168.9.200/test.rm

for a test.

== Conclusion ==

== References ==

* http://archive.linuxvirtualserver.org/html/lvs-users/2001-06/msg00097.html

{{lvs-example-stub}}

[[Category:LVS Examples|Media]]

Building Helix Server Cluster using LVS

2007-01-01T14:25:24Z

Maluyao: /* VS/NAT */

== Introduction ==

RealNetworks' streaming service uses the following protocols:

* PNM (TCP on port 7070, UDP from server -> player on ports 6970-7170)
* [[RTSP]] (TCP on port 554, similar UDP as above, but often on multiple ports)
* PNM was our original, but now mostly legacy, protocol used in our versions 1 through 5.
* There's also a HTTP/TCP-only fallback mode which is (usually) on port 8080.

== Architecture ==

== Configuration Example ==
=== VS/NAT ===
{|
|-
|| VIP/EXT IP || 192.168.9.244
|-
|| GATEWAY IP || 192.168.0.1
|-
||RS1 || 192.168.0.150
|-
||RS2 || 192.168.0.160
|}

Run the follwing script on LB Server:
ipvsadm -A -t 192.168.9.244:554 -s rr
ipvsadm -a -t 192.168.9.244:554 -r 192.168.0.150 -m
ipvsadm -a -t 192.168.9.244:554 -r 192.168.0.160 -m

Run
realplay rtsp://192.168.9.244/test.rm

for a test.

=== VS/DR ===
{|
|-
|| VIP/eth0:0 || 192.168.9.200/24
|-
|| LB || 192.168.9.244
|-
||RS1 || 192.168.9.150 with lo0:0 192.168.9.244/32 noarp
|-
||RS2 || 192.168.9.160 with lo0:0 192.168.9.244/32 noarp
|}

Run the follwing script on LB Server:
ipvsadm -C
ipvsadm -A -t 192.168.9.200:554 -s rr
ipvsadm -a -t 192.168.9.200:554 -r 192.168.9.150 -g
ipvsadm -a -t 192.168.9.200:554 -r 192.168.9.160 -g

Run
realplay rtsp://192.168.9.200/test.rm

for a test.

== Conclusion ==

== References ==

* http://archive.linuxvirtualserver.org/html/lvs-users/2001-06/msg00097.html

{{lvs-example-stub}}

[[Category:LVS Examples|Media]]

Building Helix Server Cluster using LVS

2007-01-01T13:48:10Z

Maluyao: /* VS/NAT */

Building Helix Server Cluster using LVS

2007-01-01T13:42:40Z

Maluyao: /* VS/NAT */

Building Helix Server Cluster using LVS

2007-01-01T13:42:12Z

Maluyao: /* VS/NAT */

Building Helix Server Cluster using LVS

2007-01-01T13:19:51Z

Maluyao: /* VS/NAT */

Building Helix Server Cluster using LVS

2007-01-01T13:18:16Z

Maluyao: /* vsnat mode */

Building Helix Server Cluster using LVS

2007-01-01T13:17:07Z

Maluyao: /* = */

Building Helix Server Cluster using LVS

2007-01-01T13:16:19Z

Maluyao: /* Configuration Example */

Building Helix Server Cluster using LVS

2007-01-01T13:04:36Z

Maluyao:

LVS/TUN mode with FreeBSD and Solaris realserver

2006-08-29T03:39:16Z

Maluyao: /* Config FreeBSD box */

Author: Ma Luyao RHCE maluyao at gmail dot com

August 16, 2006

== Introduction ==

The Linux Virtual Server Project (LVS) is a project started by Dr. Wensong Zhang. LVS implements three load balance technology.

a. Network Address Translation (VS/NAT)

b. Direct Routing (VS/DR)

c. IP Tunneling (VS/TUN)

VS/NAT is easy to setup. The load balancer may be a bottleneck of the whole system where the number of servers is more than 20, because both the request packets and response packets need to be rewritten by the load balancer.

VS/DR has the best performance. VS/DR uses MAC-spoofing technology, so it requires that one of the load balancer’s NIC and the real server’s NIC must be in the same IP network segment, and physical segment as well.

VS/TUN is the most scalable. The advantage is since servers connect each other by using IP-tunneling, the load balance and real servers can reside on different LAN , or even WAN.

I plan to setup a set of servers with load balance technique. Realserver run FreeBSD and Solaris system in different places on our business. So it is the only choice for us to use VS/TUN.

We install Fedora Core 5 Linux on our Load Balance server. Kernel-2.6.15 is default.

FreeBSD version is 5.4, we recompile the kernel in order to improve the performance. The default kernel can work too.

Solaris box runs Solaris 10 x86.

== Normal Linux Solaris/FreeBSD IP-tunneling ==

IP tunneling (IP encapsulation) is a technique to encapsulate IP datagram within IP datagram, which allows datagram destined for one IP address to be wrapped and redirected to another IP address. Many modern OS support IP tunneling, such as Linux, FreeBSD and Solaris.

Let’s do some test to setup normal Linux-Solaris and Linux-FreeBSD IPtunneling without LVS.

=== Testing environment ===

{|
|-
|| OS IP || Address || Tunnel IP Address
|-
|| Fedora Core 5 Linux || 10.0.0.1 || 172.16.0.1
|-
|| Solaris 10 x86 || 10.0.0.2 || 172.16.0.2
|-
|| FreeBSD 5.4 || 10.0.0.3 || 172.16.0.3
|}

=== Config Linux box ===

In order to create an IP-Tunneling between Linux box and Solaris box, run command:

ip tunnel add tun0 mode ipip remote 10.0.0.2 local 10.0.0.1

ifconfig tun0 172.16.0.1 pointopoint 172.16.0.2

ip tunnel add tun1 mode ipip remote 10.0.0.3 local 10.0.0.1

ifconfig tun1 172.16.0.1 pointopoint 172.16.0.3

The ”tun0” is a virtual NIC device name. It can be replaced by other names, such as ”abc0” or ”def0”.

=== Config Solaris box ===

To create an IP-tunneling, we have two methods, First method of creating tunnel is using three commands as below.

/sbin/ifconfig ip.tun0 plumb

/sbin/ifconfig ip.tun0 tsrc 10.0.0.2 tdst 10.0.0.1

/sbin/ifconfig ip.tun0 172.16.0.2 172.16.0.1

It takes effect immediately.

Second method is creating /etc/hostname.ip.tun0 file which contains two lines as below.

tsrc 10.0.0.2 tdst 10.0.0.1

172.16.0.2 172.16.0.1 netmask 255.255.255.255 up

If we use the second method, the tunnel will exist automatically after we reboot system.

=== Config FreeBSD box ===

On FreeBSD box, we also have two methods to create an IP-Tunneling, one way is to run three commands:

ifconfig gif0 create

ifconfig gif0 tunnel 10.0.0.3 10.0.0.1

ifconfig gif0 inet 172.16.0.3 172.168.0.1 netmask 255.255.255.0

If we use the second method, the tunnel will exist automatically after we reboot system.

Add three lines on /etc/rc.conf file as below, the tunnel will exist automatically after we reboot system.

gif_interfaces=”gif0”

gifconfig_gif0=”10.0.0.3 10.0.0.1”

ifconfig_gif0=”172.16.0.3 172.16.0.1 netmask 255.255.255.0”

=== Using IP-tunneling ===

In all the cases on Linux/FreeBSD/Solaris, the netmask value will be default if we ignore to set it.

After creating two tunnels, we can ping 10.0.0.2 and 10.0.0.3 at Linux box successfully, and ping 10.0.0.1 at Solaris or FreeBSD box successfully. All applications layers protocol will work correctly.

== Working example of VS/TUN ==

In VS/TUN mode, because realserver don’t send any datagram to Load balancer, we need not create any tunnel on Linux box and must create proper tunnel on FreeBSD/Solaris box.

=== Topological graph ===

[[Image:Topo.png|center|600px]]

We can treat all machines on internet and have their real ip. The client can send their requests to VIP of LB Server. LB Server redirects the requests realserver through IP-tunneling. Realserver can send datagram to client.

Because both the gif0 of FreeBSD and ip.tun0 both are NO-ARP device, they are invisible to client.

=== Config FreeBSD box ===

On FreeBSD box, edit /etc/rc.conf as below:

...

ifconfig_lnc0=”inet 192.168.0.170 netmask 0xffffff00”

gif_interfaces=”gif0”

gifconfig_gif0=”192.168.0.170 192.168.10.100”

ifconfig_gif0=”192.168.10.200 192.168.10.99 netmask 0xffffffff”

apache_enable=”YES”

...

192.168.10.99 looks like a terminal of this tunnel. It can be any idle IP and never use.

=== Config Solaris box ===
On Solaris , edit /etc/hostname.ip.tun0 as below:

tsrc 192.168.0.180 tdst 192.168.10.100

192.168.10.200 192.168.10.99 netmask 255.255.255.255 up

192.168.10.99 looks like a terminal of this tunnel. It can be any IP address and never use it.

=== Config Load balancer ===

Run ipvsadm command on Linux Box: Linux kernel must have ip forward enable.

Edit /etc/sysctl.conf and set to:

net.ipv4.ip_forward = 1

While running command:

sysctl -p

It will take effect.

Run the scripts:

ipvsadm -C

ipvsadm -A -t 192.168.10.200:80 -s wlc

ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.170 -i

ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.180 -i

In this script, the port of LB Server and realerver must be same. Good idea is not setting the port of realserver.Now VS/TUN is finished. When a client access 192.168.10.200 at port 80, the datagram will be sent to 192.168.0.170 or 192.168.180 port 80.

== Conclusions and Future Work ==

I believe AIX and HP-UX can also work with VS/TUN, I have no Aix or HP-UX servers. Who can help to provide me with such an environment?

References:

[1] http://www.linuxvirtualserver.org

[2] http://www.triload.com/en/manual/index.html

[3] http://blog.kovyrin.net/2006/03/17/how-to-create-ip-ip-tunnel-between-freebsd-and-linux/

[4] http://docs.sun.com/app/docs/doc/817-0573/6mgc65bcq?a=view

[[Category:LVS Examples|Tunnel]]

LVS/TUN mode with FreeBSD and Solaris realserver

2006-08-29T03:37:28Z

Maluyao: /* Config FreeBSD box */

Author: Ma Luyao RHCE maluyao at gmail dot com

August 16, 2006

== Introduction ==

The Linux Virtual Server Project (LVS) is a project started by Dr. Wensong Zhang. LVS implements three load balance technology.

a. Network Address Translation (VS/NAT)

b. Direct Routing (VS/DR)

c. IP Tunneling (VS/TUN)

VS/NAT is easy to setup. The load balancer may be a bottleneck of the whole system where the number of servers is more than 20, because both the request packets and response packets need to be rewritten by the load balancer.

VS/DR has the best performance. VS/DR uses MAC-spoofing technology, so it requires that one of the load balancer’s NIC and the real server’s NIC must be in the same IP network segment, and physical segment as well.

VS/TUN is the most scalable. The advantage is since servers connect each other by using IP-tunneling, the load balance and real servers can reside on different LAN , or even WAN.

I plan to setup a set of servers with load balance technique. Realserver run FreeBSD and Solaris system in different places on our business. So it is the only choice for us to use VS/TUN.

We install Fedora Core 5 Linux on our Load Balance server. Kernel-2.6.15 is default.

FreeBSD version is 5.4, we recompile the kernel in order to improve the performance. The default kernel can work too.

Solaris box runs Solaris 10 x86.

== Normal Linux Solaris/FreeBSD IP-tunneling ==

IP tunneling (IP encapsulation) is a technique to encapsulate IP datagram within IP datagram, which allows datagram destined for one IP address to be wrapped and redirected to another IP address. Many modern OS support IP tunneling, such as Linux, FreeBSD and Solaris.

Let’s do some test to setup normal Linux-Solaris and Linux-FreeBSD IPtunneling without LVS.

=== Testing environment ===

{|
|-
|| OS IP || Address || Tunnel IP Address
|-
|| Fedora Core 5 Linux || 10.0.0.1 || 172.16.0.1
|-
|| Solaris 10 x86 || 10.0.0.2 || 172.16.0.2
|-
|| FreeBSD 5.4 || 10.0.0.3 || 172.16.0.3
|}

=== Config Linux box ===

In order to create an IP-Tunneling between Linux box and Solaris box, run command:

ip tunnel add tun0 mode ipip remote 10.0.0.2 local 10.0.0.1

ifconfig tun0 172.16.0.1 pointopoint 172.16.0.2

ip tunnel add tun1 mode ipip remote 10.0.0.3 local 10.0.0.1

ifconfig tun1 172.16.0.1 pointopoint 172.16.0.3

The ”tun0” is a virtual NIC device name. It can be replaced by other names, such as ”abc0” or ”def0”.

=== Config Solaris box ===

To create an IP-tunneling, we have two methods, First method of creating tunnel is using three commands as below.

/sbin/ifconfig ip.tun0 plumb

/sbin/ifconfig ip.tun0 tsrc 10.0.0.2 tdst 10.0.0.1

/sbin/ifconfig ip.tun0 172.16.0.2 172.16.0.1

It takes effect immediately.

Second method is creating /etc/hostname.ip.tun0 file which contains two lines as below.

tsrc 10.0.0.2 tdst 10.0.0.1

172.16.0.2 172.16.0.1 netmask 255.255.255.255 up

If we use the second method, the tunnel will exist automatically after we reboot system.

=== Config FreeBSD box ===

On FreeBSD box, we also have two methods to create an IP-Tunneling, one way is to run three commands:

ifconfig gif0 create

ifconfig gif0 tunnel 10.0.0.3 10.0.0.1

ifconfig gif0 inet 172.16.0.3 172.168.0.1 netmask 255.255.255.0

If we use the second method, the tunnel will exist automatically after we reboot system.

Add three lines on /etc/rc.conf file as below, the tunnel will exist automatically after we reboot system.

gif_interfaces=”gif0”

gifconfig_gif0=”10.0.0.3 10.0.0.1”

ifconfig_gif0=”172.16.0.3 172.16.0.1 netmask 255.255.255.0”

=== Using IP-tunneling ===

In all the cases on Linux/FreeBSD/Solaris, the netmask value will be default if we ignore to set it.

After creating two tunnels, we can ping 10.0.0.2 and 10.0.0.3 at Linux box successfully, and ping 10.0.0.1 at Solaris or FreeBSD box successfully. All applications layers protocol will work correctly.

== Working example of VS/TUN ==

In VS/TUN mode, because realserver don’t send any datagram to Load balancer, we need not create any tunnel on Linux box and must create proper tunnel on FreeBSD/Solaris box.

=== Topological graph ===

[[Image:Topo.png|center|600px]]

We can treat all machines on internet and have their real ip. The client can send their requests to VIP of LB Server. LB Server redirects the requests realserver through IP-tunneling. Realserver can send datagram to client.

Because both the gif0 of FreeBSD and ip.tun0 both are NO-ARP device, they are invisible to client.

=== Config FreeBSD box ===

On FreeBSD box, edit /etc/rc.conf as below:

...

ifconfig lnc0=”inet 192.168.0.170 netmask 0xffffff00”

gif interfaces=”gif0”

gifconfig gif0=”192.168.0.170 192.168.10.100”

ifconfig gif0=”192.168.10.200 192.168.10.99 netmask 0xffffffff”

apache enable=”YES”

...

192.168.10.99 looks like a terminal of this tunnel. It can be any idle IP and never use.

=== Config Solaris box ===
On Solaris , edit /etc/hostname.ip.tun0 as below:

tsrc 192.168.0.180 tdst 192.168.10.100

192.168.10.200 192.168.10.99 netmask 255.255.255.255 up

192.168.10.99 looks like a terminal of this tunnel. It can be any IP address and never use it.

=== Config Load balancer ===

Run ipvsadm command on Linux Box: Linux kernel must have ip forward enable.

Edit /etc/sysctl.conf and set to:

net.ipv4.ip_forward = 1

While running command:

sysctl -p

It will take effect.

Run the scripts:

ipvsadm -C

ipvsadm -A -t 192.168.10.200:80 -s wlc

ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.170 -i

ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.180 -i

In this script, the port of LB Server and realerver must be same. Good idea is not setting the port of realserver.Now VS/TUN is finished. When a client access 192.168.10.200 at port 80, the datagram will be sent to 192.168.0.170 or 192.168.180 port 80.

== Conclusions and Future Work ==

I believe AIX and HP-UX can also work with VS/TUN, I have no Aix or HP-UX servers. Who can help to provide me with such an environment?

References:

[1] http://www.linuxvirtualserver.org

[2] http://www.triload.com/en/manual/index.html

[3] http://blog.kovyrin.net/2006/03/17/how-to-create-ip-ip-tunnel-between-freebsd-and-linux/

[4] http://docs.sun.com/app/docs/doc/817-0573/6mgc65bcq?a=view

[[Category:LVS Examples|Tunnel]]

LVS/TUN mode with FreeBSD and Solaris realserver

2006-08-16T09:36:19Z

Maluyao: /* Testing environment */

Author: Ma Luyao RHCE maluyao at gmail dot com

August 16, 2006

== Introduction ==

The Linux Virtual Server Project (LVS) is a project started by Dr. Wensong Zhang. LVS implements three load balance technology.

a. Network Address Translation (VS/NAT)

b. Direct Routing (VS/DR)

c. IP Tunneling (VS/TUN)

VS/NAT is easy to setup. The load balancer may be a bottleneck of the whole system where the number of servers is more than 20, because both the request packets and response packets need to be rewritten by the load balancer.

VS/DR has the best performance. VS/DR uses MAC-spoofing technology, so it requires that one of the load balancer’s NIC and the real server’s NIC must be in the same IP network segment, and physical segment as well.

VS/TUN is the most scalable. The advantage is since servers connect each other by using IP-tunneling, the load balance and real servers can reside on different LAN , or even WAN.

I plan to setup a set of servers with load balance technique. Realserver run FreeBSD and Solaris system in different places on our business. So it is the only choice for us to use VS/TUN.

We install Fedora Core 5 Linux on our Load Balance server. Kernel-2.6.15 is default.

FreeBSD version is 5.4, we recompile the kernel in order to improve the performance. The default kernel can work too.

Solaris box runs Solaris 10 x86.

== Normal Linux Solaris/FreeBSD IP-tunneling ==

IP tunneling (IP encapsulation) is a technique to encapsulate IP datagram within IP datagram, which allows datagram destined for one IP address to be wrapped and redirected to another IP address. Many modern OS support IP tunneling, such as Linux, FreeBSD and Solaris.

Let’s do some test to setup normal Linux-Solaris and Linux-FreeBSD IPtunneling without LVS.

=== Testing environment ===

OS IP Address Tunnel IP Address

Fedora Core 5 Linux 10.0.0.1 172.16.0.1

Solaris 10 x86 10.0.0.2 172.16.0.2

FreeBSD 5.4 10.0.0.3 172.16.0.3

=== Config Linux box ===

In order to create an IP-Tunneling between Linux box and Solaris box, run command:

ip tunnel add tun0 mode ipip remote 10.0.0.2 local 10.0.0.1

ifconfig tun0 172.16.0.1 pointopoint 172.16.0.2

ip tunnel add tun1 mode ipip remote 10.0.0.3 local 10.0.0.1

ifconfig tun1 172.16.0.1 pointopoint 172.16.0.3

The ”tun0” is a virtual NIC device name. It can be replaced by other names, such as ”abc0” or ”def0”.

=== Config Solaris box ===

To create an IP-tunneling, we have two methods, First method of creating tunnel is using three commands as below.

/sbin/ifconfig ip.tun0 plumb

/sbin/ifconfig ip.tun0 tsrc 10.0.0.2 tdst 10.0.0.1

/sbin/ifconfig ip.tun0 172.16.0.2 172.16.0.1

It takes effect immediately.

Second method is creating /etc/hostname.ip.tun0 file which contains two lines as below.

tsrc 10.0.0.2 tdst 10.0.0.1

172.16.0.2 172.16.0.1 netmask 255.255.255.255 up

If we use the second method, the tunnel will exist automatically after we reboot system.

=== Config FreeBSD box ===

On FreeBSD box, we also have two methods to create an IP-Tunneling, one way is to run three commands:

ifconfig gif0 create

ifconfig gif0 tunnel 10.0.0.3 10.0.0.1

ifconfig gif0 inet 172.16.0.3 172.168.0.1 netmask 255.255.255.0

If we use the second method, the tunnel will exist automatically after we reboot system.

Add three lines on /etc/rc.conf file as below, the tunnel will exist automatically after we reboot system.

gif interfaces=”gif0”

gifconfig gif0=”10.0.0.3 10.0.0.1”

ifconfig gif0=”172.16.0.3 172.16.0.1 netmask 255.255.255.0”

=== Using IP-tunneling ===

In all the cases on Linux/FreeBSD/Solaris, the netmask value will be default if we ignore to set it.

After creating two tunnels, we can ping 10.0.0.2 and 10.0.0.3 at Linux box successfully, and ping 10.0.0.1 at Solaris or FreeBSD box successfully. All applications layers protocol will work correctly.

== Working example of VS/TUN ==

In VS/TUN mode, because realserver don’t send any datagram to Load balancer, we need not create any tunnel on Linux box and must create proper tunnel on FreeBSD/Solaris box.

=== Topological graph ===

[[Image:Topo.png]]

We can treat all machines on internet and have their real ip. The client can send their requests to VIP of LB Server. LB Server redirects the requests realserver through IP-tunneling. Realserver can send datagram to client.

Because both the gif0 of FreeBSD and ip.tun0 both are NO-ARP device, they are invisible to client.

=== Config FreeBSD box ===

On FreeBSD box, edit /etc/rc.conf as below:

...

ifconfig lnc0=”inet 192.168.0.170 netmask 0xffffff00”

gif interfaces=”gif0”

gifconfig gif0=”192.168.0.170 192.168.10.100”

ifconfig gif0=”192.168.10.200 192.168.10.99 netmask 0xffffffff”

apache enable=”YES”

...

192.168.10.99 looks like a terminal of this tunnel. It can be any idle IP and never use.

=== Config Solaris box ===
On Solaris , edit /etc/hostname.ip.tun0 as below:

tsrc 192.168.0.180 tdst 192.168.10.100

192.168.10.200 192.168.10.99 netmask 255.255.255.255 up

192.168.10.99 looks like a terminal of this tunnel. It can be any IP address and never use it.

=== Config Load balancer ===

Run ipvsadm command on Linux Box: Linux kernel must have ip forward enable.

Edit /etc/sysctl.conf and set to:

net.ipv4.ip forward = 1

While running command:

sysctl -p

It will take effect.

Run the scripts:

ipvsadm -C

ipvsadm -A -t 192.168.10.200:80 -s wlc

ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.170 -i

ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.180 -i

In this script, the port of LB Server and realerver must be same. Good idea is not setting the port of realserver.Now VS/TUN is finished. When a client access 192.168.10.200 at port 80, the datagram will be sent to 192.168.0.170 or 192.168.180 port 80.

== Conclusions and Future Work ==

I believe AIX and HP-UX can also work with VS/TUN, I have no Aix or HP-UX servers. Who can help to provide me with such an environment?

References:

[1] http://www.linuxvirtualserver.org

[2] http://www.triload.com/en/manual/index.html

[3] http://blog.kovyrin.net/2006/03/17/how-to-create-ip-ip-tunnel-between-freebsd-and-linux/

[4] http://docs.sun.com/app/docs/doc/817-0573/6mgc65bcq?a=view

LVS/TUN mode with FreeBSD and Solaris realserver

2006-08-16T08:57:03Z

Maluyao:

Author: Ma Luyao RHCE maluyao at gmail dot com

August 16, 2006

== Introduction ==

The Linux Virtual Server Project (LVS) is a project started by Dr. Wensong Zhang. LVS implements three load balance technology.

a. Network Address Translation (VS/NAT)

b. Direct Routing (VS/DR)

c. IP Tunneling (VS/TUN)

VS/NAT is easy to setup. The load balancer may be a bottleneck of the whole system where the number of servers is more than 20, because both the request packets and response packets need to be rewritten by the load balancer.

VS/DR has the best performance. VS/DR uses MAC-spoofing technology, so it requires that one of the load balancer’s NIC and the real server’s NIC must be in the same IP network segment, and physical segment as well.

VS/TUN is the most scalable. The advantage is since servers connect each other by using IP-tunneling, the load balance and real servers can reside on different LAN , or even WAN.

I plan to setup a set of servers with load balance technique. Realserver run FreeBSD and Solaris system in different places on our business. So it is the only choice for us to use VS/TUN.

We install Fedora Core 5 Linux on our Load Balance server. Kernel-2.6.15 is default.

FreeBSD version is 5.4, we recompile the kernel in order to improve the performance. The default kernel can work too.

Solaris box runs Solaris 10 x86.

== Normal Linux Solaris/FreeBSD IP-tunneling ==

IP tunneling (IP encapsulation) is a technique to encapsulate IP datagram within IP datagram, which allows datagram destined for one IP address to be wrapped and redirected to another IP address. Many modern OS support IP tunneling, such as Linux, FreeBSD and Solaris.

Let’s do some test to setup normal Linux-Solaris and Linux-FreeBSD IPtunneling without LVS.

=== Testing environment ===

OS IP Address Tunnel IP Address

Fedora Core 5 Linux 10.0.0.1 172.16.0.1

Solaris 10 x86 10.0.0.2 172.16.0.2

FreeBSD 5.4 10.0.0.3 172.16.0.3

=== Config Linux box ===

In order to create an IP-Tunneling between Linux box and Solaris box, run command:

ip tunnel add tun0 mode ipip remote 10.0.0.2 local 10.0.0.1

ifconfig tun0 172.16.0.1 pointopoint 172.16.0.2

ip tunnel add tun1 mode ipip remote 10.0.0.3 local 10.0.0.1

ifconfig tun1 172.16.0.1 pointopoint 172.16.0.3

The ”tun0” is a virtual NIC device name. It can be replaced by other names, such as ”abc0” or ”def0”.

=== Config Solaris box ===

To create an IP-tunneling, we have two methods, First method of creating tunnel is using three commands as below.

/sbin/ifconfig ip.tun0 plumb

/sbin/ifconfig ip.tun0 tsrc 10.0.0.2 tdst 10.0.0.1

/sbin/ifconfig ip.tun0 172.16.0.2 172.16.0.1

It takes effect immediately.

Second method is creating /etc/hostname.ip.tun0 file which contains two lines as below.

tsrc 10.0.0.2 tdst 10.0.0.1

172.16.0.2 172.16.0.1 netmask 255.255.255.255 up

If we use the second method, the tunnel will exist automatically after we reboot system.

=== Config FreeBSD box ===

On FreeBSD box, we also have two methods to create an IP-Tunneling, one way is to run three commands:

ifconfig gif0 create

ifconfig gif0 tunnel 10.0.0.3 10.0.0.1

ifconfig gif0 inet 172.16.0.3 172.168.0.1 netmask 255.255.255.0

If we use the second method, the tunnel will exist automatically after we reboot system.

Add three lines on /etc/rc.conf file as below, the tunnel will exist automatically after we reboot system.

gif interfaces=”gif0”

gifconfig gif0=”10.0.0.3 10.0.0.1”

ifconfig gif0=”172.16.0.3 172.16.0.1 netmask 255.255.255.0”

=== Using IP-tunneling ===

In all the cases on Linux/FreeBSD/Solaris, the netmask value will be default if we ignore to set it.

After creating two tunnels, we can ping 10.0.0.2 and 10.0.0.3 at Linux box successfully, and ping 10.0.0.1 at Solaris or FreeBSD box successfully. All applications layers protocol will work correctly.

== Working example of VS/TUN ==

In VS/TUN mode, because realserver don’t send any datagram to Load balancer, we need not create any tunnel on Linux box and must create proper tunnel on FreeBSD/Solaris box.

=== Topological graph ===

[[Image:Topo.png]]

We can treat all machines on internet and have their real ip. The client can send their requests to VIP of LB Server. LB Server redirects the requests realserver through IP-tunneling. Realserver can send datagram to client.

Because both the gif0 of FreeBSD and ip.tun0 both are NO-ARP device, they are invisible to client.

=== Config FreeBSD box ===

On FreeBSD box, edit /etc/rc.conf as below:

...

ifconfig lnc0=”inet 192.168.0.170 netmask 0xffffff00”

gif interfaces=”gif0”

gifconfig gif0=”192.168.0.170 192.168.10.100”

ifconfig gif0=”192.168.10.200 192.168.10.99 netmask 0xffffffff”

apache enable=”YES”

...

192.168.10.99 looks like a terminal of this tunnel. It can be any idle IP and never use.

=== Config Solaris box ===
On Solaris , edit /etc/hostname.ip.tun0 as below:

tsrc 192.168.0.180 tdst 192.168.10.100

192.168.10.200 192.168.10.99 netmask 255.255.255.255 up

192.168.10.99 looks like a terminal of this tunnel. It can be any IP address and never use it.

=== Config Load balancer ===

Run ipvsadm command on Linux Box: Linux kernel must have ip forward enable.

Edit /etc/sysctl.conf and set to:

net.ipv4.ip forward = 1

While running command:

sysctl -p

It will take effect.

Run the scripts:

ipvsadm -C

ipvsadm -A -t 192.168.10.200:80 -s wlc

ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.170 -i

ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.180 -i

In this script, the port of LB Server and realerver must be same. Good idea is not setting the port of realserver.Now VS/TUN is finished. When a client access 192.168.10.200 at port 80, the datagram will be sent to 192.168.0.170 or 192.168.180 port 80.

== Conclusions and Future Work ==

I believe AIX and HP-UX can also work with VS/TUN, I have no Aix or HP-UX servers. Who can help to provide me with such an environment?

References:

[1] http://www.linuxvirtualserver.org

[2] http://www.triload.com/en/manual/index.html

[3] http://blog.kovyrin.net/2006/03/17/how-to-create-ip-ip-tunnel-between-freebsd-and-linux/

[4] http://docs.sun.com/app/docs/doc/817-0573/6mgc65bcq?a=view

LVS/TUN mode with FreeBSD and Solaris realserver

2006-08-16T08:47:17Z

Maluyao:

Author: Ma Luyao RHCE maluyao at gmail dot com

August 16, 2006

== Introduction ==

The Linux Virtual Server Project (LVS) is a project started by Dr. Wensong Zhang. LVS implements three load balance technology.

a. Network Address Translation (VS/NAT)

b. Direct Routing (VS/DR)

c. IP Tunneling (VS/TUN)

VS/NAT is easy to setup. The load balancer may be a bottleneck of the whole system where the number of servers is more than 20, because both the request packets and response packets need to be rewritten by the load balancer.

VS/DR has the best performance. VS/DR uses MAC-spoofing technology, so it requires that one of the load balancer’s NIC and the real server’s NIC must be in the same IP network segment, and physical segment as well.

VS/TUN is the most scalable. The advantage is since servers connect each other by using IP-tunneling, the load balance and real servers can reside on different LAN , or even WAN.

I plan to setup a set of servers with load balance technique. Realserver run FreeBSD and Solaris system in different places on our business. So it is the only choice for us to use VS/TUN.

We install Fedora Core 5 Linux on our Load Balance server. Kernel-2.6.15 is default.

FreeBSD version is 5.4, we recompile the kernel in order to improve the performance. The default kernel can work too.

Solaris box runs Solaris 10 x86.

== Normal Linux Solaris IP-tunneling ==

IP tunneling (IP encapsulation) is a technique to encapsulate IP datagram within IP datagram, which allows datagram destined for one IP address to be wrapped and redirected to another IP address. Many modern OS support IP tunneling, such as Linux, FreeBSD and Solaris.

Let’s do some test to setup normal Linux-Solaris and Linux-FreeBSD IPtunneling without LVS.

=== Testing environment ===

OS IP Address Tunnel IP Address

Fedora Core 5 Linux 10.0.0.1 172.16.0.1

Solaris 10 x86 10.0.0.2 172.16.0.2

FreeBSD 5.4 10.0.0.3 172.16.0.3

=== Config Linux box ===

In order to create an IP-Tunneling between Linux box and Solaris box, run command:

ip tunnel add tun0 mode ipip remote 10.0.0.2 local 10.0.0.1

ifconfig tun0 172.16.0.1 pointopoint 172.16.0.2

ip tunnel add tun1 mode ipip remote 10.0.0.3 local 10.0.0.1

ifconfig tun1 172.16.0.1 pointopoint 172.16.0.3

The ”tun0” is a virtual NIC device name. It can be replaced by other names, such as ”abc0” or ”def0”.

=== Config Solaris box ===

To create an IP-tunneling, we have two methods, First method of creating tunnel is using three commands as below.

/sbin/ifconfig ip.tun0 plumb

/sbin/ifconfig ip.tun0 tsrc 10.0.0.2 tdst 10.0.0.1

/sbin/ifconfig ip.tun0 172.16.0.2 172.16.0.1

It takes effect immediately.

Second method is creating /etc/hostname.ip.tun0 file which contains two lines as below.

tsrc 10.0.0.2 tdst 10.0.0.1

172.16.0.2 172.16.0.1 netmask 255.255.255.255 up

If we use the second method, the tunnel will exist automatically after we reboot system.

=== Config FreeBSD box ===

On FreeBSD box, we also have two methods to create an IP-Tunneling, one way is to run three commands:

ifconfig gif0 create

ifconfig gif0 tunnel 10.0.0.3 10.0.0.1

ifconfig gif0 inet 172.16.0.3 172.168.0.1 netmask 255.255.255.0

If we use the second method, the tunnel will exist automatically after we reboot system.

Add three lines on /etc/rc.conf file as below, the tunnel will exist automatically after we reboot system.

gif interfaces=”gif0”

gifconfig gif0=”10.0.0.3 10.0.0.1”

ifconfig gif0=”172.16.0.3 172.16.0.1 netmask 255.255.255.0”

=== Using IP-tunneling ===

In all the cases on Linux/FreeBSD/Solaris, the netmask value will be default if we ignore to set it.

After creating two tunnels, we can ping 10.0.0.2 and 10.0.0.3 at Linux box successfully, and ping 10.0.0.1 at Solaris or FreeBSD box successfully. All applications layers protocol will work correctly.

== Working example of VS/TUN ==

In VS/TUN mode, because realserver don’t send any datagram to Load balancer, we need not create any tunnel on Linux box and must create proper tunnel on FreeBSD/Solaris box.

=== Topological graph ===

[[Image:Topo.png]]

We can treat all machines on internet and have their real ip. The client can send their requests to VIP of LB Server. LB Server redirects the requests realserver through IP-tunneling. Realserver can send datagram to client.

Because both the gif0 of FreeBSD and ip.tun0 both are NO-ARP device, they are invisible to client.

=== Config FreeBSD box ===

On FreeBSD box, edit /etc/rc.conf as below:

...

ifconfig lnc0=”inet 192.168.0.170 netmask 0xffffff00”

gif interfaces=”gif0”

gifconfig gif0=”192.168.0.170 192.168.10.100”

ifconfig gif0=”192.168.10.200 192.168.10.99 netmask 0xffffffff”

apache enable=”YES”

...

192.168.10.99 looks like a terminal of this tunnel. It can be any idle IP and never use.

=== Config Solaris box ===
On Solaris , edit /etc/hostname.ip.tun0 as below:

tsrc 192.168.0.180 tdst 192.168.10.100

192.168.10.200 192.168.10.99 netmask 255.255.255.255 up

192.168.10.99 looks like a terminal of this tunnel. It can be any IP address and never use it.

=== Config Load balancer ===

Run ipvsadm command on Linux Box: Linux kernel must have ip forward enable.

Edit /etc/sysctl.conf and set to:

net.ipv4.ip forward = 1

While running command:

sysctl -p

It will take effect.

Run the scripts:

ipvsadm -C

ipvsadm -A -t 192.168.10.200:80 -s wlc

ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.170 -i

ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.180 -i

In this script, the port of LB Server and realerver must be same. Good idea is not setting the port of realserver.Now VS/TUN is finished. When a client access 192.168.10.200 at port 80, the datagram will be sent to 192.168.0.170 or 192.168.180 port 80.

== Conclusions and Future Work ==

I believe AIX and HP-UX can also work with VS/TUN, I have no Aix or HP-UX servers. Who can help to provide me with such an environment?

References:

[1] http://www.linuxvirtualserver.org

[2] http://www.triload.com/en/manual/index.html

[3] http://blog.kovyrin.net/2006/03/17/how-to-create-ip-ip-tunnel-between-freebsd-and-linux/

[4] http://docs.sun.com/app/docs/doc/817-0573/6mgc65bcq?a=view

LVS/TUN mode with FreeBSD and Solaris realserver

2006-08-16T08:46:24Z

Maluyao:

LVS/TUN mode with FreeBSD and Solaris

Author: Ma Luyao RHCE maluyao at gmail dot com

August 16, 2006

== Introduction ==

The Linux Virtual Server Project (LVS) is a project started by Dr. Wensong Zhang. LVS implements three load balance technology.

a. Network Address Translation (VS/NAT)

b. Direct Routing (VS/DR)

c. IP Tunneling (VS/TUN)

VS/NAT is easy to setup. The load balancer may be a bottleneck of the whole system where the number of servers is more than 20, because both the request packets and response packets need to be rewritten by the load balancer.

VS/DR has the best performance. VS/DR uses MAC-spoofing technology, so it requires that one of the load balancer’s NIC and the real server’s NIC must be in the same IP network segment, and physical segment as well.

VS/TUN is the most scalable. The advantage is since servers connect each other by using IP-tunneling, the load balance and real servers can reside on different LAN , or even WAN.

I plan to setup a set of servers with load balance technique. Realserver run FreeBSD and Solaris system in different places on our business. So it is the only choice for us to use VS/TUN.

We install Fedora Core 5 Linux on our Load Balance server. Kernel-2.6.15 is default.

FreeBSD version is 5.4, we recompile the kernel in order to improve the performance. The default kernel can work too.

Solaris box runs Solaris 10 x86.

== Normal Linux Solaris IP-tunneling ==

IP tunneling (IP encapsulation) is a technique to encapsulate IP datagram within IP datagram, which allows datagram destined for one IP address to be wrapped and redirected to another IP address. Many modern OS support IP tunneling, such as Linux, FreeBSD and Solaris.

Let’s do some test to setup normal Linux-Solaris and Linux-FreeBSD IPtunneling without LVS.

=== Testing environment ===

OS IP Address Tunnel IP Address

Fedora Core 5 Linux 10.0.0.1 172.16.0.1

Solaris 10 x86 10.0.0.2 172.16.0.2

FreeBSD 5.4 10.0.0.3 172.16.0.3

=== Config Linux box ===

In order to create an IP-Tunneling between Linux box and Solaris box, run command:

ip tunnel add tun0 mode ipip remote 10.0.0.2 local 10.0.0.1

ifconfig tun0 172.16.0.1 pointopoint 172.16.0.2

ip tunnel add tun1 mode ipip remote 10.0.0.3 local 10.0.0.1

ifconfig tun1 172.16.0.1 pointopoint 172.16.0.3

The ”tun0” is a virtual NIC device name. It can be replaced by other names, such as ”abc0” or ”def0”.

=== Config Solaris box ===

To create an IP-tunneling, we have two methods, First method of creating tunnel is using three commands as below.

/sbin/ifconfig ip.tun0 plumb

/sbin/ifconfig ip.tun0 tsrc 10.0.0.2 tdst 10.0.0.1

/sbin/ifconfig ip.tun0 172.16.0.2 172.16.0.1

It takes effect immediately.

Second method is creating /etc/hostname.ip.tun0 file which contains two lines as below.

tsrc 10.0.0.2 tdst 10.0.0.1

172.16.0.2 172.16.0.1 netmask 255.255.255.255 up

If we use the second method, the tunnel will exist automatically after we reboot system.

=== Config FreeBSD box ===

On FreeBSD box, we also have two methods to create an IP-Tunneling, one way is to run three commands:

ifconfig gif0 create

ifconfig gif0 tunnel 10.0.0.3 10.0.0.1

ifconfig gif0 inet 172.16.0.3 172.168.0.1 netmask 255.255.255.0

If we use the second method, the tunnel will exist automatically after we reboot system.

Add three lines on /etc/rc.conf file as below, the tunnel will exist automatically after we reboot system.

gif interfaces=”gif0”

gifconfig gif0=”10.0.0.3 10.0.0.1”

ifconfig gif0=”172.16.0.3 172.16.0.1 netmask 255.255.255.0”

=== Using IP-tunneling ===

In all the cases on Linux/FreeBSD/Solaris, the netmask value will be default if we ignore to set it.

After creating two tunnels, we can ping 10.0.0.2 and 10.0.0.3 at Linux box successfully, and ping 10.0.0.1 at Solaris or FreeBSD box successfully. All applications layers protocol will work correctly.

== Working example of VS/TUN ==

In VS/TUN mode, because realserver don’t send any datagram to Load balancer, we need not create any tunnel on Linux box and must create proper tunnel on FreeBSD/Solaris box.

=== Topological graph ===

[[Image:Topo.png]]

We can treat all machines on internet and have their real ip. The client can send their requests to VIP of LB Server. LB Server redirects the requests realserver through IP-tunneling. Realserver can send datagram to client.

Because both the gif0 of FreeBSD and ip.tun0 both are NO-ARP device, they are invisible to client.

=== Config FreeBSD box ===

On FreeBSD box, edit /etc/rc.conf as below:

...

ifconfig lnc0=”inet 192.168.0.170 netmask 0xffffff00”

gif interfaces=”gif0”

gifconfig gif0=”192.168.0.170 192.168.10.100”

ifconfig gif0=”192.168.10.200 192.168.10.99 netmask 0xffffffff”

apache enable=”YES”

...

192.168.10.99 looks like a terminal of this tunnel. It can be any idle IP and never use.

=== Config Solaris box ===
On Solaris , edit /etc/hostname.ip.tun0 as below:

tsrc 192.168.0.180 tdst 192.168.10.100

192.168.10.200 192.168.10.99 netmask 255.255.255.255 up

192.168.10.99 looks like a terminal of this tunnel. It can be any IP address and never use it.

=== Config Load balancer ===

Run ipvsadm command on Linux Box: Linux kernel must have ip forward enable.

Edit /etc/sysctl.conf and set to:

net.ipv4.ip forward = 1

While running command:

sysctl -p

It will take effect.

Run the scripts:

ipvsadm -C

ipvsadm -A -t 192.168.10.200:80 -s wlc

ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.170 -i

ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.180 -i

In this script, the port of LB Server and realerver must be same. Good idea is not setting the port of realserver.Now VS/TUN is finished. When a client access 192.168.10.200 at port 80, the datagram will be sent to 192.168.0.170 or 192.168.180 port 80.

== Conclusions and Future Work ==

I believe AIX and HP-UX can also work with VS/TUN, I have no Aix or HP-UX servers. Who can help to provide me with such an environment?

References:

[1] http://www.linuxvirtualserver.org

[2] http://www.triload.com/en/manual/index.html

[3] http://blog.kovyrin.net/2006/03/17/how-to-create-ip-ip-tunnel-between-freebsd-and-linux/

[4] http://docs.sun.com/app/docs/doc/817-0573/6mgc65bcq?a=view

LVS/TUN mode with FreeBSD and Solaris realserver

2006-08-16T08:39:41Z

Maluyao: LVS/TUN mode with FreeBSD and Solaris realserver

LVS/TUN mode with FreeBSD and Solaris realserver

2006-08-16T08:38:28Z

Maluyao:

LVS/TUN mode with FreeBSD and Solaris

Author: Ma Luyao RHCE maluyao at gmail dot com

August 16, 2006

== Introduction ==

The Linux Virtual Server Project (LVS) is a project started by Dr. Wensong Zhang. LVS implements three load balance technology.

a. Network Address Translation (VS/NAT)

b. Direct Routing (VS/DR)

c. IP Tunneling (VS/TUN)

VS/NAT is easy to setup. The load balancer may be a bottleneck of the whole system where the number of servers is more than 20, because both the request packets and response packets need to be rewritten by the load balancer.

VS/DR has the best performance. VS/DR uses MAC-spoofing technology, so it requires that one of the load balancer’s NIC and the real server’s NIC must be in the same IP network segment, and physical segment as well.

VS/TUN is the most scalable. The advantage is since servers connect each other by using IP-tunneling, the load balance and real servers can reside on different LAN , or even WAN.

I plan to setup a set of servers with load balance technique. Realserver run FreeBSD and Solaris system in different places on our business. So it is the only choice for us to use VS/TUN.

We install Fedora Core 5 Linux on our Load Balance server. Kernel-2.6.15 is default.

FreeBSD version is 5.4, we recompile the kernel in order to improve the performance. The default kernel can work too.

Solaris box runs Solaris 10 x86.

== Normal Linux Solaris IP-tunneling ==

IP tunneling (IP encapsulation) is a technique to encapsulate IP datagram within IP datagram, which allows datagram destined for one IP address to be wrapped and redirected to another IP address. Many modern OS support IP tunneling, such as Linux, FreeBSD and Solaris.

Let’s do some test to setup normal Linux-Solaris and Linux-FreeBSD IPtunneling without LVS.

=== Testing environment ===

OS IP Address Tunnel IP Address

Fedora Core 5 Linux 10.0.0.1 172.16.0.1

Solaris 10 x86 10.0.0.2 172.16.0.2

FreeBSD 5.4 10.0.0.3 172.16.0.3

=== Config Linux box ===

In order to create an IP-Tunneling between Linux box and Solaris box, run command:

ip tunnel add tun0 mode ipip remote 10.0.0.2 local 10.0.0.1

ifconfig tun0 172.16.0.1 pointopoint 172.16.0.2

ip tunnel add tun1 mode ipip remote 10.0.0.3 local 10.0.0.1

ifconfig tun1 172.16.0.1 pointopoint 172.16.0.3

The ”tun0” is a virtual NIC device name. It can be replaced by other names, such as ”abc0” or ”def0”.

=== Config Solaris box ===

To create an IP-tunneling, we have two methods, First method of creating tunnel is using three commands as below.

/sbin/ifconfig ip.tun0 plumb

/sbin/ifconfig ip.tun0 tsrc 10.0.0.2 tdst 10.0.0.1

/sbin/ifconfig ip.tun0 172.16.0.2 172.16.0.1

It takes effect immediately.

Second method is creating /etc/hostname.ip.tun0 file which contains two lines as below.

tsrc 10.0.0.2 tdst 10.0.0.1

172.16.0.2 172.16.0.1 netmask 255.255.255.255 up

If we use the second method, the tunnel will exist automatically after we reboot system.

=== Config FreeBSD box ===

On FreeBSD box, we also have two methods to create an IP-Tunneling, one way is to run three commands:

ifconfig gif0 create

ifconfig gif0 tunnel 10.0.0.3 10.0.0.1

ifconfig gif0 inet 172.16.0.3 172.168.0.1 netmask 255.255.255.0

If we use the second method, the tunnel will exist automatically after we reboot system.

Add three lines on /etc/rc.conf file as below, the tunnel will exist automatically after we reboot system.

gif interfaces=”gif0”

gifconfig gif0=”10.0.0.3 10.0.0.1”

ifconfig gif0=”172.16.0.3 172.16.0.1 netmask 255.255.255.0”

=== Using IP-tunneling ===

In all the cases on Linux/FreeBSD/Solaris, the netmask value will be default if we ignore to set it.

After creating two tunnels, we can ping 10.0.0.2 and 10.0.0.3 at Linux box successfully, and ping 10.0.0.1 at Solaris or FreeBSD box successfully. All applications layers protocol will work correctly.

== Working example of VS/TUN ==

In VS/TUN mode, because realserver don’t send any datagram to Load balancer, we need not create any tunnel on Linux box and must create proper tunnel on FreeBSD/Solaris box.

=== Topological graph ===

[[Image:Topo.png]]

We can treat all machines on internet and have their real ip. The client can send their requests to VIP of LB Server. LB Server redirects the requests realserver through IP-tunneling. Realserver can send datagram to client.

Because both the gif0 of FreeBSD and ip.tun0 both are NO-ARP device, they are invisible to client.

=== Config FreeBSD box ===

On FreeBSD box, edit /etc/rc.conf as below:

...

ifconfig lnc0=”inet 192.168.0.170 netmask 0xffffff00”

gif interfaces=”gif0”

gifconfig gif0=”192.168.0.170 192.168.10.100”

ifconfig gif0=”192.168.10.200 192.168.10.99 netmask 0xffffffff”

apache enable=”YES”

...

192.168.10.99 looks like a terminal of this tunnel. It can be any idle IP and never use.

=== Config Solaris box ===
On Solaris , edit /etc/hostname.ip.tun0 as below:

tsrc 192.168.0.180 tdst 192.168.10.100

192.168.10.200 192.168.10.99 netmask 255.255.255.255 up

192.168.10.99 looks like a terminal of this tunnel. It can be any IP address and never use it.

=== Config Load balancer ===

Run ipvsadm command on Linux Box: Linux kernel must have ip forward enable.

Edit /etc/sysctl.conf and set to:

net.ipv4.ip forward = 1

While running command:

sysctl -p

It will take effect.

Run the scripts:

ipvsadm -C

ipvsadm -A -t 192.168.10.200:80 -s wlc

ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.170 -i

ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.180 -i

In this script, the port of LB Server and realerver must be same. Good idea is not setting the port of realserver.Now VS/TUN is finished. When a client access 192.168.10.200 at port 80, the datagram will be sent to 192.168.0.170 or 192.168.180 port 80.

== Conclusions and Future Work ==

I believe AIX and HP-UX can also work with VS/TUN, I have no Aix or HP-UX servers. Who can help to provide me with such an environment?
References:

[1] http://www.linuxvirtualserver.org

[2] http://www.triload.com/en/manual/index.html

LVS/TUN mode with FreeBSD and Solaris realserver

2006-08-16T08:26:44Z

Maluyao:

LVS/TUN mode with FreeBSD and Solaris

Author: Ma Luyao RHCE maluyao at gmail dot com

August 16, 2006

== 1 Introduction ==

The Linux Virtual Server Project (LVS) is a project started by Dr. Wensong Zhang. LVS implements three load balance technology.

a. Network Address Translation (VS/NAT)

b. Direct Routing (VS/DR)

c. IP Tunneling (VS/TUN)

VS/NAT is easy to setup. The load balancer may be a bottleneck of the whole system where the number of servers is more than 20, because both the request packets and response packets need to be rewritten by the load balancer.

VS/DR has the best performance. VS/DR uses MAC-spoofing technology, so it requires that one of the load balancer’s NIC and the real server’s NIC must be in the same IP network segment, and physical segment as well.

VS/TUN is the most scalable. The advantage is since servers connect each other by using IP-tunneling, the load balance and real servers can reside on different LAN , or even WAN.

I plan to setup a set of servers with load balance technique. Realserver run FreeBSD and Solaris system in different places on our business. So it is the only choice for us to use VS/TUN.

We install Fedora Core 5 Linux on our Load Balance server. Kernel-2.6.15 is default.

FreeBSD version is 5.4, we recompile the kernel in order to improve the performance. The default kernel can work too.

Solaris box runs Solaris 10 x86.

2 Normal Linux Solaris IP-tunneling

IP tunneling (IP encapsulation) is a technique to encapsulate IP datagram within IP datagram, which allows datagram destined for one IP address to be wrapped and redirected to another IP address. Many modern OS support IP tunneling, such as Linux, FreeBSD and Solaris.

Let’s do some test to setup normal Linux-Solaris and Linux-FreeBSD IPtunneling without LVS.

2.1 Testing environment

OS IP Address Tunnel IP Address

Fedora Core 5 Linux 10.0.0.1 172.16.0.1

Solaris 10 x86 10.0.0.2 172.16.0.2

FreeBSD 5.4 10.0.0.3 172.16.0.3

2.2 Config Linux box

In order to create an IP-Tunneling between Linux box and Solaris box, run command:

ip tunnel add tun0 mode ipip remote 10.0.0.2 local 10.0.0.1

ifconfig tun0 172.16.0.1 pointopoint 172.16.0.2

ip tunnel add tun1 mode ipip remote 10.0.0.3 local 10.0.0.1

ifconfig tun1 172.16.0.1 pointopoint 172.16.0.3

The ”tun0” is a virtual NIC device name. It can be replaced by other names, such as ”abc0” or ”def0”.

2.3 Config Solaris box

To create an IP-tunneling, we have two methods, First method of creating tunnel is using three commands as below.

/sbin/ifconfig ip.tun0 plumb

/sbin/ifconfig ip.tun0 tsrc 10.0.0.2 tdst 10.0.0.1

/sbin/ifconfig ip.tun0 172.16.0.2 172.16.0.1

It takes effect immediately.

Second method is creating /etc/hostname.ip.tun0 file which contains two lines as below.

tsrc 10.0.0.2 tdst 10.0.0.1

172.16.0.2 172.16.0.1 netmask 255.255.255.255 up

If we use the second method, the tunnel will exist automatically after we reboot system.

2.4 Config FreeBSD box

On FreeBSD box, we also have two methods to create an IP-Tunneling, one way is to run three commands:

ifconfig gif0 create

ifconfig gif0 tunnel 10.0.0.3 10.0.0.1

ifconfig gif0 inet 172.16.0.3 172.168.0.1 netmask 255.255.255.0

If we use the second method, the tunnel will exist automatically after we reboot system.

Add three lines on /etc/rc.conf file as below, the tunnel will exist automatically after we reboot system.

gif interfaces=”gif0”

gifconfig gif0=”10.0.0.3 10.0.0.1”

ifconfig gif0=”172.16.0.3 172.16.0.1 netmask 255.255.255.0”

2.5 Using IP-tunneling

In all the cases on Linux/FreeBSD/Solaris, the netmask value will be default if we ignore to set it.

After creating two tunnels, we can ping 10.0.0.2 and 10.0.0.3 at Linux box successfully, and ping 10.0.0.1 at Solaris or FreeBSD box successfully. All applications layers protocol will work correctly.

==
3 Working example of VS/TUN ==

In VS/TUN mode, because realserver don’t send any datagram to Load balancer, we need not create any tunnel on Linux box and must create proper tunnel on FreeBSD/Solaris box.

3.1 Topological graph

[[Image:Topo.png]]

We can treat all machines on internet and have their real ip. The client can send their requests to VIP of LB Server. LB Server redirects the requests realserver through IP-tunneling. Realserver can send datagram to client.

Because both the gif0 of FreeBSD and ip.tun0 both are NO-ARP device, they are invisible to client.

3.2 Config FreeBSD box

On FreeBSD box, edit /etc/rc.conf as below:

...

ifconfig lnc0=”inet 192.168.0.170 netmask 0xffffff00”

gif interfaces=”gif0”

gifconfig gif0=”192.168.0.170 192.168.10.100”

ifconfig gif0=”192.168.10.200 192.168.10.99 netmask 0xffffffff”

apache enable=”YES”

...

192.168.10.99 looks like a terminal of this tunnel. It can be any idle IP and never use.

3.3 Config Solaris box
On Solaris , edit /etc/hostname.ip.tun0 as below:

tsrc 192.168.0.180 tdst 192.168.10.100

192.168.10.200 192.168.10.99 netmask 255.255.255.255 up

192.168.10.99 looks like a terminal of this tunnel. It can be any IP address and never use it.

3.4 Config Load balancer

Run ipvsadm command on Linux Box: Linux kernel must have ip forward enable.

Edit /etc/sysctl.conf and set to:

net.ipv4.ip forward = 1

While running command:

sysctl -p

It will take effect.

Run the scripts:

ipvsadm -C

ipvsadm -A -t 192.168.10.200:80 -s wlc

ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.170 -i

ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.180 -i

In this script, the port of LB Server and realerver must be same. Good idea is not setting the port of realserver.Now VS/TUN is finished. When a client access 192.168.10.200 at port 80, the datagram will be sent to 192.168.0.170 or 192.168.180 port 80.

==
4 Conclusions and Future Work ==

I believe AIX and HP-UX can also work with VS/TUN, I have no Aix or HP-UX servers. Who can help to provide me with such an environment?
References:

http://www.linuxvirtualserver.org http://www.triload.com/en/manual/index.html
References

[1] http://www.linuxvirtualserver.org

[2] http://www.triload.com/en/manual/index.html

File:Topo.png

2006-08-16T08:25:26Z

Maluyao:

LVS/TUN mode with FreeBSD and Solaris realserver

2006-08-16T08:23:44Z

Maluyao:

LVS/TUN mode with FreeBSD and Solaris

Author: Ma Luyao RHCE maluyao at gmail dot com

August 16, 2006

== 1 Introduction ==

The Linux Virtual Server Project (LVS) is a project started by Dr. Wensong Zhang. LVS implements three load balance technology.

a. Network Address Translation (VS/NAT)

b. Direct Routing (VS/DR)

c. IP Tunneling (VS/TUN)

VS/NAT is easy to setup. The load balancer may be a bottleneck of the whole system where the number of servers is more than 20, because both the request packets and response packets need to be rewritten by the load balancer.

VS/DR has the best performance. VS/DR uses MAC-spoofing technology, so it requires that one of the load balancer’s NIC and the real server’s NIC must be in the same IP network segment, and physical segment as well.

VS/TUN is the most scalable. The advantage is since servers connect each other by using IP-tunneling, the load balance and real servers can reside on different LAN , or even WAN.

I plan to setup a set of servers with load balance technique. Realserver run FreeBSD and Solaris system in different places on our business. So it is the only choice for us to use VS/TUN.

We install Fedora Core 5 Linux on our Load Balance server. Kernel-2.6.15 is default.

FreeBSD version is 5.4, we recompile the kernel in order to improve the performance. The default kernel can work too.

Solaris box runs Solaris 10 x86.

2 Normal Linux Solaris IP-tunneling

IP tunneling (IP encapsulation) is a technique to encapsulate IP datagram within IP datagram, which allows datagram destined for one IP address to be wrapped and redirected to another IP address. Many modern OS support IP tunneling, such as Linux, FreeBSD and Solaris.

Let’s do some test to setup normal Linux-Solaris and Linux-FreeBSD IPtunneling without LVS.

2.1 Testing environment

OS IP Address Tunnel IP Address

Fedora Core 5 Linux 10.0.0.1 172.16.0.1

Solaris 10 x86 10.0.0.2 172.16.0.2

FreeBSD 5.4 10.0.0.3 172.16.0.3

2.2 Config Linux box

In order to create an IP-Tunneling between Linux box and Solaris box, run command:

ip tunnel add tun0 mode ipip remote 10.0.0.2 local 10.0.0.1

ifconfig tun0 172.16.0.1 pointopoint 172.16.0.2

ip tunnel add tun1 mode ipip remote 10.0.0.3 local 10.0.0.1

ifconfig tun1 172.16.0.1 pointopoint 172.16.0.3

The ”tun0” is a virtual NIC device name. It can be replaced by other names, such as ”abc0” or ”def0”.

2.3 Config Solaris box

To create an IP-tunneling, we have two methods, First method of creating tunnel is using three commands as below.

/sbin/ifconfig ip.tun0 plumb

/sbin/ifconfig ip.tun0 tsrc 10.0.0.2 tdst 10.0.0.1

/sbin/ifconfig ip.tun0 172.16.0.2 172.16.0.1

It takes effect immediately.

Second method is creating /etc/hostname.ip.tun0 file which contains two lines as below.

tsrc 10.0.0.2 tdst 10.0.0.1

172.16.0.2 172.16.0.1 netmask 255.255.255.255 up

If we use the second method, the tunnel will exist automatically after we reboot system.

2.4 Config FreeBSD box

On FreeBSD box, we also have two methods to create an IP-Tunneling, one way is to run three commands:

ifconfig gif0 create

ifconfig gif0 tunnel 10.0.0.3 10.0.0.1

ifconfig gif0 inet 172.16.0.3 172.168.0.1 netmask 255.255.255.0

If we use the second method, the tunnel will exist automatically after we reboot system.

Add three lines on /etc/rc.conf file as below, the tunnel will exist automatically after we reboot system.

gif interfaces=”gif0”

gifconfig gif0=”10.0.0.3 10.0.0.1”

ifconfig gif0=”172.16.0.3 172.16.0.1 netmask 255.255.255.0”

2.5 Using IP-tunneling

In all the cases on Linux/FreeBSD/Solaris, the netmask value will be default if we ignore to set it.

After creating two tunnels, we can ping 10.0.0.2 and 10.0.0.3 at Linux box successfully, and ping 10.0.0.1 at Solaris or FreeBSD box successfully. All applications layers protocol will work correctly.

==
3 Working example of VS/TUN ==

In VS/TUN mode, because realserver don’t send any datagram to Load balancer, we need not create any tunnel on Linux box and must create proper tunnel on FreeBSD/Solaris box.

3.1 Topological graph

[[Image:http://zh.linuxvirtualserver.org/files/topo.png]]

We can treat all machines on internet and have their real ip. The client can send their requests to VIP of LB Server. LB Server redirects the requests realserver through IP-tunneling. Realserver can send datagram to client.

Because both the gif0 of FreeBSD and ip.tun0 both are NO-ARP device, they are invisible to client.

3.2 Config FreeBSD box

On FreeBSD box, edit /etc/rc.conf as below:

...

ifconfig lnc0=”inet 192.168.0.170 netmask 0xffffff00”

gif interfaces=”gif0”

gifconfig gif0=”192.168.0.170 192.168.10.100”

ifconfig gif0=”192.168.10.200 192.168.10.99 netmask 0xffffffff”

apache enable=”YES”

...

192.168.10.99 looks like a terminal of this tunnel. It can be any idle IP and never use.

3.3 Config Solaris box
On Solaris , edit /etc/hostname.ip.tun0 as below:

tsrc 192.168.0.180 tdst 192.168.10.100

192.168.10.200 192.168.10.99 netmask 255.255.255.255 up

192.168.10.99 looks like a terminal of this tunnel. It can be any IP address and never use it.

3.4 Config Load balancer

Run ipvsadm command on Linux Box: Linux kernel must have ip forward enable.

Edit /etc/sysctl.conf and set to:

net.ipv4.ip forward = 1

While running command:

sysctl -p

It will take effect.

Run the scripts:

ipvsadm -C

ipvsadm -A -t 192.168.10.200:80 -s wlc

ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.170 -i

ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.180 -i

In this script, the port of LB Server and realerver must be same. Good idea is not setting the port of realserver.Now VS/TUN is finished. When a client access 192.168.10.200 at port 80, the datagram will be sent to 192.168.0.170 or 192.168.180 port 80.

==
4 Conclusions and Future Work ==

I believe AIX and HP-UX can also work with VS/TUN, I have no Aix or HP-UX servers. Who can help to provide me with such an environment?
References:

http://www.linuxvirtualserver.org http://www.triload.com/en/manual/index.html
References

[1] http://www.linuxvirtualserver.org

[2] http://www.triload.com/en/manual/index.html

LVS/TUN mode with FreeBSD and Solaris realserver

2006-08-16T08:20:07Z

Maluyao:

LVS/TUN mode with FreeBSD and Solaris
Author: Ma Luyao RHCE maluyao at gmail dot com
August 16, 2006
1 Introduction
The Linux Virtual Server Project (LVS) is a project started by Dr. Wensong Zhang. LVS implements three load balance technology.
1. Network Address Translation (VS/NAT)
2. Direct Routing (VS/DR)
3. IP Tunneling (VS/TUN)

VS/NAT is easy to setup. The load balancer may be a bottleneck of the whole system where the number of servers is more than 20, because both the request packets and response packets need to be rewritten by the load balancer.

VS/DR has the best performance. VS/DR uses MAC-spoofing technology, so it requires that one of the load balancer’s NIC and the real server’s NIC must be in the same IP network segment, and physical segment as well.

VS/TUN is the most scalable. The advantage is since servers connect each other by using IP-tunneling, the load balance and real servers can reside on different LAN , or even WAN.

I plan to setup a set of servers with load balance technique. Realserver run FreeBSD and Solaris system in different places on our business. So it is the only choice for us to use VS/TUN.

We install Fedora Core 5 Linux on our Load Balance server. Kernel-2.6.15 is default.

FreeBSD version is 5.4, we recompile the kernel in order to improve the performance. The default kernel can work too.

Solaris box runs Solaris 10 x86.

2 Normal Linux Solaris IP-tunneling

IP tunneling (IP encapsulation) is a technique to encapsulate IP datagram within IP datagram, which allows datagram destined for one IP address to be wrapped and redirected to another IP address. Many modern OS support IP tunneling, such as Linux, FreeBSD and Solaris.

Let’s do some test to setup normal Linux-Solaris and Linux-FreeBSD IPtunneling without LVS.

2.1 Testing environment
OS IP Address Tunnel IP Address
Fedora Core 5 Linux 10.0.0.1 172.16.0.1
Solaris 10 x86 10.0.0.2 172.16.0.2
FreeBSD 5.4 10.0.0.3 172.16.0.3

2.2 Config Linux box

In order to create an IP-Tunneling between Linux box and Solaris box, run command:

ip tunnel add tun0 mode ipip remote 10.0.0.2 local 10.0.0.1
ifconfig tun0 172.16.0.1 pointopoint 172.16.0.2
ip tunnel add tun1 mode ipip remote 10.0.0.3 local 10.0.0.1
ifconfig tun1 172.16.0.1 pointopoint 172.16.0.3

The ”tun0” is a virtual NIC device name. It can be replaced by other names, such as ”abc0” or ”def0”.

2.3 Config Solaris box

To create an IP-tunneling, we have two methods, First method of creating tunnel is using three commands as below.
/sbin/ifconfig ip.tun0 plumb
/sbin/ifconfig ip.tun0 tsrc 10.0.0.2 tdst 10.0.0.1
/sbin/ifconfig ip.tun0 172.16.0.2 172.16.0.1

It takes effect immediately.

Second method is creating /etc/hostname.ip.tun0 file which contains two lines as below.

tsrc 10.0.0.2 tdst 10.0.0.1
172.16.0.2 172.16.0.1 netmask 255.255.255.255 up

If we use the second method, the tunnel will exist automatically after we reboot system.

2.4 Config FreeBSD box

On FreeBSD box, we also have two methods to create an IP-Tunneling, one way is to run three commands:
ifconfig gif0 create
ifconfig gif0 tunnel 10.0.0.3 10.0.0.1
ifconfig gif0 inet 172.16.0.3 172.168.0.1 netmask 255.255.255.0

If we use the second method, the tunnel will exist automatically after we reboot system.

Add three lines on /etc/rc.conf file as below, the tunnel will exist automatically after we reboot system.

gif interfaces=”gif0”
gifconfig gif0=”10.0.0.3 10.0.0.1”
ifconfig gif0=”172.16.0.3 172.16.0.1 netmask 255.255.255.0”

2.5 Using IP-tunneling

In all the cases on Linux/FreeBSD/Solaris, the netmask value will be default if we ignore to set it.

After creating two tunnels, we can ping 10.0.0.2 and 10.0.0.3 at Linux box successfully, and ping 10.0.0.1 at Solaris or FreeBSD box successfully. All applications layers protocol will work correctly.

3 Working example of VS/TUN

In VS/TUN mode, because realserver don’t send any datagram to Load balancer, we need not create any tunnel on Linux box and must create proper
tunnel on FreeBSD/Solaris box.

3.1 Topological graph
[[Image:http://zh.linuxvirtualserver.org/files/topo.png]]

We can treat all machines on internet and have their real ip. The client can send their requests to VIP of LB Server. LB Server redirects the requests realserver through IP-tunneling. Realserver can send datagram to client.

Because both the gif0 of FreeBSD and ip.tun0 both are NO-ARP device, they are invisible to client.

3.2 Config FreeBSD box

On FreeBSD box, edit /etc/rc.conf as below:
...
ifconfig lnc0=”inet 192.168.0.170 netmask 0xffffff00”
gif interfaces=”gif0”
gifconfig gif0=”192.168.0.170 192.168.10.100”
ifconfig gif0=”192.168.10.200 192.168.10.99 netmask 0xffffffff”
apache enable=”YES”
...

192.168.10.99 looks like a terminal of this tunnel. It can be any idle IP and never use.

3.3 Config Solaris box
On Solaris , edit /etc/hostname.ip.tun0 as below:
tsrc 192.168.0.180 tdst 192.168.10.100
192.168.10.200 192.168.10.99 netmask 255.255.255.255 up
192.168.10.99 looks like a terminal of this tunnel. It can be any IP address
and never use it.

3.4 Config Load balancer

Run ipvsadm command on Linux Box: Linux kernel must have ip forward enable.
Edit /etc/sysctl.conf and set to:
net.ipv4.ip forward = 1
While running command:
sysctl -p
It will take effect.
Run the scripts:
ipvsadm -C
ipvsadm -A -t 192.168.10.200:80 -s wlc
ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.170 -i
ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.180 -i

In this script, the port of LB Server and realerver must be same. Good idea is not setting the port of realserver.Now VS/TUN is finished. When a client access 192.168.10.200 at port 80, the datagram will be sent to 192.168.0.170 or 192.168.180 port 80.

4 Conclusions and Future Work
I believe AIX and HP-UX can also work with VS/TUN, I have no Aix or HP-UX servers. Who can help to provide me with such an environment?
References:

http://www.linuxvirtualserver.org http://www.triload.com/en/manual/index.html
References
[1] http://www.linuxvirtualserver.org
[2] http://www.triload.com/en/manual/index.html

LVS/TUN mode with FreeBSD and Solaris realserver

2006-08-16T08:05:33Z

Maluyao:

LVS/TUN mode with FreeBSD and Solaris

Author: Ma Luyao RHCE maluyao at gmail dot com

Introduction

The Linux Virtual Server Project (LVS) is a project started by Dr. Wensong Zhang. LVS implements three load balance technology.

Network Address Translation (VS/NAT)
Direct Routing (VS/DR)
IP Tunneling (VS/TUN)

VS/NAT is easy to setup. The load balancer may be a bottleneck of the whole system where the number of servers is more than 20, because both the request packets and response packets need to be rewritten by the load balancer.

VS/DR has the best performance. VS/DR uses MAC-spoofing technology, so it requires that one of the load balancer's NIC and the real server's NIC must be in the same IP network segment, and physical segment as well.

VS/TUN is the most scalable. The advantage is since servers connect each other by using IP-tunneling, the load balance and real servers can reside on different LAN , or even WAN.

I plan to setup a set of servers with load balance technique. Realserver run FreeBSD and Solaris system in different places on our business. So it is the only choice for us to use VS/TUN.

According to the official document of LVS, FreeBSD and Solaris system can not be a realserver, but I did do it after some hard testing.

We install Fedora Core 5 Linux on our Load Balance server. Kernel-2.6.15 is default.

FreeBSD version is 5.4, we recompile the kernel in order to improve the performance. The default kernel can work too.

Solaris box runs Solaris 10 x86.

Normal Linux Solaris IP-tunneling

IP tunneling (IP encapsulation) is a technique to encapsulate IP datagram within IP datagram, which allows datagram destined for one IP address to be wrapped and redirected to another IP address. Many modern OS support IP tunneling, such as Linux, FreeBSD and Solaris.

Let's do some test to setup normal Linux-Solaris and Linux-FreeBSD IP-tunneling without LVS.
OS | IP Address | Tunnel IP Address

Fedora Core 5 Linux | 10.0.0.1 | 172.16.0.1
Solaris 10 x86 | 10.0.0.2 | 172.16.0.2
FreeBSD 5.4 | 10.0.0.3 | 172.16.0.3

Config Linux box

In order to create an IP-Tunneling between Linux box and Solaris box, run command:

ip tunnel add tun0 mode ipip remote 10.0.0.2 local 10.0.0.1
ifconfig tun0 172.16.0.1 pointopoint 172.16.0.2
ip tunnel add tun1 mode ipip remote 10.0.0.3 local 10.0.0.1
ifconfig tun1 172.16.0.1 pointopoint 172.16.0.3

The "tun0" is a virtual NIC device name. It can be replaced by other names, such as "abc0" or "def0".

Config Solaris box

To create an IP-tunneling, we have two methods, First method of creating tunnel is using three commands as below.

/sbin/ifconfig ip.tun0 plumb
/sbin/ifconfig ip.tun0 tsrc 10.0.0.2 tdst 10.0.0.1
/sbin/ifconfig ip.tun0 172.16.0.2 172.16.0.1

It takes effect immediately.

Second method is creating /etc/hostname.ip.tun0 file which contains two lines as below.

tsrc 10.0.0.2 tdst 10.0.0.1
172.16.0.2 172.16.0.1 netmask 255.255.255.255 up

If we use the second method, the tunnel will exist automatically after we reboot system.

Config FreeBSD box
On FreeBSD box, we also have two methods to create an IP-Tunneling, one way is to run three commands:

ifconfig gif0 create
ifconfig gif0 tunnel 10.0.0.3 10.0.0.1
ifconfig gif0 inet 172.16.0.3 172.168.0.1 netmask 255.255.255.0

If we use the second method, the tunnel will exist automatically after we reboot system.

Add three lines on /etc/rc.conf file as below, the tunnel will exist automatically after we reboot system.

gif_interfaces="gif0"
gifconfig_gif0="10.0.0.3 10.0.0.1"
ifconfig_gif0="172.16.0.3 172.16.0.1 netmask 255.255.255.0"

Using IP-tunneling

In all the cases on Linux/FreeBSD/Solaris, the netmask value will be default if we ignore to set it.

After creating two tunnels, we can ping 10.0.0.2 and 10.0.0.3 at Linux box successfully, and ping 10.0.0.1 at Solaris or FreeBSD box successfully. All applications layers protocol will work correctly.

Working example of VS/TUN

In VS/TUN mode, because realserver don't send any datagram to Load balancer, we need not create any tunnel on Linux box and must create proper tunnel on FreeBSD/Solaris box.

Topological graph

topo.eps

We can treat all machines on internet and have their real ip. The client can send their requests to VIP of LB Server. LB Server redirects the requests realserver through IP-tunneling. Realserver can send datagram to client.

Because both the gif0 of FreeBSD and ip.tun0 both are NO-ARP device, they are invisible to client.

Config FreeBSD box
On FreeBSD box, edit /etc/rc.conf as below:

...
ifconfig_lnc0="inet 192.168.0.170 netmask 0xffffff00"
gif_interfaces="gif0"
gifconfig_gif0="192.168.0.170 192.168.10.100"
ifconfig_gif0="192.168.10.200 192.168.10.99 netmask 0xffffffff"
apache_enable="YES"
...

192.168.10.99 looks like a terminal of this tunnel. It can be any idle IP and never use.

Config Solaris box

On Solaris , edit /etc/hostname.ip.tun0 as below:

tsrc 192.168.0.180 tdst 192.168.10.100
192.168.10.200 192.168.10.99 netmask 255.255.255.255 up

192.168.10.99 looks like a terminal of this tunnel. It can be any IP address and never use it.

Config Load balancer

Run ipvsadm command on Linux Box:
Linux kernel must have ip_forward enable. Edit /etc/sysctl.conf and set to:

net.ipv4.ip_forward = 1

While running command:

sysctl -p

It will take effect.

Run the scripts:

ipvsadm -C
ipvsadm -A -t 192.168.10.200:80 -s wlc
ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.170 -i
ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.180 -i

In this script, the port of LB Server and realerver must be same. Good idea is not setting the port of realserver.

Now VS/TUN is finished. When a client access 192.168.10.200 at port 80, the
datagram will be sent to 192.168.0.170 or 192.168.180 port 80.

Conclusions and Future Work

I believe AIX and HP-UX can also work with VS/TUN, I have no Aix or HP-UX servers. Who can help to provide me with such an environment?

References:

http://www.linuxvirtualserver.org
http://www.triload.com/en/manual/index.html

Examples

2006-08-16T08:03:59Z

Maluyao: /* Linux/Unix Cluster */

This page is to design examples using LVS, so please feel free to write your LVS systems here and share them with other LVS users.

== Linux/Unix Cluster ==

* [[Building Scalable Web Cluster using LVS]]
* [[Building Web Cache Cluster using LVS]]
* [[Building clusterized proxy farms using LVS]]
* [[Building Scalable Mail Cluster using LVS]]
* [[Building Scalable FTP Cluster using LVS]]
* [[Building Scalable TFTP Cluster using LVS]]
* [[Building MySQL Cluster using LVS]]
* [[Building Scalable DNS Cluster using LVS]]
* [[Building Tomcat Cluster using LVS]]
* [[Building Two-Node Directors/Real Servers using LVS and Keepalived]]
* [[Building an LDAP cluster using LVS and NetWare real servers]]
* [[Building Scalable DHCP Cluster using LVS]]
* [[LVS/TUN mode with FreeBSD and Solaris realserver]]

== Media Service Cluster ==

* [[Building Scalable Media Cluster using LVS]]
* [[Building Windows Media Service Cluster using LVS]]
* [[Building Darwin Streaming Service Cluster using LVS]]
* [[Building Helix Server Cluster using LVS]]

== Terminal Service Cluster ==

* [[Building Linux Terminal Service Cluster using LVS]]
* [[Building Windows Terminal Service Cluster using LVS]]