LVS/TUN mode with FreeBSD and Solaris realserver - Revision history

ChielEltle: lidomtatroc

2008-12-25T02:39:51Z

lidomtatroc

Rhinofly: /* Config FreeBSD box */

2008-03-18T08:47:22Z

‎Config FreeBSD box

Wensong: /* Introduction */

2006-11-11T15:20:03Z

‎Introduction

Wensong: /* Introduction */

2006-11-11T15:19:36Z

‎Introduction

Wensong at 15:28, 13 October 2006

2006-10-13T15:28:38Z

Maluyao: /* Config FreeBSD box */

2006-08-29T03:39:16Z

‎Config FreeBSD box

Maluyao: /* Config FreeBSD box */

2006-08-29T03:37:28Z

‎Config FreeBSD box

Wensong: categorize

2006-08-18T17:18:47Z

categorize

Wensong at 05:51, 18 August 2006

2006-08-18T05:51:10Z

Wensong: formatting

2006-08-16T16:03:51Z

formatting

@@ Line 1: / Line 1: @@
 Author: Ma Luyao RHCE   maluyao at gmail dot com
@@ Line 15: / Line 16: @@
 [[LVS/NAT|VS/NAT]] is easy to setup. The load balancer may be a bottleneck of the whole system where the number of servers is more than 20, because both the request packets and response packets need to be rewritten by the [[load balancer]].
-[[LVS/DR|VS/DR]] has the best performance. VS/DR uses MAC-spoofing technology, so it requires that one of the load balancer’s NIC and the [[real server]]’s NIC must be in the same IP network segment, and physical segment as well.
+[[LVS/DR|VS/DR]] has the best performance. VS/DR uses MAC-spoofing technology, so it requires that one of the load balancerâs NIC and the [[real server]]âs NIC must be in the same IP network segment, and physical segment as well.
 [[LVS/TUN|VS/TUN]] is the most scalable. The advantage is that since servers connect each other by using IP-tunneling, the load balancer and the real servers can reside on different LAN , or even WAN.
@@ Line 31: / Line 32: @@
 IP tunneling (IP encapsulation) is a technique to encapsulate IP datagram within IP datagram, which allows datagram destined for one IP address to be wrapped and redirected to another IP address. Many modern OS support IP tunneling, such as Linux, FreeBSD and Solaris.
-Let’s do some test to setup normal Linux-Solaris and Linux-FreeBSD IPtunneling without LVS.
+Letâs do some test to setup normal Linux-Solaris and Linux-FreeBSD IPtunneling without LVS.
 === Testing environment ===
@@ Line 58: / Line 59: @@
   ifconfig tun1 172.16.0.1 pointopoint 172.16.0.3
-The ”tun0” is a virtual NIC device name. It can be replaced by other names, such as ”abc0” or ”def0”.
+The âtun0â is a virtual NIC device name. It can be replaced by other names, such as âabc0â or âdef0â.
 === Config Solaris box ===
@@ Line 94: / Line 95: @@
 Add three lines on /etc/rc.conf file as below, the tunnel will exist automatically after we reboot system.
-  gif_interfaces=”gif0”
+  gif_interfaces=âgif0â
-  gifconfig_gif0=”10.0.0.3 10.0.0.1”
+  gifconfig_gif0=â10.0.0.3 10.0.0.1â
-  ifconfig_gif0=”172.16.0.3 172.16.0.1 netmask 255.255.255.0”
+  ifconfig_gif0=â172.16.0.3 172.16.0.1 netmask 255.255.255.0â
 ===  Using IP-tunneling ===
@@ Line 108: / Line 109: @@
 == Working example of VS/TUN ==
-In VS/TUN mode, because realserver don’t send any datagram to Load balancer, we need not create any tunnel on Linux box and must create proper tunnel on FreeBSD/Solaris box.
+In VS/TUN mode, because realserver donât send any datagram to Load balancer, we need not create any tunnel on Linux box and must create proper tunnel on FreeBSD/Solaris box.
 === Topological graph ===
@@ Line 124: / Line 125: @@
   ...
-  ifconfig_lnc0=”inet 192.168.0.170 netmask 0xffffff00”
+  ifconfig_lnc0=âinet 192.168.0.170 netmask 0xffffff00â
-  gif_interfaces=”gif0”
+  gif_interfaces=âgif0â
-  gifconfig_gif0=”192.168.0.170 192.168.10.100”
+  gifconfig_gif0=â192.168.0.170 192.168.10.100â
-  ifconfig_gif0=”192.168.10.200 192.168.10.99 netmask 0xffffffff”
+  ifconfig_gif0=â192.168.10.200 192.168.10.99 netmask 0xffffffffâ
-  apache_enable=”YES”
+  apache_enable=âYESâ
   ...

@@ Line 88: / Line 88: @@
   ifconfig gif0 tunnel 10.0.0.3 10.0.0.1
-  ifconfig gif0 inet 172.16.0.3 172.168.0.1 netmask 255.255.255.0
+  ifconfig gif0 inet 172.16.0.3 172.16.0.1 netmask 255.255.255.0
 If we use the second method, the tunnel will exist automatically after we reboot system.

@@ Line 17: / Line 17: @@
 [[LVS/DR|VS/DR]] has the best performance. VS/DR uses MAC-spoofing technology, so it requires that one of the load balancer’s NIC and the [[real server]]’s NIC must be in the same IP network segment, and physical segment as well.
-[LVS/TUN|VS/TUN]] is the most scalable. The advantage is that since servers connect each other by using IP-tunneling, the load balancer and the real servers can reside on different LAN , or even WAN.
+[[LVS/TUN|VS/TUN]] is the most scalable. The advantage is that since servers connect each other by using IP-tunneling, the load balancer and the real servers can reside on different LAN , or even WAN.
 I plan to setup a set of servers with [[load balancing]] technique. [[Real server]]s run FreeBSD and Solaris system in different places on our business. So it is the only choice for us to use VS/TUN.

@@ Line 13: / Line 13: @@
 c. IP Tunneling ([[LVS/TUN|VS/TUN]])
-VS/NAT is easy to setup. The load balancer may be a bottleneck of the whole system where the number of servers is more than 20, because both the request packets and response packets need to be rewritten by the [[load balancer]].
+[[LVS/NAT|VS/NAT]] is easy to setup. The load balancer may be a bottleneck of the whole system where the number of servers is more than 20, because both the request packets and response packets need to be rewritten by the [[load balancer]].
-VS/DR has the best performance. VS/DR uses MAC-spoofing technology, so it requires that one of the load balancer’s NIC and the [[real server]]’s NIC must be in the same IP network segment, and physical segment as well.
+[[LVS/DR|VS/DR]] has the best performance. VS/DR uses MAC-spoofing technology, so it requires that one of the load balancer’s NIC and the [[real server]]’s NIC must be in the same IP network segment, and physical segment as well.
-VS/TUN is the most scalable. The advantage is that since servers connect each other by using IP-tunneling, the load balancer and the real servers can reside on different LAN , or even WAN.
+[LVS/TUN|VS/TUN]] is the most scalable. The advantage is that since servers connect each other by using IP-tunneling, the load balancer and the real servers can reside on different LAN , or even WAN.
-I plan to setup a set of servers with load balancing technique. [[Real server]]s run FreeBSD and Solaris system in different places on our business. So it is the only choice for us to use VS/TUN.
+I plan to setup a set of servers with [[load balancing]] technique. [[Real server]]s run FreeBSD and Solaris system in different places on our business. So it is the only choice for us to use VS/TUN.
 We install Fedora Core 5 Linux on our Load Balance server. Kernel-2.6.15 is default.

@@ Line 5: / Line 5: @@
 == Introduction ==
-The Linux Virtual Server Project (LVS) is a project started by Dr. Wensong Zhang. LVS implements three load balance technology.
+The Linux Virtual Server Project ([[LVS]]) is a project started by Dr. Wensong Zhang. LVS implements three load balance technology.
-a. Network Address Translation (VS/NAT)
+a. Network Address Translation ([[LVS/NAT|VS/NAT]])
-b. Direct Routing (VS/DR)
+b. Direct Routing ([[LVS/DR|VS/DR]])
-c. IP Tunneling (VS/TUN)
+c. IP Tunneling ([[LVS/TUN|VS/TUN]])
-VS/NAT is easy to setup. The load balancer may be a bottleneck of the whole system where the number of servers is more than 20, because both the request packets and response packets need to be rewritten by the load balancer.
+VS/NAT is easy to setup. The load balancer may be a bottleneck of the whole system where the number of servers is more than 20, because both the request packets and response packets need to be rewritten by the [[load balancer]].
-VS/DR has the best performance. VS/DR uses MAC-spoofing technology, so it requires that one of the load balancer’s NIC and the real server’s NIC must be in the same IP network segment, and physical segment as well.
+VS/DR has the best performance. VS/DR uses MAC-spoofing technology, so it requires that one of the load balancer’s NIC and the [[real server]]’s NIC must be in the same IP network segment, and physical segment as well.
-VS/TUN is the most scalable. The advantage is since servers connect each other by using IP-tunneling, the load balance and real servers can reside on different LAN , or even WAN.
+VS/TUN is the most scalable. The advantage is that since servers connect each other by using IP-tunneling, the load balancer and the real servers can reside on different LAN , or even WAN.
-I plan to setup a set of servers with load balance technique. Realserver run FreeBSD and Solaris system in different places on our business. So it is the only choice for us to use VS/TUN.
+I plan to setup a set of servers with load balancing technique. [[Real server]]s run FreeBSD and Solaris system in different places on our business. So it is the only choice for us to use VS/TUN.
 We install Fedora Core 5 Linux on our Load Balance server. Kernel-2.6.15 is default.
@@ Line 114: / Line 114: @@
 [[Image:Topo.png|center|600px]]
-We can treat all machines on internet and have their real ip. The client can send their requests to VIP of LB Server. LB Server redirects the requests realserver through IP-tunneling. Realserver can send datagram to client.
+We can treat all machines on internet and have their real ip. The client can send their requests to [[VIP]] of LB Server. LB Server redirects the requests realserver through IP-tunneling. Realserver can send datagram to client.
 Because both the gif0 of FreeBSD and ip.tun0 both are NO-ARP device, they are invisible to client.

@@ Line 124: / Line 124: @@
   ...
-  ifconfig lnc0=”inet 192.168.0.170 netmask 0xffffff00”
+  ifconfig_lnc0=”inet 192.168.0.170 netmask 0xffffff00”
-  gif interfaces=”gif0”
+  gif_interfaces=”gif0”
-  gifconfig gif0=”192.168.0.170 192.168.10.100”
+  gifconfig_gif0=”192.168.0.170 192.168.10.100”
-  ifconfig gif0=”192.168.10.200 192.168.10.99 netmask 0xffffffff”
+  ifconfig_gif0=”192.168.10.200 192.168.10.99 netmask 0xffffffff”
-  apache enable=”YES”
+  apache_enable=”YES”
   ...

← Older revision		Revision as of 17:18, 18 August 2006
Line 187:		Line 187:

	[4] http://docs.sun.com/app/docs/doc/817-0573/6mgc65bcq?a=view		[4] http://docs.sun.com/app/docs/doc/817-0573/6mgc65bcq?a=view
		+
		+	[[Category:LVS Examples\|Tunnel]]

@@ Line 144: / Line 144: @@
 .168.10.200 192.168.10.99 netmask 255.255.255.255 up
-−
 .168.10.99 looks like a terminal of this tunnel. It can be any IP address and never use it.
@@ Line 153: / Line 153: @@
 Edit /etc/sysctl.conf and set to:
-  net.ipv4.ip forward = 1
+  net.ipv4.ip_forward = 1
 While running command:

@@ Line 26: / Line 26: @@
 Solaris box runs Solaris 10 x86.
 == Normal Linux Solaris/FreeBSD IP-tunneling ==
 IP tunneling (IP encapsulation) is a technique to encapsulate IP datagram within IP datagram, which allows datagram destined for one IP address to be wrapped and redirected to another IP address. Many modern OS support IP tunneling, such as Linux, FreeBSD and Solaris.
@@ Line 76: / Line 74: @@
 Second method is creating /etc/hostname.ip.tun0 file which contains two lines as below.
-tsrc 10.0.0.2 tdst 10.0.0.1
+ tsrc 10.0.0.2 tdst 10.0.0.1
-−
 .16.0.2 172.16.0.1 netmask 255.255.255.255 up
 If we use the second method, the tunnel will exist automatically after we reboot system.
@@ Line 86: / Line 84: @@
 On FreeBSD box, we also have two methods to create an IP-Tunneling, one way is to run three commands:
-ifconfig gif0 create
+ ifconfig gif0 create
-−
-ifconfig gif0 tunnel 10.0.0.3 10.0.0.1
+ ifconfig gif0 tunnel 10.0.0.3 10.0.0.1
-−
-ifconfig gif0 inet 172.16.0.3 172.168.0.1 netmask 255.255.255.0
+ ifconfig gif0 inet 172.16.0.3 172.168.0.1 netmask 255.255.255.0
 If we use the second method, the tunnel will exist automatically after we reboot system.
@@ Line 96: / Line 94: @@
 Add three lines on /etc/rc.conf file as below, the tunnel will exist automatically after we reboot system.
-gif interfaces=”gif0”
+ gif interfaces=”gif0”
-−
-gifconfig gif0=”10.0.0.3 10.0.0.1”
+ gifconfig gif0=”10.0.0.3 10.0.0.1”
-−
-ifconfig gif0=”172.16.0.3 172.16.0.1 netmask 255.255.255.0”
+ ifconfig gif0=”172.16.0.3 172.16.0.1 netmask 255.255.255.0”
 ===  Using IP-tunneling ===
@@ Line 109: / Line 107: @@
 == Working example of VS/TUN ==
 In VS/TUN mode, because realserver don’t send any datagram to Load balancer, we need not create any tunnel on Linux box and must create proper tunnel on FreeBSD/Solaris box.
@@ Line 115: / Line 112: @@
 === Topological graph ===
-[[Image:Topo.png]]
+[[Image:Topo.png|center|600px]]
 We can treat all machines on internet and have their real ip. The client can send their requests to VIP of LB Server. LB Server redirects the requests realserver through IP-tunneling. Realserver can send datagram to client.
@@ Line 125: / Line 122: @@
 On FreeBSD box, edit /etc/rc.conf as below:
-...
+ ...
-−
-ifconfig lnc0=”inet 192.168.0.170 netmask 0xffffff00”
+ ifconfig lnc0=”inet 192.168.0.170 netmask 0xffffff00”
-−
-gif interfaces=”gif0”
+ gif interfaces=”gif0”
-−
-gifconfig gif0=”192.168.0.170 192.168.10.100”
+ gifconfig gif0=”192.168.0.170 192.168.10.100”
-−
-ifconfig gif0=”192.168.10.200 192.168.10.99 netmask 0xffffffff”
+ ifconfig gif0=”192.168.10.200 192.168.10.99 netmask 0xffffffff”
-−
-apache enable=”YES”
+ apache enable=”YES”
-−
-...
+ ...
 .168.10.99 looks like a terminal of this tunnel. It can be any idle IP and never use.
@@ Line 144: / Line 141: @@
 On Solaris , edit /etc/hostname.ip.tun0 as below:
-tsrc 192.168.0.180 tdst 192.168.10.100
+ tsrc 192.168.0.180 tdst 192.168.10.100
-−
 .168.10.200 192.168.10.99 netmask 255.255.255.255 up
-−
 .168.10.99 looks like a terminal of this tunnel. It can be any IP address and never use it.
@@ Line 156: / Line 153: @@
 Edit /etc/sysctl.conf and set to:
-net.ipv4.ip forward = 1
+ net.ipv4.ip forward = 1
 While running command:
-sysctl -p
+ sysctl -p
 It will take effect.
@@ Line 166: / Line 163: @@
 Run the scripts:
-ipvsadm -C
+ ipvsadm -C
-−
-ipvsadm -A -t 192.168.10.200:80 -s wlc
+ ipvsadm -A -t 192.168.10.200:80 -s wlc
-−
-ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.170 -i
+ ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.170 -i
-−
-ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.180 -i
+ ipvsadm -a -t 192.168.10.200:80 -r 192.168.0.180 -i
 In this script, the port of LB Server and realerver must be same. Good idea is not setting the port of realserver.Now VS/TUN is finished. When a client access 192.168.10.200 at port 80, the datagram will be sent to 192.168.0.170 or 192.168.180 port 80.