SorryServer

From LVSKB
Revision as of 13:15, 14 January 2008 by Galbum (Talk | contribs) (Setting up a Sorry-Server)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Setting up a Sorry-Server

Sometime all of your real-servers might be down or overloaded. This can be due to a scheduled maintainace interval, multiple hardware failure, badly written dynamic web-content, broken databases or script kiddies bringing your site down. In such a situation it is much more polite to tell your customers that you are sorry but your site currently is down, rather than just being unavailable.

To set up such a webserver you normally would serve only some static content telling your customers that you are sorry, but your web-services currently are not available. However the requests your customers send, are regular URLs which are not resolvable by your Sorry-Server, and therefore would be answered with an Error-Code: 404 - Not Found. This Error-Code can be very annoying, if at the moment of the outage a robot (such as Googlebot) is indexing your webpages. In such a situation most of your webpages will fall out of the Google-index and your web-site might have to wait for up to a week to return back to the Google-index, depending on the number visits of Googlebot per month.

Using TUX as Sorry-Server

TUX is a tiny webserver which runs inside the Linux kernel and is therefore predestined as a Sorry-Server. If your Loadbalancer is able to pass your web-traffic to a number of real-servers, TUX running on the Loadbalancer is able to handle all that traffic by itself.

The only precaution using TUX is not to send a Status 200 - Ok neither an Error-Code 404 - Not Found as as Sorry-Page. The best fitting Error-Code to send in such a situation is 502 - Server Overloaded.

However this is not the default behaviour, nor can TUX be configured to do so. You must apply a tiny patch onto the TUX-kernel-code, recompile that kernel-module and reinstall it onto your Loadbalancer.

Patch for Linux-kernel-2.4:

diff -u kernel-2.4.21/linux-2.4.21/net/tux/proto_http.c linux-2.4.21-tux502/net/tux/proto_http.c
--- linux-2.4.21/net/tux/proto_http.c     2005-03-07 15:09:45.000000000 +0100
+++ linux-2.4.21-tux502/net/tux/proto_http.c    2005-03-07 15:08:13.000000000 +0100
@@ -1172,7 +1172,7 @@
        "<HTML> Forbidden </HTML>";

 static const char not_found[] =
-       "HTTP/1.1 404 Not Found\r\n"
+       "HTTP/1.1 502 Service temporarily down\r\n"
        "Connection: Keep-Alive\r\n" \
        "Content-Length: 29\r\n\r\n"
        "<HTML> Page Not Found </HTML>";
@@ -1838,7 +1838,7 @@
                "HTTP/1.1 206 Partial Content"

 #define HEADER_PART1C \
-               "HTTP/1.1 404 Page Not Found\r\n" \
+               "HTTP/1.1 502 Service temporarily down\r\n" \
                "Content-Type: "

 #define HEADER_PART1D \

Patch for Linux-kernel-2.6:

diff -ur linux-2.6.9/net/tux/proto_http.c linux-2.6.9-tux502/net/tux/proto_http.c
--- linux-2.6.9/net/tux/proto_http.c    2005-03-15 12:08:56.660827776 +0100
+++ linux-2.6.9-tux502/net/tux/proto_http.c     2005-03-15 13:13:49.437035840 +0100
@@ -1174,7 +1174,7 @@
        "<HTML> Forbidden </HTML>";

 static const char not_found[] =
-       "HTTP/1.1 404 Not Found\r\n"
+       "HTTP/1.1 502 Service temporarily down\r\n"
        "Connection: Keep-Alive\r\n" \
        "Content-Length: 29\r\n\r\n"
        "<HTML> Page Not Found </HTML>";
@@ -1840,7 +1840,7 @@
                "HTTP/1.1 206 Partial Content"

 #define HEADER_PART1C \
-               "HTTP/1.1 404 Page Not Found\r\n" \
+               "HTTP/1.1 502 Service temporarily down\r\n" \
                "Content-Type: "

 #define HEADER_PART1D \

Running the Sorry-Server

First start the TUX kernel-thread, by invoking

/etc/init.d/tux start

The Sorry-Server can run all the time, even if you don't need it.

In case all of your real-servers are down, you must reroute the traffic onto the Sorry-Server. First flush the Real-Server Tables of your IPVS invoking

ipvsadm -d -t service-address -r server-address

for each of your real-servers. Next add a route the web-traffic onto the Sorry-Server, this normally is done by invoking

ipvsadm -a -t service-address -r 127.0.0.1:80

once. All web-traffic now is redirected onto the Sorry-Server.

Depending on where the TUX-configuration says to have its DOCUMENTROOT, go to that directory and edit the file 404.html adding some text such as Momentary system interruption. You may also add stylesheet- and image-files to the Sorry-Server DOCUMENTROOT, you should however delete any index.html file since this would be served by default and with a HTTP-Status 200 - OK. This is certainly not your intention.

Using keepalived

The keepalived is a program to check real-servers and manage the IPVS-Tables accordingly. Keepalived can be configured to automatically reroute all the web-traffic onto a Sorry-Server in case all real-servers are down. Simply add this line to each virtual_server section in the keepalived's configuration file:

virtual_server service_address {
        ...snip...
        sorry_server 127.0.0.1 80
        ...snip...
}

CREDITS

TUX is a Linux kernel-project maintained by RedHat.

This patch is maintained by Jacob Rief.