Everyone who has experience with hosting of popular web sites will agree with the next statement: “Apache is a memory eater!” Yes, Apache can be optimized but optimization goes more into direction of making it perform faster instead of making it resource friendly. One of the ways to overcome this is to use Apache to proxy requests for virtual hosts which generates high load to another “lightweight” web server, while leaving others on Apache.

My choice of lightweight web server is lighttpd. Developed by Jan Kneschke (he also stands behind MySQL-Proxy), lighttpd (pronounced lighty) was created with exactly this in mind – lightweight, high-performance web server with very small memory footprint. And all that he really is. Unlike Apache’s multi-threaded design, lighttpd has only one thread which has two bonuses

• less consumption of memory – threads are using shared memory, but still every thread needs to reserve some amount of memory for itself
• saving CPU time – creation of new threads can be very CPU costly

Apache provides modules for various server side technologies (PHP, Python, Ruby, etc). Lighttpd does not provide the same feature, but the same functionality is possible through FastCGI. And this, again, improves it’s performance and memory footprint.

In this small how-to I will describe how lighttpd and Apache are configured to work side by side.

Coming from old “Slackware school” of sys admins, I still sometimes prefer to compile software from source insted of using already built packages. Maybe it is a bad habit but it is hard to quit. :). So, we first need to install everything we need, and that is Apache with proxy support, PHP with FastCGI support and lighttpd with Lua support (needed for various rewrite support).

Building software from source is fairly simple and straight-froward process if you have all the libraries needed. We will start with Apache. Except all standard compile options (–enable-so, –enable-mods-shared=all, –enable-ssl=shared, etc) we will also use options which will include Apache proxy modules. After make install Apache is installed in /usr/local/httpd-2.0.59 folder.

# ./configure < standard apache compile options > \
> –prefix=/usr/local/httpd-2.0.59 \
> –enable-proxy \
> –enable-proxy-connect \
> –enable-proxy-http
# make
# make install

Next PHP. Story is the same as for Apache – besides standard options we are also building support for FastCGI. PHP is installed in /usr/local/php-5.2.3.

# ./configure < standard php compile options > \
> –prefix=/usr/local/php-5.2.3 \
> –enable-fastcgi \
> –enable-discard-path \
> –enable-force-redirect
# make
# make install

And on the end the star of this how-to, lighttpd. Lua support is necessary for mod_magnet which is used for certain rewrite capabilities (I will cover that in some other how-to). PKG_CONFIG_PATH needs to be adjusted if lua.pc is not already in the right location. It can be found in etc folder in Lua source tree. Lighty is installed in /usr/local/lighttpd-1.4.16.

# PKG_CONFIG_PATH=$PKG_CONFIG_PATH:$LUA_SOURCE_DIR/etc ./configure \
> –prefix=/usr/local/lighttpd-1.4.16 \
> –with-lua \
> –with-memcache
# make
# make install

By now you are probably wondering why I use such insane paths for software installation. This method actually provides easy way to maintain and upgrade software. Once everything is installed you just go to /usr/local and create sym links from these folders to more generic ones and when you perform some upgrade all you need to change is the symlink and you are ready to use the new version. In case you want to revert back to the old version procedure is the same – change the destination of the sym link and you are back. Very simple.

# cd /usr/local/
# ln -sf httpd-2.0.59 httpd
# ln -sf php-5.2.3 php5
# ln -sf lighttpd-1.4.16 lighttpd

Now, let’s configure lighttpd.

Lighttpd comes with an example config file which you can customize to create configuration that will suit your needs. As a matter of fact there are only few things you should change to make lighttpd works. Here is my starting configuration

server.modules = ( “mod_rewrite”,
                   ”mod_redirect”,
                   ”mod_access”,
                   ”mod_fastcgi”,
                   ”mod_evhost”,
                   ”mod_compress”,
                   ”mod_accesslog”,
                   ”mod_magnet” )
 
server.document-root = “/var/www/html”
 
server.errorlog = “/var/log/lighttpd/error.log”
accesslog.filename = “/var/log/lighttpd/access.log”
server.pid-file = “/var/run/lighttpd.pid”
 
url.access-deny = ( “~”, “.inc” )
 
index-file.names = ( “index.php”,
                     ”index.html”,
                     ”index.htm”,
                     ”default.htm” )
 
static-file.exclude-extensions = ( “.php”,
                                   ”.pl”,
                                   ”.fcgi” )
 
$HTTP["url"] =~ “\.pdf$” {
    server.range-requests = “disable”
}
 
server.username = “nobody”
server.groupname = “nogroup”
 
dir-listing.activate = “enable”

As you can see, I included some additional modules – lighttpd has only mod_access and mod_accesslog activated by default. I also changed paths to main document root, access and error log files and pid file.

Now let’s do some real changes. By default lighty listens on port 80, but that port is already taken by Apache so we will move lighttpd to port 8080 and make it listens only on a loopback interface, no need for outside world connections.

server.bind = “127.0.0.1″
server.port = 8080

Path to virtual hosts doc root is in /home/<domain>/public_html so we need to tell lighty where to find them

evhost.path-pattern = “/home/%0/public_html/”

Now when lighttpd receives connection it will look at “Host” part of the HTTP header and check if /home/<host>/public_html exists and serve the proper page from this folder. If the folder does not exists it will try to serve a page from a default document root (we defined this above with server.document-root option).

And now the final part – FastCGI for PHP support.

Lighttpd can use remote FastCGI servers to easen the server load even more, but in this case we don’t need this high-end feature as we will be running FastCGI on the same server. With next options we instruct lighttpd to execute all files with .php extension through the FastCGI server on localhost and we also define the PHP binary what will be used for parsing these files.

fastcgi.server = ( “.php” =>
    ( “localhost” =>
      (
        ”socket” => “/tmp/php-fastcgi.socket”,
        ”bin-path” => “/usr/local/php5/bin/php-cgi”
      )
    )
)

We can now start lighty.

/etc/init.d/rc.lighttpd start

And that was all for lighttpd, we can now move to Apache configuration.

Configuring Apache is even simplier. We first need to load proxy support.

LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so
LoadModule proxy_http_module modules/mod_proxy_http.so

And than in the definition of the virtual host we want to transfer to lighttpd we add these proxy options and simple rewrite rule which will actually do all the proxying necessary.

<VirtualHost *:80>
    ServerAdmin web@miljan.no-spam-please.org
    ServerName miljan.org
    ServerAlias www.miljan.org
    ProxyRequests Off
    ProxyPreserveHost On
    RewriteEngine On
    RewriteRule ^/(.*) http://127.0.0.1:8080/$1 [P,L]
</VirtualHost>

And that’s it. You web site is now running under lighttpd. And this is very easy to check too.

# telnet miljan.org 80
Trying 67.19.93.28…
Connected to miljan.org.
Escape character is ‘^]’.
GET / HTTP/1.0
 
HTTP/1.1 301 Moved Permanently
Date: Fri, 24 Aug 2007 17:49:12 GMT
Server: Apache
Location: https://www.miljan.org/main/
Content-Length: 235
Content-Type: text/html; charset=iso-8859-1
Via: 1.1 Application and Content Networking System Software 5.3.5
Connection: Close

# telnet miljan.org 80
Trying 67.19.93.28…
Connected to miljan.org.
Escape character is ‘^]’.
GET / HTTP/1.0
Host: www.miljan.org
 
HTTP/1.1 302 Found
Date: Fri, 24 Aug 2007 17:51:44 GMT
Server: lighttpd/1.4.16
X-Powered-By: PHP/5.2.3
Location: https://www.miljan.org/main/
Content-type: text/html
Content-Length: 0
Via: 1.1 Application and Content Networking System Software 5.3.5
Connection: Close

Note:Only shortcuming of this method is that your PHP scripts will not see the correct IP address of your visitors but rather the IP address of the Apache server. To overcome this you can use $_SERVER['HTTP_X_FORWARDED_FOR'] instead of $_SERVER['REMOTE_ADDR'] and gethostbyaddr($_SERVER['HTTP_X_FORWARDED_FOR']) instead of $_SERVER['REMOTE_HOST']. Also, lighttpd logs will be almost useless for any traffic analysis, you will need to use Apache logs for that.