Archive for September, 2007


Building Scalable Web Architectures

I just read a very interesting presentation done by Aaron Bannert for ApacheCon 2005. Presentation is on “Building Scalable Web Architectures” and it is a very good reading for anyone interested in high scale web environments. Here is the link to the presentation.


Listen to…

Red Union – The Partisan


Easy way to read MBR?

10$ question. Sometime ago you have created backup of your systems Master Boot Record (MBR). Now, after some change, you noticed you did a fatal mistake and your partition table is corrupted and you need to recover it from the backup you created, but you are not sure if it is the correct version. The question is, what is the easiest way to read partition table from the backup of your MBR? No, Hex editor is not the easiest way to do it (and it is bad for your eyes :)). I wonder how many of you said ‘file’ command? Yes, magical file command is able to read the data from the mbr dump and prints you the actual partition table. Here is an example from my laptop.

# file mbr.bin
mbr.bin: x86 boot sector;
partition 1: ID=0×83, active, starthead 1, startsector 63, 40949622 sectors;
partition 2: ID=0×82, starthead 254, startsector 40949685, 2088450 sectors;
partition 3: ID=0x8e, starthead 254, startsector 43038135, 74172105 sectors, code offset 0×48

As you can see I have only 3 partitions on the disk. First one has type 0×83, which is HEX id for ext3 type of partition and it is my / partition (you don’t see it here, but I know it :)). It is also active partition, it means that it is used for booting the system. You can also see the size of the partition in sectors. Knowing that one sector has length of 512 bytes you can easily find out the size of the partition.

# echo $(((40949622/2)/1024))
# df -k /
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 19833488 6504676 12305072 35% /

That’s it, correct. :)

Next partition is 0×82 which is swap partition. And last partition is 0x8e which is id for Linux LVM partition.

While I am here, I could also explain what is MBR and how it is used.

MBRMain Boot Record resides in first 512 bytes of your bootable disk. Besides partition table it also holds bootloader and something called a magic number. As you can see on the picture, bootloader takes the biggest part of MBR, whole 446 bytes. During the boot process BIOS search for a bootable devices attached to your system and once it finds it it looks at the MBR and loads the bootloader, also called primary bootloader. Primary bootloader looks at the partition table inside MBR (next 64 bytes after the bootloader) and searches for an active partition. When it finds active partition it loads the secondary boot loader from that partitions boot record which, in turn, loads the kernel, and so on.

Magic number is used for sanity check of your MBR. It holds only 2 bytes and should be 0xAA55.

So, in short words, MBR is used to easily locate and load kernel from the correct device. (It is also used by your operating system to find the layout of the disk, but that is another story.)

PS: You can create a dump of your MBR by issuing next command:

# dd if=/dev/sda of=mbr.bin bs=512 count=1

Replace /dev/sda with the correct address to your disk.

PS 2: Sorry for bad quality of the MBR scheme, but I didn’t have much time to work on it and I am not a graphic designer. :D


I am on Planet SysAdmin :)

It may be old news, but I have just found out (late as always :P). This blog was added to Planet SysAdmin list of blogs. Woohoo! ;) Seems that someone is reading this after all. :)


Playing with lighttpd

Apache mod_rewrite is a very powerful tool used in hundreds of web applications for various purposes, most notably for SEO friendly (or user friendly :)) URLs. Lighttpd also has mod_rewrite but some of the most important functionalities can be achieved using mod_redirect and mod_magnet.

mod_redirect does exactly what its name says – it will redirect visitor from matched URL to a desired URL. One of the cases when this is very useful is redirecting visitors who were too lazy to type ‘www’ part of your web site address to a proper address. In Apache you would use mod_rewrite with a similar syntax.

RewriteCond %{HTTP_HOST} ^miljan\.org$ [NC]
RewriteRule ^(.*)$$1 [R=301,L]

In lighttpd you can use mod_redirect with the following syntax:

$HTTP["host"] =~ “^miljan\.org” {
    url.redirect = (
        ”^/(.*)$” => “$1″

As you can see regular expressions in Apache and lighttpd are the same. Only difference is in configuration syntax.

mod_redirect can be used also for simple redirections:

url.redirect = (
    ”^/main/gallery” => “”,
    ”^/main/travelblog” => “”,
    ”^/main/tips-n-tricks” => “”

In Apache, this would look like:

Redirect permanent /main/gallery
Redirect permanent /main/travelblog
Redirect permanent /main/tips-n-tricks

Note: you need mod_alias in order to use Redirect directive.

Now, SEO URLs. Standard way of doing this with Apache and mod_rewrite is:

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule (.*) /index.php [L]

Basically, it will check if the filename/folder requested exists on the filesystem and if not all requests will be transfered to a desired file (index.php in this case). It works pretty much as a 404 error page.

Lighttpd does this also in a similar way. It is only necessary to set a 404 error page and it will work. But for some reason this doesn’t work for me. It might be due to a bug in server.error-handler-404 (lighttpd 1.4.17 was released 5 days ago with this bug fixed). But there is a workaround for this. Lighttpd has mod_magnet which uses capabilities of LUA script language – so we can use a small LUA script to check if the file/folders exists and if not to transfer all requests to index.php file.

First we setup lighttpd:

$HTTP["url"] =~ “^.*(html|php|/).*$” {
    magnet.attract-physical-path-to = (“/etc/lighttpd/magnet-rewrite.lua”)

Than we create /etc/lighttpd/magnet-rewrite.lua script:

if (not lighty.stat(lighty.env["physical.path"])) then
    lighty.env["physical.path"] = lighty.env["physical.doc-root"] .. “/index.php”

And that is all. Not as simple as Apache, and not even close as simple as setting up a 404 page in lighttpd, but it works!