Creating Remote Boot Clusters

Use OpenMosix with LTSP to remotely boot cluster nodes from a NAS. This can help in getting customized cluster nodes up and running in no time

Friday, May 05, 2006

Print

Comment

Email

Digg

Del.icio.us

Reddit

Twitter

This month we respond to the popular demand from of our readers. We received lots of requests by mail and even personally to do a story on deploying a remote boot cluster using something like OpenMosix. And we have it for you this month-a remote boot cluster that's been created using OpenMosix and LTSP (Linux Terminal Services Project).

At first glance, it may look like a very easy task because installing LTSP with the LTSPadmin is like a breeze. All you have to do is run some pre-configured shell scripts. In that case the setup and configuration are done automatically. The LTSP server is capable of projecting a remote bootable image of any installation over the network.  So if we install an OpenMosix kernel using the LTSP server, boot with that kernel, and then run diskless nodes with it, it should be able to boot with the same OpenMosix kernel. But, this technique doesn't work. Do you know why? Because when you boot a thin client using LTSP, the thin client becomes an exact copy of the server. Everything from its IP address to all settings and parameters for that node are copied over. If you try installing OpenMosix on top of that, it will find all the nodes with the same IP address and will treat them all as one and they won't be able to connect with each other. So, how do we do this? The workaround to this problem is fairly intensive, but that's what we delve into this month.

Pre-requisites
For this kind of setup, the list of prerequisites is pretty long. First you'll need to have the nodes with an integrated PXE (Pre-eXecutable Environment) boot ROM enabled network cards, because LTSP requires a TFTP server, which in turn requires PXE-enabled network cards to work. If you don't have these cards, you will need floppy drives in all the nodes to make them boot up from the TFTP server. Also you will need a regular production class workgroup server with a Gigabit interface. This will be required since all nodes will boot from this server over the network, and it will also have the cluster message passing over it. In fact, we recommend that all nodes have a Gigabit NIC. Plus, you'll also need a NAS box.

You need a NAS box Previously whenever we talked about OpenMosix, all our client nodes had their own hard drives. However, the charm of our present cluster setup is that, the nodes don't need any secondary memory to boot from. Everything is done from the remote boot server's hard drive. Due to this, you won't be able to use the capabilities of MFS (Mosix File System). Just to give you a refresher, MFS is the feature of OpenMosix, which lets the cluster share all hard disks virtually as a single storage. So no matter which node is initiating the task, virtually any node can access it and execute that in its memory space. But since no hard drives are involved in this setup, MFS is of no use. That's why a high-speed NAS is a must.

After you have completed the installation of OpenMosix, you can monitor the cluster from this console

The installation
In order to be able to do this installation, you must have sound knowledge of both Linux and Linux Kernel, as well as Networking. Knowledge of LTSP and OpenMosix is also needed, for which you can refer to the articles in this series from our April and January 2006 issues. Also of course, don't forget to fill up your coffee mug to the brim and those refills coming, because this is going to take quite some time doing this.

Installing LTSP
When installing LTSP, automated shell script called LTSP-admin is generally used to do the installation. This makes the process easier. But in this case this won't work because the LTSPadmin installs the latest version of LTSP automatically, which is what we don't want. In our case, you will require an older version of it because our modified OpenMosix and LTSP-OpenMosix plugin will only work on this version of LTSP. So to install LTSP you have to go to the LTSP's website and from the download section, select LTSP 3.0. (Direct Link:  http://www.ltsp.org/instructions-3.0.html). From here download the core and the 'x core' set of RPMs. Now install them with following commands.

#rpm –ivh ltsp_core-3.0.9-0.i386.rpm  
#rpm –ivh ltsp_kernel-3.0.15-0.i386.rpm  
#rpm –ivh ltsp_x_core-3.0.4-0.i386.rpm  
#rpm –ivh ltsp_x_fonts-3.0.0-0.i386.rpm

Here, while installing, mind the sequence of the packages or your installation might fail. In the link given above you will see many more drivers and modules for LTSP. You can select, download and install the additional packages to fit your needs.

After you are done with installing the RPMs, the next thing is to configure the DHCP server for the LTSP server. Below we have given a sample configuration file, which you can modify to meet your network's settings.

ddns-update-style     none;  
default-lease-time     21600;  
max-lease-time         21600;  
option subnet-mask  255.255.255.0;  
option broadcast-address   192.168.5.255;  
option routers      192.168.5.1;  
option domain-name-servers    192.168.5.1;  
option domain-name    "labsdomain.local";  
option root-path       "192.168.5.32:/opt/ltsp/i386";
shared-network nodes{    subnet 192.168.5.0 netmask 255.255.255.0 }

In the above configuration file, we have assumed that LTSP is running on the 192.168.5.0 network while the gateway for the network is 192.168.5.1. The server on which LTSP is running is 192.168.5.32. The second thing that has to be done is modify your /etc/exports file and add the following line.

/opt/ltsp/i386    *

This will make the /opt/ltsp/i386 folder available on the network an NFS share, and the share will be readable for all. Now restart the NFS and the DHCP servers using the following commands.

#service dhcpd restart  
#service nfsd restat

To make sure that NFS is working, go to any node that has Linux installed on it and make a new folder using:  

#mkdir /tmp/test

Now mount the NFS share like this

mount 192.168.5.32:/opt/ltsp/i386 /tmp/test

This command should run without any errors and after the command succeeds, you should be able access the contents of the /opt/ltsp/i386 folder, which is present on the LTSP server machine on the /tmp/test folder. With this we can assume that your LTSP server is up and running.

Installing OpenMosix  
Now you have to download, build and install the kernel for OpenMosix. For this, first download the source tarball for kernel 2.4.22 (you can download it from kernel.org). We have to stick to this kernel because this patched application doesn't run with any other Linux kernel. After you have downloaded the kernel source tarball, download the same version of OpenMosix's kernel patch, ie, openMosix-2.4.22-2.bz2 (http://prdownloads. sourceforge.net/openMosix/openMosix-2.4.22-1.bz2?download). Do not download the RPM for these because you have to build the kernel with some specific settings. For this first untar and unzip the Linux kernel tarball and then untar and unzip OpenMosix. To do this, use the following commands.

#bunxip linux.2.4.22.tar.bz2
#tar –xvf linux.2.4.22.tar
#tar -jxvf openMosix-2.4.22-1.bz2

Now install the kernel with the following set of commands.

#make mrproper  
#make oldconfig
#make bzImage
#make modules
#make modules_install
#make install

After having built the kernel, restart the LTSP server and boot with the newly created kernel. At boot up, you will see a new entry in grub/lilo, which tells you to boot from Kernel 2.4.22.

After the reboot, go inside the OpenMosix folder and open the configuration file. Here find the following line and change the values accordingly. Comment out the statements that are indicated below.

## openMosix  
#CONFIG_MOSIX=y  
# CONFIG_MOSIX_TOPOLOGY is not set  
CONFIG_MOSIX_SECUREPORTS=y  
CONFIG_MOSIX_DISCLOSURE=1  
CONFIG_MOSIX_FS=y  
CONFIG_MOSIX_DFSA=y  
# CONFIG_MOSIX_PIPE_EXCEPTIONS is not set  
# CONFIG_openMosix_NO_OOM is not set  
# CONFIG_MOSIX_LOADLIMIT is not set  
Now run the following commands to configure and install it.  
#./configure  
#make  
#make install

After you are done with this, restart and you will see another kernel instance, which say 'kernel-2.4.22-openMosix'. Boot from this kernel and your server is ready with OpenMosix. One more thing to be kept in mind is to modify your /etc/inittab to lock processes on the server.

This way all critical processes will be locked to the server. This is needed because with Mosix there is no failure protection. This means that if a process is currently migrated to work station 1 and somebody suddenly powers off that workstation, then the process is dead. When you want a process to be able to migrate, launch it by prefixing it with mosrun –l. You can do it by opening the /etc/inittab file and changing the line /etc/init.d/rc 5 to  /bin/mosrun -l /etc/init.d/rc 5

Installing OM-LTSP patch
This is the last step of our installation. Here you have to install the special packages designed for LTSP and OpenMosix to run in harmony. For this download the following packages: 'http://www.femto-st.fr/~daniau/ltsp-mosix/v1.0/ltsp_openMosix_userland_0.3.4.tar.gz' and 'http://www.femto-st.fr/~daniau/ltsp-mosix/2.4.22-openMosix-2-stuff.html'Here, the first link leads you to a single tar.gz file. All you have to do is untar and unzip it and then do a configure, make and make install to make it work. The second link has a set of kernel patches.

You have to download the relevant one according to your system architecture. For example, you will see files like 'ltsp_openMosix-2_kernel_2.4.22_.tar.gz'.  You have to select the appropriate one and again install it by untaring, unzipping and then running the install.sh shell script.

After all this is done, reboot the server once. Then one by one, boot all your cluster nodes with LAN as the first boot device. The node should boot with OpenMosix kernel without any problems, and should start working as a single cluster. 

You can get more details and FAQs on how to install LTSP-OpenMosix from the official website http://www.femto-st.fr/~ daniau/ ltsp-mosix/.

Anindya Roy

Page(s)   1