Network Booting of Diskless Systems ----------------------------------- This is a discussion from Bruce on setting up diskless RedHat 7.2 systems which boot over the network from a server. The troubles I've had arose from two sources: 1) In booting Linux using an initial ramdisk (initrd), RH now uses the linuxrc script - which is parsed by a special shell nash. nash isn't able to mount NFS volumes before performing a "pivot_root" operation to change root midstream. This is documented in the man page for nash. Both linuxrc and nash appear to be new to 7.2, at least new relative to RH7.0 I discovered this by examining the initrd image, usually created by the mkinitrd command, which you can do in the following way: see (http://surfer.nmr.mgh.harvard.edu/partition/ramdisk.html) # cat initrd-2.2.16-3ext3.img | gunzip > /tmp/myimage # file /tmp/myimage # mount /tmp/myimage /mnt/tmp -t ext2 -o loop=/dev/loop3 # ls /mnt/tmp If you examine linuxrc, you see an explicit mount to an implicit NFS volume. By the way, if you are in the business of making your own initrd, then the above reference helps, as well as the following prescription - which I cribbed from somewhere: > #!/bin/bash > > rm -rf initrd.gz > umount /dev/ram > mke2fs -m0 /dev/ram 4000 > mount -t ext2 /dev/ram /mnt/ramdisk > cp -a rootfs/* /mnt/ramdisk > umount /dev/ram > dd if=/dev/ram bs=1k count=4000 of=initrd > gzip initrd 2) Removing any initrd from the boot-image, by omitting it from the argument list passed to mknbi-linux (more on that later) causes the nfs root to be mounted directly and control to be passed to init. This also fails, though, with a complaint about not being able to find the NFS server - if you haven't read the mknbi man pages carefully. For kernels later than 2.2.18 (I was using 2.2.16 on my RH6.1++ diskless systems) you MUST specify whether and how the kernel is to autoconfigure IP: also, CONFIG_IP_PNP must be explicitly enabled in the kernel .config The command mknbi-linux --output=bootImage -ip=bootp vmlinuz.diskless makes the correct boot image. --------------------------------------------- The following is a description of how you can produce a system. It is a simple system in many ways, and should be refined a little. It also doesn't correspond exactly to what I have set up at the moment ... but no matter. By preface, I note that there are many HOWTOs (see linuxdoc.org) dealing with the matter, and in particular which specify how to configure your kernel to allow root-over-nfs. In the future, I expect one should move to a standard prescription (see, for example ltsp.org. Soon, the etherboot manual - checkout etherboot.org will provide more info too.). But for the moment, here is what I do. ---- Diskless systems at RAL: a sample setup. Set up a standard disk based system. You do not HAVE to do this on the Concurrent CPU, but it is probably best to do so because the you can configure everything, including X and all relevent post-installation packages, at one go before moving the system to a disk server. In my case, I have an EIDE disk, CDROM and a standard legacy-floppy which I can connect to the Concurrent P2 breakout - The drives sit in what were SCSI boxes and are connected through their backpanels with SCSI like connectors. The kernel must have all crucial components compiled in (because there won't be an initrd to help you out!): Important settings are: Networking Options: IP: kernel level autoconfiguration IP: BOOTP support Network File Systems: Root file system on NFS Compile in NFS, NFS-v3, ethernet-drivers, ide drivers. 2) Assume that you want to network boot host atl04 from atl01, with the later also providing nfs services. On the machine which is to be your boot server (atl01), create the directory /diskless. This should ideally be a mount point corresponding to a partition dedicated to the diskless hosts. Within /diskless, create the following directories: usr: this will contain the complete contents of the /usr partition. atl04: this will contain all other partitions. In principle, some of these CAN be shared among the net-booted machines, but the space saving is minimal. Add the /diskless directories to the /etc/exports file on atl01 and do an exportfs. Mount /diskless/usr and /diskless/atl04 on mountpoints of those names on the machine atl04, which is currently running off the disk based system which you installed above. use cp -a to copy the contents of atl04:/usr to /diskless/usr. Look at the contents of atl04:/. copy bin, boot, drivers, dev, etc, misc, root, sbin, usr, var - using "cp - a" onto /diskless/atl04. Make new directories home, initrd, mnt, opt, home, proc, tmp on /diskless/atl04. I also make a /home/atlun, a /opt/local and an /opt/share which are mount points, but these last details rather depend on your network disk sharing strategy. Be careful, here, with the permissions. /tmp must be global rw! Now create a directory /tftpboot on the server. Again, in preference, this should be a mount point to a separate partition. within this directory, there should be: 1) A network boot image (nbi) - or separate boot images for h/w which is significantly different or requiring different bootoptions (eg: bigphysarea or mem). (By the way, you should probably also run tftp in the "secure" mode.) 2) A symbolic link to /diskless/atl04. Moving to the /boot directory on the to-be-client, use mknbi to create the network boot image from your running (appropriately configured) kernel. mknbi -ip=bootp --output=linuxboot.nbi vmlinuz copy linuxboot.nbi to atl01:/tftpboot. Now you need to set up tftpd and bootpd to provide network bootfiles to your diskless hosts. This will require configuring hosts.allow and hosts.deny appropriately, and starting the servers. I set in.tftpd and portmap to ALL in hosts.deny, and then enable the appropriate hosts/netmasks in hosts.allow. In the bootptab file you might have, for example: # Concurrent CP1, S/N M2927-005 atl04:\ bs:\ hn:\ ht=ether:\ vm=rfc1048:\ to=auto:\ hd=/tftpboot:\ bf=linuxboot:\ rp=/diskless/atl04:\ ha=00409E002D81:\ sm=255.255.252.0:\ ds=130.246.40.240:\ gw=130.246.40.254:\ ip=130.246.41.188 You will need to set up the /etc/fstab correctly for the diskless system. For example: atl01:/diskless/atl04 / nfs rw,rsize=8192,wsize=8192,timeo=14,intr atl01:/diskless/usr /usr nfs rw,rsize=8192,wsize=8192,timeo=14,intr none /proc proc defaults 0 0 none /dev/pts devpts gid=5,mode=620 0 0 none /dev/shm tmpfs defaults 0 0 atl01:/home/atlun /home/atlun nfs rw,rsize=8192,wsize=8192,timeo=14,intr 0 0 Finally, there are a few alterations that rc.sysinit requires. In particular, fsck shouldn't run on nfs volumes. This can be ensured by creating an /sbin/fsck.nfs which returns successful by default without doing much else.