====== Introduction ====== ===== Scope ===== This How-To is intended for anyone who wants to install Gentoo Linux on an iSCSI target. == What is covered: == * Connecting to the iSCSI target * Installing a generic base system * Installing the kernel * Installing the bootloader == What is not covered: == * Understanding and configuring a network environment ===== Credits ===== The information in this document is based on personal experience and information found on the internet. The following pages proved to be a great resource of information: * http://etherboot.org/wiki/sanboot/ubuntu_iscsi (installing ubuntu on iscsi) * http://www.gentoo-wiki.info/ISCSI (examples how to use open-iscsi on gentoo) * http://www.mail-archive.com/open-iscsi@googlegroups.com/msg00736.html (a bug that kept me busy for a while) ====== Installation ====== ===== Connect to your iSCSI disk ===== ==== Method 1: use (x)ubuntu boot CD (safest) ==== (I've only tried this with xubuntu, but other ubuntu versions should work as well.) This is by far the safest way to install, since you are booting from the live-cd. Why not just disconnect all your hard drives to avoid mistakes? === Install open-iscsi === sudo aptitude install open-iscsi === Discover your iSCSI target === sudo iscsi_discovery One of the lines should be your target, for example: Logout of [[sid:|8, target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest, portal: 10.9.8.250,3260]]: successful === Connect to your target === sudo iscsiadm -m node -p -T iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest -l You should see something similar to: Logging in to [[iface:|default, target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest, portal: 10.9.8.250,3260]] Login to [[iface:|default, target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest, portal: 10.9.8.250,3260]]: successful ==== Method 2: use an existing/working gentoo system ==== === Update kernel === Make sure your current kernel supports iSCSI: Roll a new kernel if necessary. * I had trouble when CRC32c was compiled as a module, but it worked when built-in. * Open-iscsi will not start when ''iSCSI Initiator over TCP/IP'' is built-in, so you have to make it a module for now. * Open-iscsi will not start when ''iSCSI Transport Attributes'' is built-in, so you have to make it a module for now. Device Drivers ---> SCSI device support ---> [[*]] SCSI device support <*> SCSI disk support [[*]] SCSI low-level drivers ---> iSCSI Initiator over TCP/IP SCSI Transports ---> {M} iSCSI Transport Attributes Cryptographic options ---> [[*]] Cryptographic API <*> CRC32c CRC algorithm === Install open-iscsi === emerge sys-block/open-iscsi === Start the iscsi service === /etc/init.d/iscsid start If you receive an error here that looks like this: * Checking open-iSCSI configuration ... * Loading iSCSI modules ... * Loading libiscsi: not found ... [[|!! ]] then you have to adapt your kernel: iSCSI Initiator over TCP/IP {M} iSCSI Transport Attributes It has to be compiled as module! If you see an error like this: * Checking open-iSCSI configuration ... * Loading iSCSI modules ... * Loading libiscsi ... [[|ok ]] * Loading scsi_transport_iscsi ... [[|ok ]] * Loading iscsi_tcp ... [[|ok ]] * Starting iscsid ... * Setting up iSCSI targets ... [[|!! ]] iscsiadm: No records found! Then that's fine, since we don't have any targets yet. === Discover your iscsi-target === Find the MAC address of your network card: ifconfig eth0 | grep HWaddr You should see something like: eth0 Link encap:Ethernet HWaddr 00:11:22:33:44:55 Discover the target: iscsiadm -m iface -I iface0 --op=new You should see: New interface iface0 added next: iscsiadm -m iface -I iface0 --op=update -n iface.hwaddress -v You should see: iface0 updated. next: iscsiadm -m discovery -t st -p -P 1 One of the lines should be your target, for example: Target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest Portal: 10.9.8.250:3260,1 Iface Name: iface0 === Connect to your iscsi-target === iscsiadm -m node -T iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest -l You should see something like: Logging in to [[iface:|iface0, target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest, portal: 10.9.8.250,3260]] Login to [[iface:|iface0, target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest, portal: 10.9.8.250,3260]]: successful ===== Chroot into your iSCSI disk ===== (Make sure you are root from now on) === Find out where the disk is attached to === (In this case the "disk" is a file filled with zeroes, so fdisk will complain there is no valid partition table. It does not matter.) fdisk -l You should see something similar to: Disk /dev/sda: 4194 MB, 4194304000 bytes 130 heads, 62 sectors/track, 1016 cylinders Units = cylinders of 8060 * 512 = 4126720 bytes Disk identifier: 0x00000000 Disk /dev/sda doesn't contain a valid partition table **WARNING: Pay attention to what device your iscsi disk is attached, if you don't have other disks connected to your computer the device will probably be /dev/sda, if you do have other disks you have to find out what device your iscsi disk is connected to! You don't want to (re)format the wrong disk! I will refer to the disk as /dev/sdX.** === Partition the iSCSI disk === (I've made only 1 partition to make it simple. You could add boot and swap if you want) fdisk /dev/sdX === Format the partition === **Warning: read the bold warning in paragraph "Find out where the disk is attached to"** mke2fs -j /dev/sdX1 === Make a directory for your chroot environment === mkdir /mnt/gentoo === Mount your iSCSI disk === mount /dev/sdX1 /mnt/gentoo ===== Install Gentoo base system ===== From here on you have to download a stage3 just like you would in a normal installation. Unpack the stage in /mnt/gentoo. === Mount other filesystems === mount -t proc none /mnt/gentoo/proc mount -o bind /dev /mnt/gentoo/dev === Copy resolver config === cp -L /etc/resolv.conf /mnt/gentoo/etc/ ==== Set timezone ==== cp /usr/share/zoneinfo/Europe/Brussels /etc/localtime ==== Set hostname ==== nano -w /etc/conf.d/hostname ==== Edit config files ==== nano -w /etc/conf.d/clock nano -w /etc/rc.conf nano -w /etc/conf.d/rc nano -w /etc/conf.d/keymaps ==== Install system tools ==== emerge app-admin/syslog-ng sys-process/vixie-cron rc-update add syslog-ng default ; rc-update add vixie-cron default ==== Set a root password ==== passwd ==== Network configuration ==== **IMPORTANT:** Since your storage is connected over the network, do not allow init scripts to start or restart the network! nano -w /etc/conf.d/net config_eth0=( "noop" ) ==== Fstab ==== Add your iSCSI disk in fstab like any normal hard disk: nano -w /etc/fstab /dev/sda1 / ext3 noatime 0 1 ===== Kernel configuration ===== ==== Install the gentoo-sources ==== emerge sys-kernel/gentoo-sources ==== Install genkernel ==== You need at least version 3.4.10.907 of genkernel so you may have to unmask it first ACCEPT_KEYWORDS="~yourach" emerge sys-kernel/genkernel ==== Compile the sources with genkernel ==== Don't forget to add --iscsi for iscsi support. genkernel --menuconfig --iscsi all === Enable iSCSI === I had trouble when CRC32c was compiled as M so i made it *. Now iSCSI Initiator over TCP/IP has to be * not M! Now iSCSI Transport Attributes has to be * not M! Device Drivers ---> SCSI device support ---> [[*]] SCSI device support <*> SCSI disk support [[*]] SCSI low-level drivers ---> <*> iSCSI Initiator over TCP/IP SCSI Transports ---> {*} iSCSI Transport Attributes Cryptographic options ---> [[*]] Cryptographic API <*> CRC32c CRC algorithm === Enable networking === Networking support ---> Networking options ---> <*> Packet socket <*> Unix domain sockets [[*]] TCP/IP networking [[*]] IP: multicasting [[*]] IP: kernel level autoconfiguration [[*]] IP: DHCP support === Enable your network card === Enable your network card as BUILT IN [[*]], not module!! Device Drivers ---> [[*]] Network device support ---> [[?]] Ethernet (10 or 100Mbit) ---> [[?]] Ethernet (1000 Mbit) ---> [[?]] Ethernet (10000 Mbit) ---> ===== Install Bootloader ===== ==== Install GRUB ==== emerge sys-boot/grub ==== Configure GRUB ==== grub **WARNING, WARNING, WARNING: make sure you enter the right disk!! if you are running the ubuntu-cd and don't have any real harddisks connected to your computer then hdx=hd0, if you are installing from a real hard disk then hdX=hd1 or hd2 or so on!!! ** root (hdX,0) setup (hdX) quit ==== Edit GRUB configuration ==== You have to set at least these options for iSCSI to work: ip=dhcp, iscsi_target=, iscsi_address=, iscsi_initiatorname ====== nano -w /boot/grub/grub.conf default 0 timeout 10 title Gentoo Linux 2.6.31-r6 iSCSI root (hd0,0) kernel /boot/kernel-genkernel-x86_64-2.6.31-gentoo-r6 ip=dhcp root=/dev/ram0 init=/linuxrc ramdisk=8192 real_root=/dev/sda1 iscsi_target=iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest iscsi_address=10.9.8.250 iscsi_initiatorname=iqn.1993-08.org.gentooclient5:01:22b2ed5d3ccc initrd /boot/initramfs-genkernel-x86_64-2.6.31-gentoo-r6 ===== Leave chroot ===== cd ~ exit If you don't have a boot partition like me, then you can leave out /mnt/gentoo/boot umount /mnt/gentoo/dev /mnt/gentoo/proc /mnt/gentoo/boot /mnt/gentoo ====== Troubleshooting ====== ===== Made a mistake in the base system ===== You can connect to your iSCSI disk again with one of the methods above, chroot and fix the problem. ===== System won't boot and drops you in a shell ===== Enter the shell and try to figure out what went wrong. * Did the network interface come up with an ip-address? ifconfig eth0 * You may have forgotten to enable ''IP: kernel level autoconfiguration'' and/or DHCP in the kernel * You may have forgotten to enable your network card in the kernel (* not M) * Does dmesg show any warnings? dmesg If demsg complains about crc32 you probably compiled it as M not *. * Can you connect to the target manually? iscsistart --help ===== Gentoo boots partially but then complains about a read-only filesystem ===== You probably didn't edit /etc/fstab right mount -o remount rw / nano -w /etc/fstab sync reboot ===== Kernel boots but hangs/crashes when it tries to bring up eth0 ===== You cannot allow the system to start or restart the network connection, this will sever your iscsi connection! mount -o remount rw / nano -w /etc/conf.d/net config_eth0=( "noop" ) sync reboot ===== Boot fails when adding other storage to computer ===== If you add a hard disk/USB drive/USB stick/etc. to the computer, it will probably take the /dev/sda spot and your iSCSI disk will be moved to /dev/sdb or /dev/sdc, etc. Using filesystem labels will help in this situation. ===== Gentoo will not shut down properly ===== Because an iscsi mount is not recognized as netmount (like nfs) by Gentoo, the system will try to shutdown the network interface during normal shutdown/reboot procedure. This will obviously break the iscsi connection and let the system hang. To avoid this the following workaround can be used add (where eth0 is the interface being used for the iscsi connection): /etc/init.d/net.eth0 zap to /etc/init.d/local.start (or the start section of /etc/init.d/local depending on your baselayout version). Doing so will make Gentoo think eth0 is already down, so it wont try to shut it down on halt. All other init-scripts that depend on net will still be invoked to shut them down cleanly, just eth0 itself wont be stopped.