This is an old revision of the document!
====== Introduction ====== ===== Scope ===== This How-To is intended for anyone who wants to install Gentoo Linux on an iSCSI target. == What is covered: == * Connecting to the iSCSI target * Installing a generic base system * Installing the kernel * Installing the bootloader == What is not covered: == * Understanding and configuring a network environment ===== Credits ===== The information in this document is based on personal experience and information found on the internet. The following pages proved to be a great resource of information: * http://etherboot.org/wiki/sanboot/ubuntu_iscsi (installing ubuntu on iscsi) * http://www.gentoo-wiki.info/ISCSI (examples how to use open-iscsi on gentoo) * http://www.mail-archive.com/open-iscsi@googlegroups.com/msg00736.html (a bug that kept me busy for a while) ====== Installation ====== ===== Connect to your iSCSI disk ===== ==== Method 1: use (x)ubuntu boot CD (safest) ==== (I've only tried this with xubuntu, but other ubuntu versions should work as well.) This is by far the safest way to install, since you are booting from the live-cd. Why not just disconnect all your hard drives to avoid mistakes? === Install open-iscsi === <code> sudo aptitude install open-iscsi</code> === Discover your iSCSI target === <code> sudo iscsi_discovery <SAN-IP></code> One of the lines should be your target, for example: <code> Logout of [[sid:|8, target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest, portal: 10.9.8.250,3260]]: successful</code> === Connect to your target === <code> sudo iscsiadm -m node -p <SAN-IP> -T iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest -l</code> You should see something similar to: <code> Logging in to [[iface:|default, target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest, portal: 10.9.8.250,3260]] Login to [[iface:|default, target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest, portal: 10.9.8.250,3260]]: successful</code> ==== Method 2: use an existing/working gentoo system ==== === Update kernel === Make sure your current kernel supports iSCSI: Roll a new kernel if necessary. * I had trouble when CRC32c was compiled as a module, but it worked when built-in. * Open-iscsi will not start when ''iSCSI Initiator over TCP/IP'' is built-in, so you have to make it a module for now. * Open-iscsi will not start when ''iSCSI Transport Attributes'' is built-in, so you have to make it a module for now. <code> Device Drivers ---> SCSI device support ---> [[*]] SCSI device support <*> SCSI disk support [[*]] SCSI low-level drivers ---> <M> iSCSI Initiator over TCP/IP SCSI Transports ---> {M} iSCSI Transport Attributes Cryptographic options ---> [[*]] Cryptographic API <*> CRC32c CRC algorithm</code> === Install open-iscsi === <code> emerge sys-block/open-iscsi</code> === Start the iscsi service === <code> /etc/init.d/iscsid start</code> If you receive an error here that looks like this: <code> * Checking open-iSCSI configuration ... * Loading iSCSI modules ... * Loading libiscsi: not found ... [[|!! ]]</code> then you have to adapt your kernel: <code> <M> iSCSI Initiator over TCP/IP {M} iSCSI Transport Attributes</code> It has to be compiled as module! If you see an error like this: <code> * Checking open-iSCSI configuration ... * Loading iSCSI modules ... * Loading libiscsi ... [[|ok ]] * Loading scsi_transport_iscsi ... [[|ok ]] * Loading iscsi_tcp ... [[|ok ]] * Starting iscsid ... * Setting up iSCSI targets ... [[|!! ]] iscsiadm: No records found!</code> Then that's fine, since we don't have any targets yet. === Discover your iscsi-target === Find the MAC address of your network card: <code> ifconfig eth0 | grep HWaddr</code> You should see something like: <code> eth0 Link encap:Ethernet HWaddr 00:11:22:33:44:55</code> Discover the target: <code> iscsiadm -m iface -I iface0 --op=new</code> You should see: <code> New interface iface0 added</code> next: <code> iscsiadm -m iface -I iface0 --op=update -n iface.hwaddress -v <MAC-ADDRESS></code> You should see: <code> iface0 updated.</code> next: <code> iscsiadm -m discovery -t st -p <TARGET-IP> -P 1</code> One of the lines should be your target, for example: <code> Target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest Portal: 10.9.8.250:3260,1 Iface Name: iface0</code> === Connect to your iscsi-target === <code> iscsiadm -m node -T iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest -l</code> You should see something like: <code> Logging in to [[iface:|iface0, target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest, portal: 10.9.8.250,3260]] Login to [[iface:|iface0, target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest, portal: 10.9.8.250,3260]]: successful</code> ===== Chroot into your iSCSI disk ===== (Make sure you are root from now on) === Find out where the disk is attached to === (In this case the "disk" is a file filled with zeroes, so fdisk will complain there is no valid partition table. It does not matter.) <code> fdisk -l</code> You should see something similar to: <code> Disk /dev/sda: 4194 MB, 4194304000 bytes 130 heads, 62 sectors/track, 1016 cylinders Units = cylinders of 8060 * 512 = 4126720 bytes Disk identifier: 0x00000000</code> <code> Disk /dev/sda doesn't contain a valid partition table</code> **WARNING: Pay attention to what device your iscsi disk is attached, if you don't have other disks connected to your computer the device will probably be /dev/sda, if you do have other disks you have to find out what device your iscsi disk is connected to! You don't want to (re)format the wrong disk! I will refer to the disk as /dev/sdX.** === Partition the iSCSI disk === (I've made only 1 partition to make it simple. You could add boot and swap if you want) <code> fdisk /dev/sdX</code> === Format the partition === **Warning: read the bold warning in paragraph "Find out where the disk is attached to"** <code> mke2fs -j /dev/sdX1</code> === Make a directory for your chroot environment === <code> mkdir /mnt/gentoo</code> === Mount your iSCSI disk === <code> mount /dev/sdX1 /mnt/gentoo</code> ===== Install Gentoo base system ===== From here on you have to download a stage3 just like you would in a normal installation. Unpack the stage in /mnt/gentoo. === Mount other filesystems === <code> mount -t proc none /mnt/gentoo/proc mount -o bind /dev /mnt/gentoo/dev</code> === Copy resolver config === <code> cp -L /etc/resolv.conf /mnt/gentoo/etc/</code> ==== Set timezone ==== <code> cp /usr/share/zoneinfo/Europe/Brussels /etc/localtime</code> ==== Set hostname ==== <code> nano -w /etc/conf.d/hostname</code> ==== Edit config files ==== <code> nano -w /etc/conf.d/clock nano -w /etc/rc.conf nano -w /etc/conf.d/rc nano -w /etc/conf.d/keymaps</code> ==== Install system tools ==== <code> emerge app-admin/syslog-ng sys-process/vixie-cron rc-update add syslog-ng default ; rc-update add vixie-cron default</code> ==== Set a root password ==== <code> passwd</code> ==== Network configuration ==== **IMPORTANT:** Since your storage is connected over the network, do not allow init scripts to start or restart the network! <code> nano -w /etc/conf.d/net</code> <code> config_eth0=( "noop" )</code> ==== Fstab ==== Add your iSCSI disk in fstab like any normal hard disk: <code> nano -w /etc/fstab</code> <code> /dev/sda1 / ext3 noatime 0 1</code> ===== Kernel configuration ===== ==== Install the gentoo-sources ==== <code> emerge sys-kernel/gentoo-sources</code> ==== Install genkernel ==== You need at least version 3.4.10.907 of genkernel so you may have to unmask it first <code> ACCEPT_KEYWORDS="~yourach" emerge sys-kernel/genkernel</code> ==== Compile the sources with genkernel ==== Don't forget to add --iscsi for iscsi support. <code> genkernel --menuconfig --iscsi all</code> === Enable iSCSI === I had trouble when CRC32c was compiled as M so i made it *. Now iSCSI Initiator over TCP/IP has to be * not M! Now iSCSI Transport Attributes has to be * not M! <code> Device Drivers ---> SCSI device support ---> [[*]] SCSI device support <*> SCSI disk support [[*]] SCSI low-level drivers ---> <*> iSCSI Initiator over TCP/IP SCSI Transports ---> {*} iSCSI Transport Attributes Cryptographic options ---> [[*]] Cryptographic API <*> CRC32c CRC algorithm</code> === Enable networking === <code> Networking support ---> Networking options ---> <*> Packet socket <*> Unix domain sockets [[*]] TCP/IP networking [[*]] IP: multicasting [[*]] IP: kernel level autoconfiguration [[*]] IP: DHCP support</code> === Enable your network card === Enable your network card as BUILT IN [[*]], not module!! <code> Device Drivers ---> [[*]] Network device support ---> [[?]] Ethernet (10 or 100Mbit) ---> [[?]] Ethernet (1000 Mbit) ---> [[?]] Ethernet (10000 Mbit) ---></code> ===== Install Bootloader ===== ==== Install GRUB ==== <code> emerge sys-boot/grub</code> ==== Configure GRUB ==== <code> grub</code> **WARNING, WARNING, WARNING: make sure you enter the right disk!! if you are running the ubuntu-cd and don't have any real harddisks connected to your computer then hdx=hd0, if you are installing from a real hard disk then hdX=hd1 or hd2 or so on!!! ** <code> root (hdX,0) setup (hdX) quit</code> ==== Edit GRUB configuration ==== You have to set at least these options for iSCSI to work: ip=dhcp, iscsi_target=, iscsi_address=, iscsi_initiatorname ====== <code> nano -w /boot/grub/grub.conf</code> <code> default 0 timeout 10</code> <code> title Gentoo Linux 2.6.31-r6 iSCSI root (hd0,0) kernel /boot/kernel-genkernel-x86_64-2.6.31-gentoo-r6 ip=dhcp root=/dev/ram0 init=/linuxrc ramdisk=8192 real_root=/dev/sda1 iscsi_target=iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest iscsi_address=10.9.8.250 iscsi_initiatorname=iqn.1993-08.org.gentooclient5:01:22b2ed5d3ccc initrd /boot/initramfs-genkernel-x86_64-2.6.31-gentoo-r6</code> ===== Leave chroot ===== <code> cd ~ exit</code> If you don't have a boot partition like me, then you can leave out /mnt/gentoo/boot <code> umount /mnt/gentoo/dev /mnt/gentoo/proc /mnt/gentoo/boot /mnt/gentoo</code> ====== Troubleshooting ====== ===== Made a mistake in the base system ===== You can connect to your iSCSI disk again with one of the methods above, chroot and fix the problem. ===== System won't boot and drops you in a shell ===== Enter the shell and try to figure out what went wrong. * Did the network interface come up with an ip-address? <code> ifconfig eth0</code> * You may have forgotten to enable ''IP: kernel level autoconfiguration'' and/or DHCP in the kernel * You may have forgotten to enable your network card in the kernel (* not M) * Does dmesg show any warnings? <code> dmesg</code> If demsg complains about crc32 you probably compiled it as M not *. * Can you connect to the target manually? <code> iscsistart --help</code> ===== Gentoo boots partially but then complains about a read-only filesystem ===== You probably didn't edit /etc/fstab right <code> mount -o remount rw / nano -w /etc/fstab sync reboot</code> ===== Kernel boots but hangs/crashes when it tries to bring up eth0 ===== You cannot allow the system to start or restart the network connection, this will sever your iscsi connection! <code> mount -o remount rw / nano -w /etc/conf.d/net</code> <code> config_eth0=( "noop" )</code> <code> sync reboot</code> ===== Boot fails when adding other storage to computer ===== If you add a hard disk/USB drive/USB stick/etc. to the computer, it will probably take the /dev/sda spot and your iSCSI disk will be moved to /dev/sdb or /dev/sdc, etc. Using filesystem labels will help in this situation. ===== Gentoo will not shut down properly ===== Because an iscsi mount is not recognized as netmount (like nfs) by Gentoo, the system will try to shutdown the network interface during normal shutdown/reboot procedure. This will obviously break the iscsi connection and let the system hang. To avoid this the following workaround can be used: add (where eth0 is the interface being used for the iscsi connection): /etc/init.d/net.eth0 zap to /etc/init.d/local.start (or the start section of /etc/init.d/local depending on your baselayout version). Doing so will make Gentoo think eth0 is already down, so it wont try to shut it down on halt. All other init-scripts that depend on net will still be invoked to shut them down cleanly, just eth0 itself wont be stopped.
