This is an old revision of the document!
====== Booting debian over iscsi ====== Here I will be documenting my first attempt to boot linux over iscsi. ===== Reason to choose debian ===== - Chose debian because the squashfs of debian works fine and there are no other problems. - Also, all the kernel modules needed for iscsi initiator are inside initramfs so, less work for me. :-) - No considerable success yet :-( ===== Installing the target ===== The target for these experiments is installed on my laptop (Ubuntu-9.04) with following procedure. Most of these steps are taken from blog [[http://www.howtoforge.com/using-iscsi-on-ubuntu-9.04-initiator-and-target|Using iscsi over ubuntu 9.04]]\\ Installing iscsi target <code> sudo apt-get install iscsitarget </code> Configure the target by editing ''/etc/default/iscsitarget'', ''/etc/ietd.conf'' and ''/etc/initiators.allow'' files as follows\\ - Open ''/etc/default/iscsitarget'' and set ''ISCSITARGET_ENABLE=true'' - Open ''/etc/ietd.conf'', and comment out all lines, then add following paragraph <code> Target iqn.2001-04.com.example:storage.lun1 IncomingUser OutgoingUser Lun 0 Path=/var/www/debian-live-501-i386-gnome-desktop.iso,Type=fileio,IOMode=ro Alias LUN1 </code> Now we add following line to "/etc/initiators.allow" <code> ALL ALL </code> This should allow any initiator to access any lun device\\ and then, restart the target <code> sudo /etc/init.d/iscsitarget restart </code> ===== Testing the target ===== The iscsi target is tested by installing iscsi initiator on same laptop. Following is the procedure.\\ * Installing <code> sudo apt-get install open-iscsi </code> * configure iscsi initiator to start nodes automatically\\ Edit the file ''/etc/iscsi/iscsid.conf'' to have line ''node.startup = automatic'' * and then restart the initiator <code> /etc/init.d/open-iscsi restart </code> * Now, do the discovery of nodes available <code> $ sudo iscsiadm -m discovery -t st -p 192.168.0.1 192.168.0.1:3260,1 iqn.2001-04.com.example:storage.lun1 </code> * I dont know why this command is needed, but it was in the blog, so I am adding it here <code> $ sudo iscsiadm -m node 192.168.0.1:3260,1 iqn.2001-04.com.example:storage.lun1 </code> * Logging in <code> $ sudo iscsiadm -m node --targetname "iqn.2001-04.com.example:storage.lun1" --portal "192.168.0.1:3260" --login Logging in to [iface: default, target: iqn.2001-04.com.example:storage.lun1, portal: 192.168.0.1,3260] Login to [iface: default, target: iqn.2001-04.com.example:storage.lun1, portal: 192.168.0.1,3260]: successful </code> * and it is done, you can see the new device with ''fdisk -l'' command <code> $ sudo fdisk -l [sudo] password for pravin: Disk /dev/sda: 250.0 GB, 250059350016 bytes 255 heads, 63 sectors/track, 30401 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0xa9f9aa9b Device Boot Start End Blocks Id System /dev/sda1 1 5 40131 de Dell Utility [...] [...] [...] Disk /dev/sdc: 724 MB, 724785152 bytes 23 heads, 61 sectors/track, 1008 cylinders Units = cylinders of 1403 * 512 = 718336 bytes Disk identifier: 0x00000000 Disk /dev/sdc doesn't contain a valid partition table $ sudo mount -t iso9660 /dev/sdc mpoint mount: block device /dev/sdc is write-protected, mounting read-only $ mount /dev/sda6 on / type ext3 (rw,relatime,errors=remount-ro) [...] [...] [...] /dev/sdc on /home/pravin/Etherboot/git/BKO.git/pxeknife/debian/debian-live-501-i386/mpoint type iso9660 (ro) $ ls mpoint doc isolinux live md5sum.txt </code> As Now I know that my target is working correctly, I am free to try out booting from this target 8-) ===== Configuring initramfs as initiator ===== I need to add all the tools, configuration files and kernel modules to initramfs so that I can use them from there to connect. * ** Step 1 **, __Kernel Modules__ : on checking the contents of ''/etc/init.d/open-iscsi'' it seems that following kernel modules are to be inserted <code> modprobe iscsi_tcp modprobe ib_iser </code> * These modules are already in debian initramfs, so no need to worry about that :) * ** Step 2 ** __Tools needed__ : Again ''/etc/init.d/open-iscsi'' tells us that it uses ''/sbin/iscsid'' and ''/sbin/iscsiadm''\\ but there are more iscsi related tools in /sbin <code> $ ls /sbin/iscsi* /sbin/iscsiadm /sbin/iscsid /sbin/iscsi_discovery /sbin/iscsi-iname /sbin/iscsistart </code> To avoid any missing dependencies, better to add all above tools to initramfs. * ** Step 3 ** __Configuration Files__ : From man page of iscsiadm and iscsid, we can guess that following configuration files will be needed <code> /etc/iscsi/iscsid.conf /etc/iscsi/initiatorname.iscsi </code> ===== Testing new initramfs ===== The HTTPFS code has been commented out so that initramfs will give me rescue shell, which can be used to start the iscsi initiator.\\ Following commands were executed once the rescue shell is provided. <code> modprobe iscsi_tcp modprobe ib_iser mkdir /var/run /sbin/iscsid iscsiadm -m discovery -t st -p 192.168.0.1 </code> and no node is shown :(\\ I tried to run iscsid in debug mode with following command <code> /sbin/iscsid -d 8 -f iscsid : sysfs_init: sysfs_path='/sys' [...] [...] iscsid: reaped pid 994, reap_count now 0 </code> and also ran ''iscsiadm'' in debug mode with following output <code> (initramfs) iscsiadm -d 8 -m discovery -t st -p 192.168.0.1 iscsiadm: ip 192.168.0.1, port 3260, tgpt -1 iscsiadm: Max file limits 1024 1024 </code> This output is definitely very less compare to the output of same command executed on my laptop.\\ **Note:** __This was executed over qeum.__ I am not sure if qemu has created any problem. ===== Debugging the problem ===== To make sure debien in virtual environment can use iscsi with the tools, and configuration files used inside initramfs\\ - I booted debian live CD with virtualbox - inserted kernel modules just as above - Copied configuration files into live cd. - Copied tools into live cd - Executed above commands from inside live CD - The ''iscsiadm -m discovery'' command did gave output. <code> # iscsiadm -m discovery -t st -p 192.168.0.1 192.168.0.1:3260,1 iqn.2001-04.com.example:storage.lun1 </code> So, the tools do work. but may be they are able to use other files, tools and modules which are not present in initrd.\\ How to find those dependencies? ===== Debugging v2 ===== Finally, added the ''strace'' to initramfs, and now trying find what is the problem because of which iscsiadm is dying\\ <code> socket(PF_FILE, SOCK_STREAM, 0) = 3 connect(3, {sa_family=AF_FILE, path=@"ISCSIADM_ABSTRACT_NAMESPACE"...}, 110) = 0 write(3, "\n\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 5244) = -1 EPIPE (Broken pipe) </code> This is what killing the iscsiadm imatuarly. ==== Checking the code of open-iscsi ==== With help of stephana, the likely problem is located at following location <code> stefanha: http://git.kernel.org/?p=linux/kernel/git/mnc/open-iscsi.git;a=blob;f=usr/mgmt_ipc.c;h=e784bcac019a84522e62beff1b49b1ca62e37f5a;hb=HEAD (07/19/2009 04:34:21 PM) stefanha: less1: Perhaps you're hitting this: (07/19/2009 04:34:27 PM) stefanha: http://git.kernel.org/?p=linux/kernel/git/mnc/open-iscsi.git;a=blob;f=usr/mgmt_ipc.c;h=e784bcac019a84522e62beff1b49b1ca62e37f5a;hb=HEAD#l549 (07/19/2009 04:34:31 PM) stefanha: if (!mgmt_peeruser(fd, user) || strncmp(user, "root", PEERUSER_MAX)) { (07/19/2009 04:34:43 PM) stefanha: write(2) will return EPIPE if the UNIX domain socket was closed. (07/19/2009 04:35:17 PM) stefanha: mgmt_peeruser() is defined here: http://git.kernel.org/?p=linux/kernel/git/mnc/open-iscsi.git;a=blob;f=usr/mgmt_ipc.c;h=e784bcac019a84522e62beff1b49b1ca62e37f5a;hb=HEAD#l342 (07/19/2009 04:38:15 PM) less1: stefanha: is it checking if I have root privilages? I am not sure if I understand which peeruser credential is it refering. (07/19/2009 04:40:08 PM) stefanha: Your user name must be "root". (07/19/2009 04:40:25 PM) stefanha: Otherwise iscsid will not accept commands from the UNIX domain socket. </code> === Possible solution : hacking open-iscsi === modified the ''iscsiadm'' code so that, it will not fail when username is not ''root''.\\ With is modification, the discovery part is working, but now there is error at login part. Following is the error. <code> connection2:0: Could not create connection due to crc32c loading error. Make sure the crc32c module is build as module into the kernel. </code> The first attempt will be to download the kernel module from Live CD of debian after booting it from CD.\\ --- Doing it ===== Working Solution ===== There was need for two extra kernel modules - libcrc32c.ko - crc32c.ko Fetched these modules from live-CD and added them.\\ There as also need for small sleep after ''iscsiadm --login'' as ''/dev/sda'' was taking little time to appear.\\ This was handled by adding ''sleep 2''\\ Two new parameters were introduced - ''iscsifs=<IP:PORT>'' iscsi server IP and port number - ''target=<targetname>'' the iscsi target name of the disk (entire name is not needed, some part which can uniquely identify apart from other disks exported at same machine) ===== Remote ISCSI ===== Following are the errors that I get when connecting to remote iscsi <code> Connection1:0: ping timeout of 5 secs expired, last rx 307737, last ping 308987, now 310237 Connection1:0: detected conn error (1011) SQUASHFS error : Unable to read page, block 0x1b392 Bus error </code> Most of these errors are related to timing, and I believe that these errors are there because of delays in accessing required data. ===== Target behaviour ===== sometimes, you may get following error when logging in with iscsiadm <code> iscsiadm -m node --targetname "${targetname}" --portal "${ISCSIFS}" --login records not found!. </code> this is because the ${ISCSIFS} is not IP but dnsname. Dnsname may work with ''iscsiadm -m discovery -t st -p "${ISCSIFS}" '' but it may verywell fail when same dnsname is given to login. [[http://groups.google.com/group/open-iscsi/browse_thread/thread/c556508505ecc461|mailing list discussion]]
