Here I will be documenting my first attempt to boot linux over iscsi.
The target for these experiments is installed on my laptop (Ubuntu-9.04) with following procedure.
Most of these steps are taken from blog Using iscsi over ubuntu 9.04
Installing iscsi target
sudo apt-get install iscsitarget
Configure the target by editing /etc/default/iscsitarget
, /etc/ietd.conf
and /etc/initiators.allow
files as follows
/etc/default/iscsitarget
and set ISCSITARGET_ENABLE=true
/etc/ietd.conf
, and comment out all lines, then add following paragraphTarget iqn.2001-04.com.example:storage.lun1 IncomingUser OutgoingUser Lun 0 Path=/var/www/debian-live-501-i386-gnome-desktop.iso,Type=fileio,IOMode=ro Alias LUN1
Now we add following line to “/etc/initiators.allow”
ALL ALL
This should allow any initiator to access any lun device
and then, restart the target
sudo /etc/init.d/iscsitarget restart
The iscsi target is tested by installing iscsi initiator on same laptop. Following is the procedure.
sudo apt-get install open-iscsi
* configure iscsi initiator to start nodes automatically\\
Edit the file /etc/iscsi/iscsid.conf
to have line node.startup = automatic
* and then restart the initiator
/etc/init.d/open-iscsi restart
$ sudo iscsiadm -m discovery -t st -p 192.168.0.1 192.168.0.1:3260,1 iqn.2001-04.com.example:storage.lun1
$ sudo iscsiadm -m node 192.168.0.1:3260,1 iqn.2001-04.com.example:storage.lun1
$ sudo iscsiadm -m node --targetname "iqn.2001-04.com.example:storage.lun1" --portal "192.168.0.1:3260" --login Logging in to [iface: default, target: iqn.2001-04.com.example:storage.lun1, portal: 192.168.0.1,3260] Login to [iface: default, target: iqn.2001-04.com.example:storage.lun1, portal: 192.168.0.1,3260]: successful
fdisk -l
command$ sudo fdisk -l [sudo] password for pravin: Disk /dev/sda: 250.0 GB, 250059350016 bytes 255 heads, 63 sectors/track, 30401 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0xa9f9aa9b Device Boot Start End Blocks Id System /dev/sda1 1 5 40131 de Dell Utility [...] [...] [...] Disk /dev/sdc: 724 MB, 724785152 bytes 23 heads, 61 sectors/track, 1008 cylinders Units = cylinders of 1403 * 512 = 718336 bytes Disk identifier: 0x00000000 Disk /dev/sdc doesn't contain a valid partition table $ sudo mount -t iso9660 /dev/sdc mpoint mount: block device /dev/sdc is write-protected, mounting read-only $ mount /dev/sda6 on / type ext3 (rw,relatime,errors=remount-ro) [...] [...] [...] /dev/sdc on /home/pravin/Etherboot/git/BKO.git/pxeknife/debian/debian-live-501-i386/mpoint type iso9660 (ro) $ ls mpoint doc isolinux live md5sum.txt
As Now I know that my target is working correctly, I am free to try out booting from this target
I need to add all the tools, configuration files and kernel modules to initramfs so that I can use them from there to connect.
/etc/init.d/open-iscsi
it seems that following kernel modules are to be insertedmodprobe iscsi_tcp modprobe ib_iser
/etc/init.d/open-iscsi
tells us that it uses /sbin/iscsid
and /sbin/iscsiadm
but there are more iscsi related tools in /sbin
$ ls /sbin/iscsi* /sbin/iscsiadm /sbin/iscsid /sbin/iscsi_discovery /sbin/iscsi-iname /sbin/iscsistart
To avoid any missing dependencies, better to add all above tools to initramfs.
/etc/iscsi/iscsid.conf /etc/iscsi/initiatorname.iscsi
The HTTPFS code has been commented out so that initramfs will give me rescue shell, which can be used to start the iscsi initiator.
Following commands were executed once the rescue shell is provided.
modprobe iscsi_tcp modprobe ib_iser mkdir /var/run /sbin/iscsid iscsiadm -m discovery -t st -p 192.168.0.1
and no node is shown :(
I tried to run iscsid in debug mode with following command
/sbin/iscsid -d 8 -f iscsid : sysfs_init: sysfs_path='/sys' [...] [...] iscsid: reaped pid 994, reap_count now 0
and also ran iscsiadm
in debug mode with following output
(initramfs) iscsiadm -d 8 -m discovery -t st -p 192.168.0.1 iscsiadm: ip 192.168.0.1, port 3260, tgpt -1 iscsiadm: Max file limits 1024 1024
This output is definitely very less compare to the output of same command executed on my laptop.
Note: This was executed over qeum. I am not sure if qemu has created any problem.
To make sure debien in virtual environment can use iscsi with the tools, and configuration files used inside initramfs
iscsiadm -m discovery
command did gave output.# iscsiadm -m discovery -t st -p 192.168.0.1 192.168.0.1:3260,1 iqn.2001-04.com.example:storage.lun1
So, the tools do work. but may be they are able to use other files, tools and modules which are not present in initrd.
How to find those dependencies?
Finally, added the strace
to initramfs, and now trying find what is the problem because of which iscsiadm is dying
socket(PF_FILE, SOCK_STREAM, 0) = 3 connect(3, {sa_family=AF_FILE, path=@"ISCSIADM_ABSTRACT_NAMESPACE"...}, 110) = 0 write(3, "\n\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 5244) = -1 EPIPE (Broken pipe)
This is what killing the iscsiadm imatuarly.
With help of stephana, the likely problem is located at following location
stefanha: http://git.kernel.org/?p=linux/kernel/git/mnc/open-iscsi.git;a=blob;f=usr/mgmt_ipc.c;h=e784bcac019a84522e62beff1b49b1ca62e37f5a;hb=HEAD (07/19/2009 04:34:21 PM) stefanha: less1: Perhaps you're hitting this: (07/19/2009 04:34:27 PM) stefanha: http://git.kernel.org/?p=linux/kernel/git/mnc/open-iscsi.git;a=blob;f=usr/mgmt_ipc.c;h=e784bcac019a84522e62beff1b49b1ca62e37f5a;hb=HEAD#l549 (07/19/2009 04:34:31 PM) stefanha: if (!mgmt_peeruser(fd, user) || strncmp(user, "root", PEERUSER_MAX)) { (07/19/2009 04:34:43 PM) stefanha: write(2) will return EPIPE if the UNIX domain socket was closed. (07/19/2009 04:35:17 PM) stefanha: mgmt_peeruser() is defined here: http://git.kernel.org/?p=linux/kernel/git/mnc/open-iscsi.git;a=blob;f=usr/mgmt_ipc.c;h=e784bcac019a84522e62beff1b49b1ca62e37f5a;hb=HEAD#l342 (07/19/2009 04:38:15 PM) less1: stefanha: is it checking if I have root privilages? I am not sure if I understand which peeruser credential is it refering. (07/19/2009 04:40:08 PM) stefanha: Your user name must be "root". (07/19/2009 04:40:25 PM) stefanha: Otherwise iscsid will not accept commands from the UNIX domain socket.
modified the iscsiadm
code so that, it will not fail when username is not root
.
With is modification, the discovery part is working, but now there is error at login part. Following is the error.
connection2:0: Could not create connection due to crc32c loading error. Make sure the crc32c module is build as module into the kernel.
The first attempt will be to download the kernel module from Live CD of debian after booting it from CD.
— Doing it
There was need for two extra kernel modules
Fetched these modules from live-CD and added them.
There as also need for small sleep after iscsiadm –login
as /dev/sda
was taking little time to appear.
This was handled by adding sleep 2
Two new parameters were introduced
iscsifs=<IP:PORT>
iscsi server IP and port numbertarget=<targetname>
the iscsi target name of the disk (entire name is not needed, some part which can uniquely identify apart from other disks exported at same machine)Following are the errors that I get when connecting to remote iscsi
Connection1:0: ping timeout of 5 secs expired, last rx 307737, last ping 308987, now 310237 Connection1:0: detected conn error (1011) SQUASHFS error : Unable to read page, block 0x1b392 Bus error
Most of these errors are related to timing, and I believe that these errors are there because of delays in accessing required data.
sometimes, you may get following error when logging in with iscsiadm
iscsiadm -m node --targetname "${targetname}" --portal "${ISCSIFS}" --login records not found!.
this is because the ${ISCSIFS} is not IP but dnsname.
Dnsname may work with iscsiadm -m discovery -t st -p “${ISCSIFS}”
but it may verywell fail when same dnsname is given to login.
mailing list discussion
Dont use –portal “${ISCSIFS}”
option, use
iscsiadm -m node --targetname "${targetname}" --login