Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
soc:2009:pravin:journal:bkodebianiscsi [2009/07/18 11:06] less1 |
soc:2009:pravin:journal:bkodebianiscsi [2009/09/25 19:37] (current) less1 |
||
---|---|---|---|
Line 113: | Line 113: | ||
I need to add all the tools, configuration files and kernel modules to initramfs so that I can use them from there to connect. | I need to add all the tools, configuration files and kernel modules to initramfs so that I can use them from there to connect. | ||
- | * ** Step 1 **, __Kernel Modules__ on checking the contents of ''/etc/init.d/open-iscsi'' it seems that following kernel modules are to be inserted | + | * ** Step 1 **, __Kernel Modules__ : on checking the contents of ''/etc/init.d/open-iscsi'' it seems that following kernel modules are to be inserted |
<code> | <code> | ||
modprobe iscsi_tcp | modprobe iscsi_tcp | ||
Line 134: | Line 134: | ||
</code> | </code> | ||
+ | ===== Testing new initramfs ===== | ||
+ | The HTTPFS code has been commented out so that initramfs will give me rescue shell, which can be used to start the iscsi initiator.\\ | ||
+ | Following commands were executed once the rescue shell is provided. | ||
+ | <code> | ||
+ | modprobe iscsi_tcp | ||
+ | modprobe ib_iser | ||
+ | mkdir /var/run | ||
+ | /sbin/iscsid | ||
+ | iscsiadm -m discovery -t st -p 192.168.0.1 | ||
+ | </code> | ||
+ | and no node is shown :(\\ | ||
+ | I tried to run iscsid in debug mode with following command | ||
+ | <code> | ||
+ | /sbin/iscsid -d 8 -f | ||
+ | iscsid : sysfs_init: sysfs_path='/sys' | ||
+ | [...] | ||
+ | [...] | ||
+ | iscsid: reaped pid 994, reap_count now 0 | ||
+ | </code> | ||
+ | and also ran ''iscsiadm'' in debug mode with following output | ||
+ | <code> | ||
+ | (initramfs) iscsiadm -d 8 -m discovery -t st -p 192.168.0.1 | ||
+ | iscsiadm: ip 192.168.0.1, port 3260, tgpt -1 | ||
+ | iscsiadm: Max file limits 1024 1024 | ||
+ | </code> | ||
+ | This output is definitely very less compare to the output of same command executed on my laptop.\\ | ||
+ | **Note:** __This was executed over qeum.__ I am not sure if qemu has created any problem. | ||
+ | |||
+ | ===== Debugging the problem ===== | ||
+ | To make sure debien in virtual environment can use iscsi with the tools, and configuration files used inside initramfs\\ | ||
+ | - I booted debian live CD with virtualbox | ||
+ | - inserted kernel modules just as above | ||
+ | - Copied configuration files into live cd. | ||
+ | - Copied tools into live cd | ||
+ | - Executed above commands from inside live CD | ||
+ | - The ''iscsiadm -m discovery'' command did gave output. | ||
+ | <code> | ||
+ | # iscsiadm -m discovery -t st -p 192.168.0.1 | ||
+ | 192.168.0.1:3260,1 iqn.2001-04.com.example:storage.lun1 | ||
+ | </code> | ||
+ | |||
+ | |||
+ | So, the tools do work. but may be they are able to use other files, tools and modules which are not present in initrd.\\ | ||
+ | How to find those dependencies? | ||
+ | |||
+ | ===== Debugging v2 ===== | ||
+ | Finally, added the ''strace'' to initramfs, and now trying find what is the problem because of which iscsiadm is dying\\ | ||
+ | <code> | ||
+ | socket(PF_FILE, SOCK_STREAM, 0) = 3 | ||
+ | connect(3, {sa_family=AF_FILE, path=@"ISCSIADM_ABSTRACT_NAMESPACE"...}, 110) = 0 | ||
+ | write(3, "\n\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 5244) = -1 EPIPE (Broken pipe) | ||
+ | </code> | ||
+ | This is what killing the iscsiadm imatuarly. | ||
+ | |||
+ | ==== Checking the code of open-iscsi ==== | ||
+ | With help of stephana, the likely problem is located at following location | ||
+ | <code> | ||
+ | stefanha: http://git.kernel.org/?p=linux/kernel/git/mnc/open-iscsi.git;a=blob;f=usr/mgmt_ipc.c;h=e784bcac019a84522e62beff1b49b1ca62e37f5a;hb=HEAD | ||
+ | (07/19/2009 04:34:21 PM) stefanha: less1: Perhaps you're hitting this: | ||
+ | (07/19/2009 04:34:27 PM) stefanha: http://git.kernel.org/?p=linux/kernel/git/mnc/open-iscsi.git;a=blob;f=usr/mgmt_ipc.c;h=e784bcac019a84522e62beff1b49b1ca62e37f5a;hb=HEAD#l549 | ||
+ | (07/19/2009 04:34:31 PM) stefanha: if (!mgmt_peeruser(fd, user) || strncmp(user, "root", PEERUSER_MAX)) { | ||
+ | (07/19/2009 04:34:43 PM) stefanha: write(2) will return EPIPE if the UNIX domain socket was closed. | ||
+ | (07/19/2009 04:35:17 PM) stefanha: mgmt_peeruser() is defined here: http://git.kernel.org/?p=linux/kernel/git/mnc/open-iscsi.git;a=blob;f=usr/mgmt_ipc.c;h=e784bcac019a84522e62beff1b49b1ca62e37f5a;hb=HEAD#l342 | ||
+ | (07/19/2009 04:38:15 PM) less1: stefanha: is it checking if I have root privilages? I am not sure if I understand which peeruser credential is it refering. | ||
+ | (07/19/2009 04:40:08 PM) stefanha: Your user name must be "root". | ||
+ | (07/19/2009 04:40:25 PM) stefanha: Otherwise iscsid will not accept commands from the UNIX domain socket. | ||
+ | </code> | ||
+ | === Possible solution : hacking open-iscsi === | ||
+ | | ||
+ | modified the ''iscsiadm'' code so that, it will not fail when username is not ''root''.\\ | ||
+ | With is modification, the discovery part is working, but now there is error at login part. Following is the error. | ||
+ | <code> | ||
+ | connection2:0: Could not create connection due to crc32c loading error. Make sure the crc32c module is build as module into the kernel. | ||
+ | </code> | ||
+ | The first attempt will be to download the kernel module from Live CD of debian after booting it from CD.\\ | ||
+ | --- Doing it | ||
+ | |||
+ | ===== Working Solution ===== | ||
+ | There was need for two extra kernel modules | ||
+ | - libcrc32c.ko | ||
+ | - crc32c.ko | ||
+ | Fetched these modules from live-CD and added them.\\ | ||
+ | There as also need for small sleep after ''iscsiadm --login'' as ''/dev/sda'' was taking little time to appear.\\ | ||
+ | This was handled by adding ''sleep 2''\\ | ||
+ | |||
+ | Two new parameters were introduced | ||
+ | - ''iscsifs=<IP:PORT>'' iscsi server IP and port number | ||
+ | - ''target=<targetname>'' the iscsi target name of the disk (entire name is not needed, some part which can uniquely identify apart from other disks exported at same machine) | ||
+ | |||
+ | ===== Remote ISCSI ===== | ||
+ | |||
+ | Following are the errors that I get when connecting to remote iscsi | ||
+ | <code> | ||
+ | Connection1:0: ping timeout of 5 secs expired, last rx 307737, last ping 308987, now 310237 | ||
+ | Connection1:0: detected conn error (1011) | ||
+ | SQUASHFS error : Unable to read page, block 0x1b392 | ||
+ | Bus error | ||
+ | </code> | ||
+ | Most of these errors are related to timing, and I believe that | ||
+ | these errors are there because of delays in accessing required data. | ||
+ | |||
+ | ===== Target behaviour ===== | ||
+ | sometimes, you may get following error when logging in with iscsiadm | ||
+ | <code> | ||
+ | iscsiadm -m node --targetname "${targetname}" --portal "${ISCSIFS}" --login | ||
+ | records not found!. | ||
+ | </code> | ||
+ | this is because the ${ISCSIFS} is not IP but dnsname. | ||
+ | Dnsname may work with ''iscsiadm -m discovery -t st -p "${ISCSIFS}" '' | ||
+ | but it may verywell fail when same dnsname is given to login. | ||
+ | [[http://groups.google.com/group/open-iscsi/browse_thread/thread/c556508505ecc461|mailing list discussion]] | ||
+ | |||
+ | ==== Solution ==== | ||
+ | Dont use ''--portal "${ISCSIFS}"'' option, use | ||
+ | <code> | ||
+ | iscsiadm -m node --targetname "${targetname}" --login | ||
+ | </code> |