Getting iSCSI booting to work with Ubuntu including logon and ibft support!

Tested on the following releases:

• Lucid Lynx 10.04
• Maverick Meerkat 10.10

• iSCSI Qualified Name: iqn.2010-09.org.test:diskimg
• Server ip: 192.168.1.1
• Client ip: 192.168.1.2
• iSCSI username: user
• iSCSI password: password
• Client disk image: /mnt/disk-images/disk.img

On the client side, install a standard Ubuntu Desktop on a physical or virtual machine. If you use a virtual machine, make the sure disk image is in 'raw' format, meaning no headers or such on the image. If it is a valid raw image, you should be able to see the partition table with, for example “fdisk -l disk.img” or even the “disktype” command, like so:

$ disktype disk.img 

--- disk.img
Regular file, size 6 GiB (6442450944 bytes)
DOS/MBR partition map
Partition 1: 5.858 GiB (6290407424 bytes, 12285952 sectors from 2048, bootable)
  Type 0x83 (Linux)
  Ext3 file system
    UUID 745D45C4-38FA-4B7C-85A3-71D37AE23AF3 (DCE, v4)
    Volume size 5.858 GiB (6290407424 bytes, 1535744 blocks of 4 KiB)
Partition 2: 144 MiB (150994944 bytes, 294912 sectors from 12288000)
  Type 0x82 (Linux swap / Solaris)
  Linux swap, version 2, subversion 1, 4 KiB pages, little-endian
    Swap size 144.0 MiB (150986752 bytes, 36862 pages of 4 KiB)

#!/bin/sh
PREREQ=""
prereqs()
{
     echo "$PREREQ"
}

case $1 in
prereqs)
     prereqs
     exit 0
     ;;
esac

# Begin real processing below this line
if [ ! -x /sbin/iscsistart ]; then
   exit 0
fi
. /usr/share/initramfs-tools/hook-functions
copy_exec /sbin/iscsistart /sbin

# Disable stuff that can affect the network interface
if [ -x /usr/lib/pm-utils/power.d/disable_wol ]; then
   chmod -x /usr/lib/pm-utils/power.d/disable_wol         # affects usb network adapters
fi
if [ -x /sbin/iscsid ]; then
   chmod -x /sbin/iscsid                                  # iscsid will interfere by re-discovering
fi
if [ -e /lib/udev/rules.d/75-persistent-net-generator.rules ]; then
   rm /etc/udev/rules.d/70-persistent-net.rules           # prevent renaming of the network interfaces
   rm /lib/udev/rules.d/75-persistent-net-generator.rules # prevent re-creating the above rule
fi

# pcmcia/cardbus networking
copy_exec /lib/udev/pcmcia-socket-startup
copy_exec /lib/udev/pcmcia-check-broken-cis
copy_exec /lib/udev/rules.d/85-pcmcia.rules
copy_exec /etc/pcmcia/config.opts
copy_modules_dir kernel/drivers/pcmcia
copy_modules_dir kernel/drivers/net/pcmcia
for x in pcmcia_core yenta_socket pcmcia; do
   force_load ${x}
done

# usb networking
copy_modules_dir kernel/drivers/net/usb

# iscsi module dependencies
for x in scsi_transport_iscsi libiscsi libiscsi_tcp iscsi_tcp \
   crc32c iscsi_ibft; do
   force_load ${x}
done

#!/bin/sh

# Check if are in the initramfs environment
if [ "$BOOT" = "local" ]; then
   # Get any iscsi command line parameters - cmdline parsing template taken from init script in initramfs
   for x in $(cat /proc/cmdline); do
      case $x in
            iscsi_ip=*)
               iscsi_ip=${x#iscsi_ip=}
               ;;
            iscsi_iqn=*)
               iscsi_iqn=${x#iscsi_iqn=}
               ;;
            iscsi_user=*)
               iscsi_user=${x#iscsi_user=}
               ;;
            iscsi_pw=*)
               iscsi_pw=${x#iscsi_pw=}
               ;;
      esac
   done # end of cmdline parsing
   if [ -n "${iscsi_ip}" ]; then
      iscsi_name='iqn.2000-09.org.local:UNKNOWN'
   fi

   # Check if an iscsi environment exists and start up the networking environment
   if [ -e /sys/firmware/ibft/target0/target-name -o -n "${iscsi_ip}" ]; then
      x=5
      # Wait up to x seconds for a network interface to pop up
      echo -n "Searching for network cards..."
      until ifconfig -a|grep -q Ethernet; do
         sleep 1
         echo -n .
         x=$(($x - 1))
         if [ $x = 0 ]; then
            echo "No network cards detected!"
            exit 0
         fi
      done # end of until
      echo "Searching for an dhcp server on all network interfaces..."
      CNT=5
      until ifconfig ${netdev}|grep -q "inet addr"; do
         for netdev in `ifconfig -a|grep Ethernet|cut -d' ' -f1`; do # Do a round-robin search for dhcp servers
            #ip link set ${netdev} mtu 2500                          # Optional: increase mtu for performance
            ip link set ${netdev} up                                 # try to bring up the interface
            ipconfig -t 2 -c dhcp -d ${netdev}                       # Get an IP
            if ifconfig -a|grep -q "inet addr"; then break 2; fi     # If we have got an address, stop searching.
         done # end of netdev probing
         CNT=$((${CNT} - 1))
         if [ ${CNT} = 0 ]; then
            echo "No dhcp servers found!"
            exit 0
         fi
         echo "Tries left: ${CNT}"
      done # end of until
      if [ -e /sys/firmware/ibft/target0/target-name ]; then
         #iscsistart -b  # This should get the ibft parameters but it does not yet.
         # Lets do the following instead
         iscsistart \
         -i `cat /sys/firmware/ibft/initiator/initiator-name` \
         -t `cat /sys/firmware/ibft/target0/target-name` \
         -a `cat /sys/firmware/ibft/target0/ip-addr` \
         -u `cat /sys/firmware/ibft/target0/chap-name` \
         -w `cat /sys/firmware/ibft/target0/chap-secret` \
         -g 1
      else
         iscsistart -i ${iscsi_name} -t ${iscsi_iqn} -a ${iscsi_ip} -u ${iscsi_user} -w ${iscsi_pw} -g 1
      fi # end of iscsi firmware check
   fi # end of iscsi env check
fi  # end of initramfs check

Make sure that eth0 is set to manual in /etc/network/interfaces so that Network Manager does not try to reconfigure the interface:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
#auto eth0
iface eth0 inet manual
iface eth1 inet manual
iface eth2 inet manual

Then do these commands at the terminal:

sudo -s                                               # Become root, if it asks for password, use the users password
chmod +x /etc/initramfs-tools/hooks/iscsi             # Make your hook script executable
chmod +x /etc/initramfs-tools/scripts/local-top/iscsi # Make your script executable
apt-get install open-iscsi open-iscsi-utils           # Make sure you have the iscsi tools installed
update-initramfs -u                                   # Update initramfs with your changes, 
                                                      # the changes will also take effect even with upgrades

You are now finished with editing the client. Shutdown the client now.

You could just copy the image as is, but it is even smarter to make a sparse file. That way only actual data is copied and not the empty sectors. You can do this under linux with the cp command:

cp --sparse=always disk.img /media/server-data/disk-images/disk.img

If you installed to a physical machine, boot up the machine with a live linux cd, become root and do this command:

cp --sparse=always /dev/sda /media/server-data/disk-images/disk.img

I am assuming that you are using either Ubuntu or Debian linux on the server. Make sure you have a dhcp and a tftp server installed and that you have followed the directions for setting up gpxe on the dhcp server from etherboot.org. Install the “iscsitarget” package using this command:

sudo apt-get install iscsitarget

Assuming /mnt/disk-images/disk.img is your client image, create the /etc/ietf.conf file with the following contents:

Target iqn.2010-09.org.test:diskimg
        IncomingUser user password
        OutgoingUser
        Lun 0 Path=/mnt/disk-images/disk.img,IOMode=wb,Type=fileio

Make sure iscsitarget can start at boot by using this value in /etc/default/iscsitarget

ISCSITARGET_ENABLE=true

Make the gpxe script /tftpboot/iscsi.gpxe with these contents:

dhcp net0
set username user
set password password
sanboot iscsi:192.168.1.1::::iqn.2010-09.org.test:diskimg

Edit your dhcpd.conf file and make sure the following is in your gpxe section of the config file:

filename=iscsi.gpxe

That's it! Start your server by running “sudo /etc/init.d/iscsitarget restart” and boot your pxe client and see what happens.

In order to work around the kernel oops when iscsi image files are residing on a btrfs file system, change the /etc/ietf.conf file as follows:

Target iqn.2010-09.org.test:diskimg
        IncomingUser user password
        OutgoingUser
        Lun 0 Path=/dev/loop0,IOMode=wb,Type=blockio

…and create a loop device that points to the image:

losetup /dev/loop0 /mnt/disk-images/disk.img

Remember to restart the iscsitarget service.

• If you don't install any proprietary video drivers, then what you got here is an image that will boot from either iSCSI, USB, or any internal/external disk on any machine.
• Since Ubuntu is Debian based, these steps should work with Debian too but I haven't tried it myself
• If you can't use the ibft interface or need to load the kernel and initrd from another source than iscsi, then you can pass the following kernel command line arguments to get it to connect to iscsi:

      iscsi_iqn=  The iscsi qualifier name
      iscsi_ip=   The target server's ip address
      iscsi_user= The username to login to the target server
      iscsi_pw=   The password to login to the target server

• pcmcia support is now complete. All linux supported pcmcia and cardbus network cards should work
• You cannot create sparse images if you copy to a windows or samba network share, use nfs/ssh/nc (netcat) instead. Here is an example with nc (netcat):

Client side:

cat /dev/sda | nc -l -p 1234

Server side:

nc <client-ip> 1234 | cp --sparse=always /dev/stdin disk.img

— Quinn Plattel 2010/10/14

• This method disables iscsid. I have detected (probably the same) problem with it under Maverick (2.6.35 kernel), but not under Lucid (2.6.32 kernel). iscsid fails to reestablish the iscsi session and fails with response status code 0201. The problem is: I need iscsid to attach more iscsi shares… any idea how to solve that issue?

— cinquero

— Trip-G 2011/1/19

• Quinn - Simply add another iscsistart to the file /etc/initramfs-tools/scripts/local-top/iscsi
• if your second target is on the same server all you need is the iqn
• like this:

MYEXTRANAME="iqn.numbers.com.whatever:name:something.videostorage.mac"
        iscsistart \
        -i `cat /sys/firmware/ibft/initiator/initiator-name` \
        -t ${MYEXTRANAME} \
        -a `cat /sys/firmware/ibft/target0/ip-addr` \
        -g 1

if you have relevant username password stuff you can add that too. I added this after if [ -e /sys/firmware/ibft/target0/target-name ]; then and before “else” but you should be able to tag it on the end if you want.

— Trip-G