Table of Contents

Introduction

Scope

This How-To is intended for anyone who wants to install Gentoo Linux on an iSCSI target.

What is covered:
What is not covered:

Credits

The information in this document is based on personal experience and information found on the internet. The following pages proved to be a great resource of information:

Installation

Connect to your iSCSI disk

Method 1: use (x)ubuntu boot CD (safest)

(I've only tried this with xubuntu, but other ubuntu versions should work as well.)

This is by far the safest way to install, since you are booting from the live-cd. Why not just disconnect all your hard drives to avoid mistakes?

Install open-iscsi

 sudo aptitude install open-iscsi

Discover your iSCSI target

 sudo iscsi_discovery <SAN-IP>

One of the lines should be your target, for example:

 Logout of [[sid:|8, target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest, portal: 10.9.8.250,3260]]: successful

Connect to your target

 sudo iscsiadm -m node -p <SAN-IP> -T iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest -l

You should see something similar to:

 Logging in to [[iface:|default, target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest, portal: 10.9.8.250,3260]]
 Login to [[iface:|default, target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest, portal: 10.9.8.250,3260]]: successful

Method 2: use an existing/working gentoo system

Update kernel

Make sure your current kernel supports iSCSI: Roll a new kernel if necessary.

 Device Drivers  --->
  SCSI device support  --->
    [[*]] SCSI device support
      <*> SCSI disk support
    [[*]] SCSI low-level drivers  --->
      <M>   iSCSI Initiator over TCP/IP
    SCSI Transports  --->
      {M} iSCSI Transport Attributes
 Cryptographic options  --->
   [[*]] Cryptographic API
     <*> CRC32c CRC algorithm

Install open-iscsi

 emerge sys-block/open-iscsi

Start the iscsi service

 /etc/init.d/iscsid start

If you receive an error here that looks like this:

 * Checking open-iSCSI configuration ...
 * Loading iSCSI modules ...
 * Loading libiscsi: not found ...                                        [[|!! ]]

then you have to adapt your kernel:

 <M>   iSCSI Initiator over TCP/IP
 {M}   iSCSI Transport Attributes

It has to be compiled as module!

If you see an error like this:

 * Checking open-iSCSI configuration ...
 * Loading iSCSI modules ...
 * Loading libiscsi ...                                                   [[|ok ]]
 * Loading scsi_transport_iscsi ...                                       [[|ok ]]
 * Loading iscsi_tcp ...                                                  [[|ok ]]
 * Starting iscsid ...
 * Setting up iSCSI targets ...                                           [[|!! ]]
 iscsiadm: No records found!

Then that's fine, since we don't have any targets yet.

Discover your iscsi-target

Find the MAC address of your network card:

 ifconfig eth0 | grep HWaddr

You should see something like:

 eth0      Link encap:Ethernet  HWaddr 00:11:22:33:44:55

Discover the target:

 iscsiadm -m iface -I iface0 --op=new

You should see:

 New interface iface0 added

next:

 iscsiadm -m iface -I iface0 --op=update -n iface.hwaddress -v <MAC-ADDRESS>

You should see:

 iface0 updated.

next:

 iscsiadm -m discovery -t st -p <TARGET-IP> -P 1

One of the lines should be your target, for example:

 Target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest  
        Portal:	10.9.8.250:3260,1
                Iface Name: iface0

Connect to your iscsi-target

 iscsiadm -m node -T iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest -l

You should see something like:

 Logging in to [[iface:|iface0, target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest, portal: 10.9.8.250,3260]]
 Login to [[iface:|iface0, target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest, portal: 10.9.8.250,3260]]: successful

Chroot into your iSCSI disk

(Make sure you are root from now on)

Find out where the disk is attached to

(In this case the “disk” is a file filled with zeroes, so fdisk will complain there is no valid partition table. It does not matter.)

 fdisk -l

You should see something similar to:

 Disk /dev/sda: 4194 MB, 4194304000 bytes
 130 heads, 62 sectors/track, 1016 cylinders
 Units = cylinders of 8060 * 512 = 4126720 bytes
 Disk identifier: 0x00000000
 Disk /dev/sda doesn't contain a valid partition table

WARNING: Pay attention to what device your iscsi disk is attached, if you don't have other disks connected to your computer the device will probably be /dev/sda, if you do have other disks you have to find out what device your iscsi disk is connected to! You don't want to (re)format the wrong disk! I will refer to the disk as /dev/sdX.

Partition the iSCSI disk

(I've made only 1 partition to make it simple. You could add boot and swap if you want)

 fdisk /dev/sdX

Format the partition

Warning: read the bold warning in paragraph “Find out where the disk is attached to”

 mke2fs -j /dev/sdX1

Make a directory for your chroot environment

 mkdir /mnt/gentoo

Mount your iSCSI disk

 mount /dev/sdX1 /mnt/gentoo

Install Gentoo base system

From here on you have to download a stage3 just like you would in a normal installation. Unpack the stage in /mnt/gentoo.

Mount other filesystems

 mount -t proc none /mnt/gentoo/proc
 mount -o bind /dev /mnt/gentoo/dev

Copy resolver config

 cp -L /etc/resolv.conf /mnt/gentoo/etc/

Set timezone

 cp /usr/share/zoneinfo/Europe/Brussels /etc/localtime

Set hostname

 nano -w /etc/conf.d/hostname

Edit config files

 nano -w /etc/conf.d/clock
 nano -w /etc/rc.conf
 nano -w /etc/conf.d/rc
 nano -w /etc/conf.d/keymaps

Install system tools

 emerge app-admin/syslog-ng sys-process/vixie-cron
 rc-update add syslog-ng default ; rc-update add vixie-cron default

Set a root password

 passwd

Network configuration

IMPORTANT: Since your storage is connected over the network, do not allow init scripts to start or restart the network!

 nano -w /etc/conf.d/net
 config_eth0=( "noop" )

Fstab

Add your iSCSI disk in fstab like any normal hard disk:

 nano -w /etc/fstab
 /dev/sda1               /               ext3            noatime         0 1

Kernel configuration

Install the gentoo-sources

 emerge sys-kernel/gentoo-sources

Install genkernel

You need at least version 3.4.10.907 of genkernel so you may have to unmask it first

 ACCEPT_KEYWORDS="~yourach" emerge sys-kernel/genkernel

Compile the sources with genkernel

Don't forget to add –iscsi for iscsi support.

 genkernel --menuconfig --iscsi all

Enable iSCSI

I had trouble when CRC32c was compiled as M so i made it *.

Now iSCSI Initiator over TCP/IP has to be * not M! Now iSCSI Transport Attributes has to be * not M!

 Device Drivers  --->
  SCSI device support  --->
    [[*]] SCSI device support
      <*> SCSI disk support
    [[*]] SCSI low-level drivers  --->
      <*>   iSCSI Initiator over TCP/IP
    SCSI Transports  --->
      {*} iSCSI Transport Attributes
 Cryptographic options  --->
   [[*]] Cryptographic API
     <*> CRC32c CRC algorithm

Enable networking

 Networking support  --->
    Networking options --->
      <*> Packet socket
      <*> Unix domain sockets
      [[*]] TCP/IP networking
      [[*]]   IP: multicasting
      [[*]]   IP: kernel level autoconfiguration
      [[*]]     IP: DHCP support

Enable your network card

Enable your network card as BUILT IN *, not module!!

 Device Drivers  --->
   [[*]] Network device support  --->
   [[?]]   Ethernet (10 or 100Mbit)  --->
   [[?]]   Ethernet (1000 Mbit)  --->
   [[?]]   Ethernet (10000 Mbit)  --->

Install Bootloader

Install GRUB

 emerge sys-boot/grub

Configure GRUB

 grub

WARNING, WARNING, WARNING: make sure you enter the right disk!! if you are running the ubuntu-cd and don't have any real harddisks connected to your computer then hdx=hd0, if you are installing from a real hard disk then hdX=hd1 or hd2 or so on!!!

 root (hdX,0)
 setup (hdX)
 quit

Edit GRUB configuration

You have to set at least these options for iSCSI to work: ip=dhcp, iscsi_target=, iscsi_address=, iscsi_initiatorname

 nano -w /boot/grub/grub.conf
 default 0
 timeout 10
 title Gentoo Linux 2.6.31-r6 iSCSI
 root (hd0,0)
 kernel /boot/kernel-genkernel-x86_64-2.6.31-gentoo-r6 ip=dhcp root=/dev/ram0 init=/linuxrc ramdisk=8192 real_root=/dev/sda1 iscsi_target=iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest iscsi_address=10.9.8.250 iscsi_initiatorname=iqn.1993-08.org.gentooclient5:01:22b2ed5d3ccc
 initrd /boot/initramfs-genkernel-x86_64-2.6.31-gentoo-r6

Leave chroot

 cd ~
 exit

If you don't have a boot partition like me, then you can leave out /mnt/gentoo/boot

 umount /mnt/gentoo/dev /mnt/gentoo/proc /mnt/gentoo/boot /mnt/gentoo

Troubleshooting

Made a mistake in the base system

You can connect to your iSCSI disk again with one of the methods above, chroot and fix the problem.

System won't boot and drops you in a shell

Enter the shell and try to figure out what went wrong.

 ifconfig eth0
 dmesg

If demsg complains about crc32 you probably compiled it as M not *.

 iscsistart --help

Gentoo boots partially but then complains about a read-only filesystem

You probably didn't edit /etc/fstab right

 mount -o remount rw /
 nano -w /etc/fstab
 sync
 reboot

Kernel boots but hangs/crashes when it tries to bring up eth0

You cannot allow the system to start or restart the network connection, this will sever your iscsi connection!

 mount  -o remount rw /
 nano -w /etc/conf.d/net
 config_eth0=( "noop" )
 sync
 reboot

Boot fails when adding other storage to computer

If you add a hard disk/USB drive/USB stick/etc. to the computer, it will probably take the /dev/sda spot and your iSCSI disk will be moved to /dev/sdb or /dev/sdc, etc. Using filesystem labels will help in this situation.

Gentoo will not shut down properly

Because an iscsi mount is not recognized as netmount (like nfs) by Gentoo, the system will try to shutdown the network interface during normal shutdown/reboot procedure. This will obviously break the iscsi connection and let the system hang.

To avoid this the following workaround can be used add (where eth0 is the interface being used for the iscsi connection):

/etc/init.d/net.eth0 zap

to /etc/init.d/local.start (or the start section of /etc/init.d/local depending on your baselayout version). Doing so will make Gentoo think eth0 is already down, so it wont try to shut it down on halt. All other init-scripts that depend on net will still be invoked to shut them down cleanly, just eth0 itself wont be stopped.