Table of Contents
Introduction
Scope
This How-To is intended for anyone who wants to install Gentoo Linux on an iSCSI target.
What is covered:
- Connecting to the iSCSI target
- Installing a generic base system
- Installing the kernel
- Installing the bootloader
What is not covered:
- Understanding and configuring a network environment
Credits
The information in this document is based on personal experience and information found on the internet. The following pages proved to be a great resource of information:
- http://etherboot.org/wiki/sanboot/ubuntu_iscsi (installing ubuntu on iscsi)
- http://www.gentoo-wiki.info/ISCSI (examples how to use open-iscsi on gentoo)
- http://www.mail-archive.com/open-iscsi@googlegroups.com/msg00736.html (a bug that kept me busy for a while)
Installation
Connect to your iSCSI disk
Method 1: use (x)ubuntu boot CD (safest)
(I've only tried this with xubuntu, but other ubuntu versions should work as well.)
This is by far the safest way to install, since you are booting from the live-cd. Why not just disconnect all your hard drives to avoid mistakes?
Install open-iscsi
sudo aptitude install open-iscsi
Discover your iSCSI target
sudo iscsi_discovery <SAN-IP>
One of the lines should be your target, for example:
Logout of [[sid:|8, target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest, portal: 10.9.8.250,3260]]: successful
Connect to your target
sudo iscsiadm -m node -p <SAN-IP> -T iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest -l
You should see something similar to:
Logging in to [[iface:|default, target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest, portal: 10.9.8.250,3260]] Login to [[iface:|default, target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest, portal: 10.9.8.250,3260]]: successful
Method 2: use an existing/working gentoo system
Update kernel
Make sure your current kernel supports iSCSI: Roll a new kernel if necessary.
- I had trouble when CRC32c was compiled as a module, but it worked when built-in.
- Open-iscsi will not start when
iSCSI Initiator over TCP/IP
is built-in, so you have to make it a module for now. - Open-iscsi will not start when
iSCSI Transport Attributes
is built-in, so you have to make it a module for now.
Device Drivers ---> SCSI device support ---> [[*]] SCSI device support <*> SCSI disk support [[*]] SCSI low-level drivers ---> <M> iSCSI Initiator over TCP/IP SCSI Transports ---> {M} iSCSI Transport Attributes Cryptographic options ---> [[*]] Cryptographic API <*> CRC32c CRC algorithm
Install open-iscsi
emerge sys-block/open-iscsi
Start the iscsi service
/etc/init.d/iscsid start
If you receive an error here that looks like this:
* Checking open-iSCSI configuration ... * Loading iSCSI modules ... * Loading libiscsi: not found ... [[|!! ]]
then you have to adapt your kernel:
<M> iSCSI Initiator over TCP/IP {M} iSCSI Transport Attributes
It has to be compiled as module!
If you see an error like this:
* Checking open-iSCSI configuration ... * Loading iSCSI modules ... * Loading libiscsi ... [[|ok ]] * Loading scsi_transport_iscsi ... [[|ok ]] * Loading iscsi_tcp ... [[|ok ]] * Starting iscsid ... * Setting up iSCSI targets ... [[|!! ]] iscsiadm: No records found!
Then that's fine, since we don't have any targets yet.
Discover your iscsi-target
Find the MAC address of your network card:
ifconfig eth0 | grep HWaddr
You should see something like:
eth0 Link encap:Ethernet HWaddr 00:11:22:33:44:55
Discover the target:
iscsiadm -m iface -I iface0 --op=new
You should see:
New interface iface0 added
next:
iscsiadm -m iface -I iface0 --op=update -n iface.hwaddress -v <MAC-ADDRESS>
You should see:
iface0 updated.
next:
iscsiadm -m discovery -t st -p <TARGET-IP> -P 1
One of the lines should be your target, for example:
Target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest Portal: 10.9.8.250:3260,1 Iface Name: iface0
Connect to your iscsi-target
iscsiadm -m node -T iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest -l
You should see something like:
Logging in to [[iface:|iface0, target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest, portal: 10.9.8.250,3260]] Login to [[iface:|iface0, target: iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest, portal: 10.9.8.250,3260]]: successful
Chroot into your iSCSI disk
(Make sure you are root from now on)
Find out where the disk is attached to
(In this case the “disk” is a file filled with zeroes, so fdisk will complain there is no valid partition table. It does not matter.)
fdisk -l
You should see something similar to:
Disk /dev/sda: 4194 MB, 4194304000 bytes 130 heads, 62 sectors/track, 1016 cylinders Units = cylinders of 8060 * 512 = 4126720 bytes Disk identifier: 0x00000000
Disk /dev/sda doesn't contain a valid partition table
WARNING: Pay attention to what device your iscsi disk is attached, if you don't have other disks connected to your computer the device will probably be /dev/sda, if you do have other disks you have to find out what device your iscsi disk is connected to! You don't want to (re)format the wrong disk! I will refer to the disk as /dev/sdX.
Partition the iSCSI disk
(I've made only 1 partition to make it simple. You could add boot and swap if you want)
fdisk /dev/sdX
Format the partition
Warning: read the bold warning in paragraph “Find out where the disk is attached to”
mke2fs -j /dev/sdX1
Make a directory for your chroot environment
mkdir /mnt/gentoo
Mount your iSCSI disk
mount /dev/sdX1 /mnt/gentoo
Install Gentoo base system
From here on you have to download a stage3 just like you would in a normal installation. Unpack the stage in /mnt/gentoo.
Mount other filesystems
mount -t proc none /mnt/gentoo/proc mount -o bind /dev /mnt/gentoo/dev
Copy resolver config
cp -L /etc/resolv.conf /mnt/gentoo/etc/
Set timezone
cp /usr/share/zoneinfo/Europe/Brussels /etc/localtime
Set hostname
nano -w /etc/conf.d/hostname
Edit config files
nano -w /etc/conf.d/clock nano -w /etc/rc.conf nano -w /etc/conf.d/rc nano -w /etc/conf.d/keymaps
Install system tools
emerge app-admin/syslog-ng sys-process/vixie-cron rc-update add syslog-ng default ; rc-update add vixie-cron default
Set a root password
passwd
Network configuration
IMPORTANT: Since your storage is connected over the network, do not allow init scripts to start or restart the network!
nano -w /etc/conf.d/net
config_eth0=( "noop" )
Fstab
Add your iSCSI disk in fstab like any normal hard disk:
nano -w /etc/fstab
/dev/sda1 / ext3 noatime 0 1
Kernel configuration
Install the gentoo-sources
emerge sys-kernel/gentoo-sources
Install genkernel
You need at least version 3.4.10.907 of genkernel so you may have to unmask it first
ACCEPT_KEYWORDS="~yourach" emerge sys-kernel/genkernel
Compile the sources with genkernel
Don't forget to add –iscsi for iscsi support.
genkernel --menuconfig --iscsi all
Enable iSCSI
I had trouble when CRC32c was compiled as M so i made it *.
Now iSCSI Initiator over TCP/IP has to be * not M! Now iSCSI Transport Attributes has to be * not M!
Device Drivers ---> SCSI device support ---> [[*]] SCSI device support <*> SCSI disk support [[*]] SCSI low-level drivers ---> <*> iSCSI Initiator over TCP/IP SCSI Transports ---> {*} iSCSI Transport Attributes Cryptographic options ---> [[*]] Cryptographic API <*> CRC32c CRC algorithm
Enable networking
Networking support ---> Networking options ---> <*> Packet socket <*> Unix domain sockets [[*]] TCP/IP networking [[*]] IP: multicasting [[*]] IP: kernel level autoconfiguration [[*]] IP: DHCP support
Enable your network card
Enable your network card as BUILT IN *, not module!!
Device Drivers ---> [[*]] Network device support ---> [[?]] Ethernet (10 or 100Mbit) ---> [[?]] Ethernet (1000 Mbit) ---> [[?]] Ethernet (10000 Mbit) --->
Install Bootloader
Install GRUB
emerge sys-boot/grub
Configure GRUB
grub
WARNING, WARNING, WARNING: make sure you enter the right disk!! if you are running the ubuntu-cd and don't have any real harddisks connected to your computer then hdx=hd0, if you are installing from a real hard disk then hdX=hd1 or hd2 or so on!!!
root (hdX,0) setup (hdX) quit
Edit GRUB configuration
You have to set at least these options for iSCSI to work: ip=dhcp, iscsi_target=, iscsi_address=, iscsi_initiatorname
nano -w /boot/grub/grub.conf
default 0 timeout 10
title Gentoo Linux 2.6.31-r6 iSCSI root (hd0,0) kernel /boot/kernel-genkernel-x86_64-2.6.31-gentoo-r6 ip=dhcp root=/dev/ram0 init=/linuxrc ramdisk=8192 real_root=/dev/sda1 iscsi_target=iqn.1986-03.com.sun:02:86afa64e-9baf-451c-84bb-826ea71300db.gentootest iscsi_address=10.9.8.250 iscsi_initiatorname=iqn.1993-08.org.gentooclient5:01:22b2ed5d3ccc initrd /boot/initramfs-genkernel-x86_64-2.6.31-gentoo-r6
Leave chroot
cd ~ exit
If you don't have a boot partition like me, then you can leave out /mnt/gentoo/boot
umount /mnt/gentoo/dev /mnt/gentoo/proc /mnt/gentoo/boot /mnt/gentoo
Troubleshooting
Made a mistake in the base system
You can connect to your iSCSI disk again with one of the methods above, chroot and fix the problem.
System won't boot and drops you in a shell
Enter the shell and try to figure out what went wrong.
- Did the network interface come up with an ip-address?
ifconfig eth0
- You may have forgotten to enable
IP: kernel level autoconfiguration
and/or DHCP in the kernel - You may have forgotten to enable your network card in the kernel (* not M)
- Does dmesg show any warnings?
dmesg
If demsg complains about crc32 you probably compiled it as M not *.
- Can you connect to the target manually?
iscsistart --help
Gentoo boots partially but then complains about a read-only filesystem
You probably didn't edit /etc/fstab right
mount -o remount rw / nano -w /etc/fstab sync reboot
Kernel boots but hangs/crashes when it tries to bring up eth0
You cannot allow the system to start or restart the network connection, this will sever your iscsi connection!
mount -o remount rw / nano -w /etc/conf.d/net
config_eth0=( "noop" )
sync reboot
Boot fails when adding other storage to computer
If you add a hard disk/USB drive/USB stick/etc. to the computer, it will probably take the /dev/sda spot and your iSCSI disk will be moved to /dev/sdb or /dev/sdc, etc. Using filesystem labels will help in this situation.
Gentoo will not shut down properly
Because an iscsi mount is not recognized as netmount (like nfs) by Gentoo, the system will try to shutdown the network interface during normal shutdown/reboot procedure. This will obviously break the iscsi connection and let the system hang.
To avoid this the following workaround can be used add (where eth0 is the interface being used for the iscsi connection):
/etc/init.d/net.eth0 zap
to /etc/init.d/local.start (or the start section of /etc/init.d/local depending on your baselayout version). Doing so will make Gentoo think eth0 is already down, so it wont try to shut it down on halt. All other init-scripts that depend on net will still be invoked to shut them down cleanly, just eth0 itself wont be stopped.