iscsi setup guide for noobs

We had to deal with open-iscsi in Ubuntu Xenial 16.04. Basically, we wanted to provide mount points for cinder in our Openstack Ocata cluster.
So here is how pain and suffer begun because nowhere a suitable "step by step workflow" of setup described to understand iscsi mechanism. Even no article describes how things move on the timeline in correct order.
I hope it helps to you to understand basic flow of open-iscsi and you can achieve your goal with a less pain then us :)

  • Please read everything at first and then START to do your operations later!!!

Hint:

You should know this info below, before starting to setup iscsi properly.

  • iscsi user name
  • iscsi password for target(s)
  • iscsi host ip address
  • iscsi host port number

Pieces of open-iscsi

  • First install open-iscsi. Remember that, once you've installed a service, it tries to start at the very beginning. Check if it is running service open-iscsi status
  • there is a /etc/iscsi/iscsid.conf file for config settings.
  • there is a /etc/iscsi/nodes directory to hold scanned iscsi targets information.
  • you'll have iscsiadm tool to manage all the stuff in console.

How things happen when you work with iscsi in correct order?

  • Before you go any further, stop the service. service open-iscsi stop

  • You have to check iscsid.conf at first. If you have more then one targets, please comment out all password and user fields. If you have only one target to connect, please fill the necessary fields according to your authentication scheme in this file.

node.session.auth.authmethod = CHAP  
node.session.auth.username = your_user  
node.session.auth.password = your_pass

node.session.auth.username_in = your_user  
node.session.auth.password_in = your_pass

discovery.sendtargets.auth.username = your_user  
discovery.sendtargets.auth.password = your_pass

Note:

if you don't know what is multipath, do not mess with it. Don't play with the parameters that you have no idea about!

  • Now it is time to check what do you have under /etc/iscsi/nodes/ directory. Since there is no targets defined in conf file, there shouldn't be any target information directory under nodes dir.
    Run ls /etc/iscsi/nodes/ to see.

Note:

You should never directly edit the files created under nodes dir. Always use iscsiadmcommand to manage all iscsi related operations.

In case you've found any target info under nodes dir, you have to delete them all by running this command for the clean install. Be carefull if there is an existing iscsi installation runs and having alive mounts on the server.

Run iscsiadm -m node --targetname "exact_dir_name_under_nodes_dir" --portal "your_iscsi_server_ip:your_iscsi_port_3260_in_general" --op=delete
to delete discovered node.

Note:

Check how many targets you have under the nodes dir and run the same command for for all of them.

  • Ok now we are ready to discover nodes properly. You have to run this command to discover targets.

Run iscsiadm -m discovery -t sendtargets -p your_iscsi_server_ip
to see what is discovered as target(s). When this command completed you'll have target(s) listed in console and these targets going to be added to the nodes dir by checking the params in iscsid.conf.

Important Note:

Trick is every time you play with iscsid.conf file, you have to reiterate all this cycle again! Delete nodes, discover them, add user and password with iscsiadm command and check if connected device blocks are stays intact in same device name or not.

Setup is quite easy for just one target. In this guide, we will show, how you could manage more then one target.

At this point service open-iscsi is not running but you have discovered targets. Now we have to add auth method, user and password per node. Find the target names under nodes dir and apply these commands for every target listed there.

iscsiadm -m node --targetname "iqn.xxx.target_name" --portal "172.16.33.14:3260" --op=update --name node.session.auth.authmethod --value=your_auth_method_in_general_CHAP

iscsiadm -m node --targetname "iqn.xxx.target_name" --portal "172.16.33.14:3260" --op=update --name node.session.auth.username --value=user_name

iscsiadm -m node --targetname "iqn.xxx.user_name" --portal "172.16.33.14:3260" --op=update --name node.session.auth.password --value=your_password  

Nothing happens in console but actually this information added to the files under nodes dir.

It is time to login to discovered targets. To login all targets at once run iscsiadm -m node --login

To login specific target, run iscsiadm -m node --targetname "iqn.xxx.user" --portal 172.16.33.14:3260 --login

You should see success message. If you have errors, check the error description and number and search on the nets.

Show the logs of iscsi operations Run tail -n200 /var/log/syslog | less

You should see proper scsi device(s) in dmesg messages
Run dmesg | grep sd to see what is the device name. What is discovered.

See the sample screen shot below

Image and video hosting by TinyPic

In our case /dev/sdg found. Please note that device names will be automatically assigned and will be different then this device name.

Format and mount discovered target

  • If your scsci target disk is < 2TB use fdisk.
  • If your disk is => 2TB use parted

  • Run the open-iscsi service.

At this stage if there is no mistake in conf file, passwords, user names, network and rings of Jupiter still rounded, you are done.

Hope this blog post helps you to understand how pieces fit and iscsi operation done in correct sequence.

Also you can see detailed iscsiadm commands here too