We had to deal with open-iscsi in Ubuntu Xenial 16.04. Basically, we wanted to provide mount points for cinder in our Openstack Ocata cluster.
So here is how pain and suffer begun because nowhere a suitable "step by step workflow" of setup described to understand iscsi mechanism. Even no article describes how things move on the timeline in correct order.
I hope it helps to you to understand basic flow of open-iscsi and you can achieve your goal with a less pain then us :)
- Please read everything at first and then START to do your operations later!!!
You should know this info below, before starting to setup iscsi properly.
- iscsi user name
- iscsi password for target(s)
- iscsi host ip address
- iscsi host port number
Pieces of open-iscsi
- First install open-iscsi. Remember that, once you've installed a service, it tries to start at the very beginning. Check if it is running
service open-iscsi status
- there is a
/etc/iscsi/iscsid.conffile for config settings.
- there is a
/etc/iscsi/nodesdirectory to hold scanned iscsi targets information.
- you'll have
iscsiadmtool to manage all the stuff in console.
How things happen when you work with iscsi in correct order?
Before you go any further, stop the service.
service open-iscsi stop
You have to check
iscsid.confat first. If you have more then one targets, please comment out all password and user fields. If you have only one target to connect, please fill the necessary fields according to your authentication scheme in this file.
node.session.auth.authmethod = CHAP node.session.auth.username = your_user node.session.auth.password = your_pass node.session.auth.username_in = your_user node.session.auth.password_in = your_pass discovery.sendtargets.auth.username = your_user discovery.sendtargets.auth.password = your_pass
if you don't know what is multipath, do not mess with it. Don't play with the parameters that you have no idea about!
- Now it is time to check what do you have under
/etc/iscsi/nodes/directory. Since there is no targets defined in conf file, there shouldn't be any target information directory under nodes dir.
ls /etc/iscsi/nodes/to see.
You should never directly edit the files created under nodes dir. Always use
iscsiadmcommand to manage all iscsi related operations.
In case you've found any target info under nodes dir, you have to delete them all by running this command for the clean install. Be carefull if there is an existing iscsi installation runs and having alive mounts on the server.
iscsiadm -m node --targetname "exact_dir_name_under_nodes_dir" --portal "your_iscsi_server_ip:your_iscsi_port_3260_in_general" --op=delete
to delete discovered node.
Check how many targets you have under the nodes dir and run the same command for for all of them.
- Ok now we are ready to discover nodes properly. You have to run this command to discover targets.
iscsiadm -m discovery -t sendtargets -p your_iscsi_server_ip
to see what is discovered as target(s). When this command completed you'll have target(s) listed in console and these targets going to be added to the nodes dir by checking the params in
Trick is every time you play with iscsid.conf file, you have to reiterate all this cycle again! Delete nodes, discover them, add user and password with iscsiadm command and check if connected device blocks are stays intact in same device name or not.
Setup is quite easy for just one target. In this guide, we will show, how you could manage more then one target.
At this point service open-iscsi is not running but you have discovered targets. Now we have to add auth method, user and password per node. Find the target names under nodes dir and apply these commands for every target listed there.
iscsiadm -m node --targetname "iqn.xxx.target_name" --portal "172.16.33.14:3260" --op=update --name node.session.auth.authmethod --value=your_auth_method_in_general_CHAP iscsiadm -m node --targetname "iqn.xxx.target_name" --portal "172.16.33.14:3260" --op=update --name node.session.auth.username --value=user_name iscsiadm -m node --targetname "iqn.xxx.user_name" --portal "172.16.33.14:3260" --op=update --name node.session.auth.password --value=your_password
Nothing happens in console but actually this information added to the files under nodes dir.
It is time to login to discovered targets. To login all targets at once run
iscsiadm -m node --login
To login specific target, run
iscsiadm -m node --targetname "iqn.xxx.user" --portal 172.16.33.14:3260 --login
You should see success message. If you have errors, check the error description and number and search on the nets.
Show the logs of iscsi operations Run
tail -n200 /var/log/syslog | less
You should see proper scsi device(s) in dmesg messages
dmesg | grep sd to see what is the device name. What is discovered.
See the sample screen shot below
In our case /dev/sdg found. Please note that device names will be automatically assigned and will be different then this device name.
Format and mount discovered target
- If your scsci target disk is < 2TB use fdisk.
If your disk is => 2TB use parted
Run the open-iscsi service.
At this stage if there is no mistake in conf file, passwords, user names, network and rings of Jupiter still rounded, you are done.
Hope this blog post helps you to understand how pieces fit and iscsi operation done in correct sequence.
Also you can see detailed iscsiadm commands here too