ISCSI服务的搭建

一、ISCSI服务简介

当我们的系统需要大量的磁盘容量，但是身边却没有足够的存储设备，此时，我们可以使用通过网络的scsi磁盘，即Internet scsi（iscsi）。iscsi主要是通过TCP/IP的技术，将存储设备端通过iscsi target功能，做成可以提供磁盘的服务器端，再通过iscsi initiator（iscsi初始化用户）功能，做成能够挂载使用iscsi target的客户端，这样就能够通过iscsi协议来进行磁盘的应用了。
iscsi服务端需要使用的软件为targetcli
iscsi客户端需要的软件为iscsi-initiator-utils

二、实验

(1)准备要共享的设备或文件

iSCSI常用的共享有块设备（磁盘，分区，逻辑卷等）和大文件
这里演示常用的四种分区，磁盘，逻辑卷，文件。
分区
使用sdb的sdb1

[root@server /]# lsblk | grep sdb1
├─sdb1          8:17   0    1G  0 part

磁盘
使用sdc

[root@server ~]# lsblk | grep sdc
sdc             8:32   0    5G  0 disk

逻辑卷
使用sdb2作为pv，创建lv1

[root@server /]# pvcreate /dev/sdb2
  Physical volume "/dev/sdb2" successfully created
[root@server /]# vgcreate vg0 /dev/sdb2
  Volume group "vg0" successfully created
[root@server /]# lvcreate -n lv1 -L 700M vg0
  Logical volume "lv1" created.

文件
使用dd命令创建一个500M大小的文件iscsi

[root@server /]# dd if=/dev/zero of=/iscsi bs=500M count=1
1+0 records in
1+0 records out
524288000 bytes (524 MB) copied, 7.97686 s, 65.7 MB/s
[root@server /]# ll -h /| grep iscsi
-rw-r--r--.   1 root root 500M Dec  6 16:54 iscsi

(2)服务端配置

1、装包
[root@server /]# yum install targetcli -y
targetcli是用于管理iscsi服务端存储资源的专用配置命令，它能够提供类似于fdisk命令的交互式配置功能，将iscsi共享资源的配置内容抽象成“目录”的形式，我们只需要将各类配置信息填入到相应的“目录”中即可。
2、进入交互式界面
[root@server /]# targetcli

[root@server /]# targetcli
targetcli shell version 2.1.fb41
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- block .............................................. [Storage Objects: 0]
  | o- fileio ............................................. [Storage Objects: 0]
  | o- pscsi .............................................. [Storage Objects: 0]
  | o- ramdisk ............................................ [Storage Objects: 0]
  o- iscsi ........................................................ [Targets: 0]
  o- loopback ..................................................... [Targets: 0]
/>

名词	解释
backstores	后备存储
block	块设备磁盘驱动器，磁盘分区，逻辑卷，以及服务器上定义的任何b类型的设备文件
fileio	大文件
pscsi	物理scsi，通常不用
ramdisk	内存盘，其中存储的数据在服务器重启后将全部丢失

3、创建共享的设备或文件
分区，磁盘，逻辑卷都属于块设备，进入block底下创建

/> cd /backstores/block
/backstores/block> create dev=/dev/sdb1 name=lun0
/backstores/block> create dev=/dev/sdc name=lun1
/backstores/block> create dev=/dev/vg0/lv1 name=lun2

文件不属于块设备，进入fileio底下创建

/backstores/block> cd /backstores/fileio
/backstores/fileio> create file_or_dev=/iscsi name=lun3

查看一下，结构非常清晰

/backstores/fileio> cd /
/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- block .............................................. [Storage Objects: 3]
  | | o- lun0 ...................... [/dev/sdb1 (1.0GiB) write-thru deactivated]
  | | o- lun1 ....................... [/dev/sdc (5.0GiB) write-thru deactivated]
  | | o- lun2 ................. [/dev/vg0/lv1 (700.0MiB) write-thru deactivated]
  | o- fileio ............................................. [Storage Objects: 1]
  | | o- lun3 ....................... [/iscsi (500.0MiB) write-back deactivated]
  | o- pscsi .............................................. [Storage Objects: 0]
  | o- ramdisk ............................................ [Storage Objects: 0]
  o- iscsi ........................................................ [Targets: 0]
  o- loopback ..................................................... [Targets: 0]

4、设置共享的iscsi target名称及配置共享资源
说明：
1.使用lun0和lun1作为iqn.2019-12.com.redhat.linux.1:server的共享，并且设置自动生成acl节点，即客户端不需要设置iscsi客户端名称
2.使用lun2和lun3作为iqn.2019-12.com.redhat.linux.2:server的共享，并且手动设置acl节点，即客户端的iscsi客户端名称需要和服务端设置的acl一致
3.位于生产环境中的服务器可能有多块网卡，那么就可以在这儿指定由哪个网卡或IP地址对外提供共享存储资源，关闭自动创建默认portal
/> set group=global auto_add_default_portal=false
注:若未关闭则更改IP地址时需要把所有target名称底下的0.0.0.0:3260删除才能更改

设置iqn.2019-12.com.redhat.linux.1:server
1.进入iscsi目录创建target名称

/> cd /iscsi
/iscsi> create iqn.2019-12.com.redhat.linux.1:server

2.此时的目录结构

/iscsi> ls
o- iscsi .......................................................... [Targets: 1]
  o- iqn.2019-12.com.redhat.linux.1:server ........................... [TPGs: 1]
    o- tpg1 ................................................ [gen-acls, no-auth]
      o- acls ........................................................ [ACLs: 0]
      o- luns ........................................................ [LUNs: 2]
      o- portals .................................................. [Portals: 0]

3.设置自动生成acl节点

/iscsi> cd iqn.2019-12.com.redhat.linux.1:server/tpg1///进入tpg1目录下
/iscsi/iqn.20...1:server/tpg1> set attribute authentication=0 //关闭账号密码访问，默认关闭
/iscsi/iqn.20...1:server/tpg1> set attribute generate_node_acls=1//自动生成acl节点

4.设置共享的lun0和lun1

/iscsi/iqn.20...1:server/tpg1> cd luns
/iscsi/iqn.20...ver/tpg1/luns> create /backstores/block/lun0
/iscsi/iqn.20...ver/tpg1/luns> create /backstores/block/lun1

5.设置监听的网卡IP地址和端口号

/iscsi/iqn.20...ver/tpg1/luns> cd ..
/iscsi/iqn.20...1.server/tpg1> cd portals
/iscsi/iqn.20.../tpg1/portals> create 192.168.19.101 3260

6.此时的目录结构

/iscsi/iqn.20.../tpg1/portals> ls /
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- block .............................................. [Storage Objects: 3]
  | | o- lun0 ........................ [/dev/sdb1 (1.0GiB) write-thru activated]
  | | o- lun1 ......................... [/dev/sdc (5.0GiB) write-thru activated]
  | | o- lun2 ................. [/dev/vg0/lv1 (700.0MiB) write-thru deactivated]
  | o- fileio ............................................. [Storage Objects: 1]
  | | o- lun3 ....................... [/iscsi (500.0MiB) write-back deactivated]
  | o- pscsi .............................................. [Storage Objects: 0]
  | o- ramdisk ............................................ [Storage Objects: 0]
  o- iscsi ........................................................ [Targets: 1]
  | o- iqn.2019-12.com.redhat.linux.1:server ......................... [TPGs: 1]
  |   o- tpg1 .............................................. [gen-acls, no-auth]
  |     o- acls ...................................................... [ACLs: 0]
  |     o- luns ...................................................... [LUNs: 2]
  |     | o- lun0 ..................................... [block/lun0 (/dev/sdb1)]
  |     | o- lun1 ...................................... [block/lun1 (/dev/sdc)]
  |     o- portals ................................................ [Portals: 1]
  |       o- 192.168.19.101:3260 .......................................... [OK]
  o- loopback ..................................................... [Targets: 0]

设置iqn.2019-12.com.redhat.linux.2:server
1.进入iscsi目录创建target名称

/> cd /iscsi
/iscsi> create iqn.2019-12.com.redhat.linux.2:server

2.此时的目录结构

/iscsi> ls
o- iscsi .......................................................... [Targets: 2]
  o- iqn.2019-12.com.redhat.linux.1:server ........................... [TPGs: 1]
  | o- tpg1 ................................................ [gen-acls, no-auth]
  |   o- acls ........................................................ [ACLs: 0]
  |   o- luns ........................................................ [LUNs: 2]
  |   | o- lun0 ....................................... [block/lun0 (/dev/sdb1)]
  |   | o- lun1 ........................................ [block/lun1 (/dev/sdc)]
  |   o- portals .................................................. [Portals: 1]
  |     o- 192.168.19.201:3260 ............................................ [OK]
  o- iqn.2019-12.com.redhat.linux.2:server ........................... [TPGs: 1]
    o- tpg1 ............................................. [no-gen-acls, no-auth]
      o- acls ........................................................ [ACLs: 0]
      o- luns ........................................................ [LUNs: 0]
      o- portals .................................................. [Portals: 0]

3.手动设置acl节点

/iscsi> cd iqn.2019-12.com.redhat.linux.2:server/tpg1///进入tpg1目录下
/iscsi/iqn.20...2:server/tpg1> get attribute authentication//查看是否关闭账号密码，默认关闭
authentication=0
/iscsi/iqn.20...2:server/tpg1> get attribute authentication//默认需要手动设置acl节点
authentication=0
/iscsi/iqn.20...2:server/tpg1> cd acls//进入此目录设置acl节点
/iscsi/iqn.20...ver/tpg1/acls> create iqn.2019-12.com.redhat.linux:client//创建客户端名称

4.设置共享的lun2和lun3

/iscsi/iqn.20...ver/tpg1/acls> cd ..
/iscsi/iqn.20...2:server/tpg1> cd luns
/iscsi/iqn.20...ver/tpg1/luns> create /backstores/block/lun2
/iscsi/iqn.20...ver/tpg1/luns> create /backstores/fileio/lun3

5.设置监听的网卡IP地址和端口号

/iscsi/iqn.20...ver/tpg1/luns> cd ..
/iscsi/iqn.20...2:server/tpg1> cd portals
/iscsi/iqn.20.../tpg1/portals> create 192.168.19.101 3260

6.此时的目录结构

/iscsi/iqn.20...ver/tpg1/luns> ls /
o- / ...................................................................... [...]
  o- backstores ........................................................... [...]
  | o- block ............................................... [Storage Objects: 3]
  | | o- lun0 ......................... [/dev/sdb1 (1.0GiB) write-thru activated]
  | | o- lun1 .......................... [/dev/sdc (5.0GiB) write-thru activated]
  | | o- lun2 .................... [/dev/vg0/lv1 (700.0MiB) write-thru activated]
  | o- fileio .............................................. [Storage Objects: 1]
  | | o- lun3 .......................... [/iscsi (500.0MiB) write-back activated]
  | o- pscsi ............................................... [Storage Objects: 0]
  | o- ramdisk ............................................. [Storage Objects: 0]
  o- iscsi ......................................................... [Targets: 2]
  | o- iqn.2019-12.com.redhat.linux.1:server .......................... [TPGs: 1]
  | | o- tpg1 ............................................... [gen-acls, no-auth]
  | |   o- acls ....................................................... [ACLs: 0]
  | |   o- luns ....................................................... [LUNs: 2]
  | |   | o- lun0 ...................................... [block/lun0 (/dev/sdb1)]
  | |   | o- lun1 ....................................... [block/lun1 (/dev/sdc)]
  | |   o- portals ................................................. [Portals: 1]
  | |     o- 192.168.19.101:3260 ........................................... [OK]
  | o- iqn.2019-12.com.redhat.linux.2:server .......................... [TPGs: 1]
  |   o- tpg1 ............................................ [no-gen-acls, no-auth]
  |     o- acls ....................................................... [ACLs: 1]
  |     | o- iqn.2019-12.com.redhat.linux:client ............... [Mapped LUNs: 2]
  |     |   o- mapped_lun0 ............................... [lun0 block/lun2 (rw)]
  |     |   o- mapped_lun1 .............................. [lun1 fileio/lun3 (rw)]
  |     o- luns ....................................................... [LUNs: 2]
  |     | o- lun0 ................................... [block/lun2 (/dev/vg0/lv1)]
  |     | o- lun1 ........................................ [fileio/lun3 (/iscsi)]
  |     o- portals ................................................. [Portals: 1]
  |       o- 192.168.19.101:3260 ........................................... [OK]
  o- loopback ...................................................... [Targets: 0]

退出保存
/iscsi/iqn.20...2.server/tpg1> exit
5.关闭selinux，防火墙放行端口号

[root@server /]# setenforce 0
[root@server /]# firewall-cmd --add-port=3260/tcp --per
[root@server /]# firewall-cmd --reload

6.重启服务
[root@server /]# systemctl restart target
总结步骤:
1.将要共享的设备或文件添加进后备存储
2.创建服务器target名称
3.将后备存储添加到某一个target名称底下
4.设置acls（步骤3或4可颠倒）
5.设置监听的网卡IP和端口号
6.退出保存
ls /就可以很清楚的看到目录结构，然后进行配置

(3)客户端配置

1.装包
[root@client ~]# yum install iscsi-initiator-utils -y
2.重启服务
[root@client ~]# systemctl restart iscsi iscsid
3.发现iSCSI服务器

[root@client ~]# iscsiadm -m discovery -t st -p 192.168.19.101
192.168.19.101:3260,1 iqn.2012-12.com.redhat.linux.2:server
192.168.19.101:3260,1 iqn.2012-12.com.redhat.linux.1:server

4.登陆iSCSI服务器

[root@client ~]# iscsiadm -m node -T iqn.2012-12.com.redhat.linux.1:server -l
Logging in to [iface: default, target: iqn.2012-12.com.redhat.linux.1:server, portal: 192.168.19.101,3260] (multiple)
Login to [iface: default, target: iqn.2012-12.com.redhat.linux.1:server, portal: 192.168.19.101,3260] successful.

[root@client ~]# iscsiadm -m node -T iqn.2012-12.com.redhat.linux.2:server -l
Logging in to [iface: default, target: iqn.2012-12.com.redhat.linux.2:server, portal: 192.168.19.101,3260] (multiple)
iscsiadm: Could not login to [iface: default, target: iqn.2012-12.com.redhat.linux.2:server, portal: 192.168.19.101,3260].
iscsiadm: initiator reported error (24 - iSCSI login failed due to authorization failure)

这里登陆.2服务器时出错，是因为服务器做了acl，因此客户端的iSCSI名称必须和服务端acl设置的一致
5.更改iSCSI客户端名称
[root@client ~]# vim /etc/iscsi/initiatorname.iscsi

InitiatorName=iqn.2019-12.com.redhat.linux:client

重启服务
[root@client ~]# systemctl restart iscsi iscsid
重新登陆
[root@client ~]# iscsiadm -m node -T iqn.2012-12.com.redhat.linux.2:server -l

Logging in to [iface: default, target: iqn.2012-12.com.redhat.linux.2:server, portal: 192.168.19.101,3260] (multiple)
Login to [iface: default, target: iqn.2012-12.com.redhat.linux.2:server, portal: 192.168.19.101,3260] successful.

此时客户端看起来会多了几个硬盘（sdb,sdc,sdd,sde）

[root@client ~]# lsblk
NAME          MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda             8:0    0   20G  0 disk
├─sda1          8:1    0    1G  0 part /boot
└─sda2          8:2    0   18G  0 part
  ├─rhel-root 253:0    0   17G  0 lvm  /
  └─rhel-swap 253:1    0    1G  0 lvm  [SWAP]
sdb             8:16   0  700M  0 disk
sdc             8:32   0  500M  0 disk
sdd             8:48   0    1G  1 disk
sde             8:64   0    5G  1 disk

注意：没有通过acl挂载服务端的硬盘无法分区或格式化
尝试格式化上述.1服务端的sdb1（1G），客户端挂载到了sdd（1G）

[root@client ~]# mkfs.ext4 /dev/sdd
mke2fs 1.42.9 (28-Dec-2013)
/dev/sdd is entire device, not just one partition!
Proceed anyway? (y,n) y
/dev/sdd: Read-only file system while setting up superblock

尝试格式化上述.1服务端的sdc（5G），客户端挂载到了sde（5G）

[root@client ~]# fdisk /dev/sde
最终会提示
fdisk: cannot write disk label: Bad file descriptor

6.格式化后挂载到本地文件夹，这里以sdb为例

[root@client ~]# mkfs.ext4 /dev/sdb
[root@client ~]# mkdir /iscsi
[root@client ~]# blkid | grep sdb
/dev/sdb: UUID="a1d7c240-fbe4-40ae-b7e1-684b9e92d849" TYPE="ext4"
[root@client ~]# vim /etc/fstab
UUID=a1d7c240-fbe4-40ae-b7e1-684b9e92d849 /iscsi ext4 defaults,_netdev 0 0
[root@client ~]# mount -a
[root@client ~]# df -h | grep sdb
/dev/sdb               673M  1.4M  623M   1% /iscsi

注：
1.编辑/etc/fstab文件时，注意使用UUID（因为磁盘的文件名不一定是固定的）。
2.由于共享过来的是网络存储设备，而ISCSI协议是基于TCP/IP网络传输数据的，因此必须在/etc/fstab配置文件中添加上_netdev参数，表示当系统联网后再进行挂载操作，以免系统开机时间过长或开机失败。
3.若想要开机自动启动并挂载，则服务端需要设置target服务开机自动启动，客户端需要设置iscli和iscsid服务自动启动，并编辑fatab文件自动挂载。
5.实测，必须使用-T指定要登陆的节点进行登陆，重启后才可自动挂载服务端硬盘
查看连接会话

[root@client ~]# iscsiadm -m session
tcp: [5] 192.168.19.101:3260,1 iqn.2012-12.com.redhat.linux.2:server (non-flash)
tcp: [6] 192.168.19.101:3260,1 iqn.2012-12.com.redhat.linux.1:server (non-flash)

登出
-T指定节点，不指定则登出所有节点

[root@client ~]# iscsiadm -m node -u
Logging out of session [sid: 5, target: iqn.2012-12.com.redhat.linux.2:server, portal: 192.168.19.101,3260]
Logging out of session [sid: 6, target: iqn.2012-12.com.redhat.linux.1:server, portal: 192.168.19.101,3260]
Logout of [sid: 5, target: iqn.2012-12.com.redhat.linux.2:server, portal: 192.168.19.101,3260] successful.
Logout of [sid: 6, target: iqn.2012-12.com.redhat.linux.1:server, portal: 192.168.19.101,3260] successful.