1、 系统环境:ubuntu 14.10
Target : 172.16.0.100
Initiator: 172.16.0.101
172.16.0.102
2、100安装Target软件:
root@ubuntu:~# apt-cache search tgt
...
...
nagios-plugins-basic - Plugins for nagios compatible monitoring systems
tgt - Linux SCSI target user-space tools
epiphany-browser - Intuitive GNOME web browser
...
python-webkit-dev - WebKit/Gtk Python bindings: development files
系统update后可通过search查看tgt软件
root@ubuntu:~# apt-get install tgt
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
libboost-system1.54.0 libboost-thread1.54.0 libconfig-general-perl
libibverbs1 libnspr4 libnss3 libnss3-nssdb librados2 librbd1 librdmacm1
libsgutils2-2 sg3-utils
The following NEW packages will be installed:
libboost-system1.54.0 libboost-thread1.54.0 libconfig-general-perl
libibverbs1 libnspr4 libnss3 libnss3-nssdb librados2 librbd1 librdmacm1
libsgutils2-2 sg3-utils tgt
0 upgraded, 13 newly installed, 0 to remove and 0 not upgraded.
Need to get 3,858 kB of archives.
After this operation, 13.9 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
3、安装完毕后检查tgt运行状态
root@ubuntu:~# service tgt status
tgt start/running, process 1964
tgt服务正在运行进程号 1964
root@ubuntu:~# ps aux |grep 1964
root 1964 0.0 0.3 261504 6444 ? Ssl 06:15 0:00 tgtd
root 2296 0.0 0.1 8180 2072 pts/3 S+ 07:25 0:00 grep --color=auto 1964
4、tgtadmin help查看tgtadmin 命令参数
root@ubuntu:~# tgtadm --help
Linux SCSI Target administration utility, version 1.0.43
Usage: tgtadm [OPTION]
--lld <driver> --mode target --op new --tid <id> --targetname <name>
add a new target with <id> and <name>. <id> must not be zero.
--lld <driver> --mode target --op delete [--force] --tid <id>
delete the specific target with <id>.
With force option, the specific target is deleted
even if there is an activity.
--lld <driver> --mode target --op show
show all the targets.
--lld <driver> --mode target --op show --tid <id>
show the specific target's parameters.
--lld <driver> --mode target --op update --tid <id> --name <param> --value <value>
change the target parameters of the target with <id>.
--lld <driver> --mode target --op bind --tid <id> --initiator-address <address>
--lld <driver> --mode target --op bind --tid <id> --initiator-name <name>
enable the target to accept the specific initiators.
--lld <driver> --mode target --op unbind --tid <id> --initiator-address <address>
--lld <driver> --mode target --op unbind --tid <id> --initiator-name <name>
disable the specific permitted initiators.
--lld <driver> --mode logicalunit --op new --tid <id> --lun <lun>
--backing-store <path> --bstype <type> --bsopts <bs options> --bsoflags <options>
add a new logical unit with <lun> to the specific
target with <id>. The logical unit is offered
to the initiators. <path> must be block device files
(including LVM and RAID devices) or regular files.
bstype option is optional.
bsopts are specific to the bstype.
bsoflags supported options are sync and direct
(sync:direct for both).
--lld <driver> --mode logicalunit --op delete --tid <id> --lun <lun>
delete the specific logical unit with <lun> that
the target with <id> has.
--lld <driver> --mode account --op new --user <name> --password <pass>
add a new account with <name> and <pass>.
--lld <driver> --mode account --op delete --user <name>
delete the specific account having <name>.
--lld <driver> --mode account --op bind --tid <id> --user <name> [--outgoing]
add the specific account having <name> to
the specific target with <id>.
<user> could be <IncomingUser> or <OutgoingUser>.
If you use --outgoing option, the account will
be added as an outgoing account.
--lld <driver> --mode account --op unbind --tid <id> --user <name> [--outgoing]
delete the specific account having <name> from specific
target. The --outgoing option must be added if you
delete an outgoing account.
--lld <driver> --mode lld --op start
Start the specified lld without restarting the tgtd process.
--control-port <port> use control port <port>
--help
display this help and exit
Report bugs to <stgt@vger.kernel.org>.
5、创建 Target
Tgtadmin 中关于 target命令如下:
--lld <driver> --mode target --op new --tid <id> --targetname <name>
add a new target with <id> and <name>. <id> must not be zero.
--lld <driver> --mode target --op delete [--force] --tid <id>
delete the specific target with <id>.With force option, the specific target is deleted
even if there is an activity.
--lld <driver> --mode target --op show
show all the targets.
开始创建target
root@ubuntu:~# tgtadm --lld iscsi --mode target --op new --tid 1 --targetname iqn.2014-0728.com:longtang
#--lld iscsi 固定参数驱动iscsi
#--mode target 模式target
#--op new 操作new新创建
#--tid 1 target id号
#--targetname ...target名称
#查看Target记录</span>
root@ubuntu:~# tgtadm --lld iscsi --mode target --op show
Target 1: iqn.2014-0728.com:longtang
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00010000
SCSI SN: beaf10
Size: 0 MB, Block size: 1
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
SWP: No
Thin-provisioning: No
Backing store type: null
Backing store path: None
Backing store flags:
LUN: 1
Type: disk
SCSI ID: IET 00010001
SCSI SN: beaf11
Size: 107373 MB, Block size: 512
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
SWP: No
Thin-provisioning: No
Backing store type: rdwr
Backing store path: /dev/sdb1
Backing store flags:
Account information:
ACL information:
ALL
参数会列出,默认创建LUN号为0 的逻辑单元
绑定 initiator name, 如果不操作 后续initiator可能无法发现target
root@ubuntu:~# tgtadm --lld iscsi --mode target --op bind --tid 1 -I ALL
上面命令为任何 initiator都可以连接 该 target。
6、为Target新增逻辑单元
--lld <driver> --mode logicalunit --op new --tid <id> --lun <lun>
--backing-store <path> --bstype <type> --bsopts <bs options> --bsoflags <options>
add a new logical unit with <lun> to the specific
target with <id>. The logical unit is offered
to the initiators. <path> must be block device files
<span style="color:#ff0000;">(including LVM and RAID devices)</span> or<span style="color:#ff0000;"> regular files</span>.
bstype option is optional.
bsopts are specific to the bstype.
bsoflags supported options are sync and direct
(sync:direct for both).
--lld <driver> --mode logicalunit --op delete --tid <id> --lun <lun>
delete the specific logical unit with <lun> that
the target with <id> has.
root@ubuntu:~# root@ubuntu:~# tgtadm --lld iscsi --mode logicalunit --op new --tid 1 --lun 1 -b /dev/sdb1
root@ubuntu:~# tgtadm --lld iscsi --mode target --op show
Target 1: iqn.2014-0728.com:longtang
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00010000
SCSI SN: beaf10
Size: 0 MB, Block size: 1
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
SWP: No
Thin-provisioning: No
Backing store type: null
Backing store path: None
Backing store flags:
LUN: 1
Type: disk
SCSI ID: IET 00010001
SCSI SN: beaf11
Size: 107373 MB, Block size: 512
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
SWP: No
Thin-provisioning: No
Backing store type: rdwr
Backing store path: /dev/sdb1
Backing store flags:
Account information:
ACL information:
ALL
创建逻辑单元号为1 的逻辑单元,;路径可以为LVM设备或者RAID设备,或者是文件,有时利用dd创建模拟文件可以作为Path
7、安装客户端软件
root@ubuntu:~# apt-get install open-iscsi
Reading package lists... Done
Building dependency tree
...
检查客户端运行状态,如果未运行则开启
root@ubuntu:~# service open-iscsi status
Current active iSCSI sessions:
iscsiadm: No active sessions.
root@king:~# service open-iscsi start
* Starting iSCSI initiator service iscsid
iSCSI daemon already running
8、ISCSI客户端连接 Target
发现Target
root@ubuntu:~# iscsiadm -m discovery -t sendtargets -p 172.16.0.100
172.16.0.100:3260,1 iqn.2001-04.com.example:storage.lun1.disk2.sys1.xyz
如果 target的initiator name 不绑定为 all 此处会无法发现 target
显示已发现 target
root@ubuntu:~# iscsiadm -m node
172.16.0.100:3260,1 iqn.2001-04.com.example:storage.lun1.disk2.sys1.xyz
9、连接 Target
root@ubuntu:~# iscsiadm -m node -p 172.16.0.100 --login
Logging in to [iface: default, target: iqn.2001-04.com.example:storage.lun1.disk2.sys1.xyz, portal: 172.16.0.100,3260] (multiple)
Login to [iface: default, target: iqn.2001-04.com.example:storage.lun1.disk2.sys1.xyz, portal: 172.16.0.100,3260] successful.
10、查看磁盘设备
root@ubuntu:~# ls /dev/sd*
/dev/sda /dev/sda1 /dev/sda2 /dev/sda5 /dev/sdb /dev/sdb1
root@ubuntu:~# fdisk -l
Disk /dev/sda: 60 GiB, 64424509440 bytes, 125829120 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x8d75991a
Device Boot Start End Sectors Size Id Type
/dev/sda1 * 2048 121634815 121632768 58G 83 Linux
/dev/sda2 121636862 125827071 4190210 2G 5 Extended
/dev/sda5 121636864 125827071 4190208 2G 82 Linux swap / Solaris
Disk /dev/sdb: 100 GiB, 107374182400 bytes, 209715200 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xbfc71a9e
Device Boot Start End Sectors Size Id Type
/dev/sdb1 2048 209715199 209713152 100G 83 Linux
11、断开Target
root@ubuntu:~# iscsiadm -m node -p 172.16.0.100 --logout
Logging out of session [sid: 3, target: iqn.2001-04.com.example:storage.lun1.disk2.sys1.xyz, portal: 172.16.0.100,3260]
Logout of [sid: 3, target: iqn.2001-04.com.example:storage.lun1.disk2.sys1.xyz, portal: 172.16.0.100,3260] successful.
root@ubuntu:~# ls /dev/sd*
/dev/sda /dev/sda1 /dev/sda2 /dev/sda5
11、如果担心指定IP什么太麻烦,直接接连 所有的 Target
root@ubuntu:~# iscsiadm -m node --login all
注:
ISCSI 端口号:
root@ubuntu:~# netstat -anl |grep 3260
tcp 0 0 172.16.0.102:54978 172.16.0.100:3260 ESTABLISHED
/etc/service 中也存在默认端口号
测试:
1、多点挂载:数据读写
同一块盘,多点挂载,两台clien挂载后 同时通过脚本插入数据到指定文本, 最终显示的数据是最先写入数据的clien
2、访问控制测试:
Iscsi 在Server 端可以使用tgt 通过TCP/IP 控制IP地址的访问挂载
Tgtadmin所做的修改全都在内核中保存的, 重启服务器后所有的记录将会消失,若想自动生效,需要哦将配置写在 /etc/tgt/conf.d/ target.conf中。
配置内容如下:
<target iqn.2001-04.com.example:storage.lun1.disk2.sys1.xyz>
backing-store /dev/sdb1
initiator-address 172.16.0.102
</target>
说明:backing-store 共享出的块存储
Initiator-address 允许Initiator 访问的地址
3、磁盘扩容,经过测试,可以扩容 Server端扩容时, tgt连接必须先停掉, 否则无法进行扩容
4、若客户端在挂载的时候,断开Target
root@ubuntu:~# iscsiadm -m node -p 172.16.0.100 –logout
挂载的分区不能读写:
root@ubuntu:~# cd /opt/kevin/
root@ubuntu:/opt/kevin# ls
ls: reading directory .: Input/output error
root@ubuntu:/opt/kevin# cd
5、客户端: 查看Server共享出的磁盘空间为90G
root@ubuntu:~# fdisk -l
Disk /dev/sda: 60 GiB, 64424509440 bytes, 125829120 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x8d75991a
Device Boot Start End Sectors Size Id Type
/dev/sda1 * 2048 121634815 121632768 58G 83 Linux
/dev/sda2 121636862 125827071 4190210 2G 5 Extended
/dev/sda5 121636864 125827071 4190208 2G 82 Linux swap / Solaris
Disk /dev/sdb: 90 GiB, 96635715584 bytes, 188741632 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00000000
挂载后,发现空间还是原来的 19G
root@ubuntu:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 57G 4.1G 51G 8% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 988M 4.0K 988M 1% /dev
tmpfs 201M 488K 200M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 1001M 0 1001M 0% /run/shm
none 100M 0 100M 0% /run/user
/dev/sdb 19G 45M 18G 1% /opt/kevin
客户端也需要手动进行扩容分区大小
root@ubuntu:~# resize2fs /dev/sdb 88G
resize2fs 1.42.10 (18-May-2014)
Please run 'e2fsck -f /dev/sdb' first. 请检查磁盘的可用性
———————————————————————————————————————
root@ubuntu:~# e2fsck -f /dev/sdb
e2fsck 1.42.10 (18-May-2014)
Resize inode not valid. Recreate<y>? yes
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Free blocks count wrong for group #0 (31220, counted=31221).
Fix<y>? yes
Free blocks count wrong (4858522, counted=4858523).
Fix<y>? yes
/dev/sdb: ***** FILE SYSTEM WAS MODIFIED *****
/dev/sdb: 14/1245184 files (0.0% non-contiguous), 122213/4980736 blocks
root@ubuntu:~# resize2fs /dev/sdb 90G
resize2fs 1.42.10 (18-May-2014)
The containing partition (or device) is only 23592704 (4k) blocks.
You requested a new size of 23592960 blocks.
扩容磁盘,值不能超出总块值大小
root@ubuntu:~# resize2fs /dev/sdb 88G
resize2fs 1.42.10 (18-May-2014)
Resizing the filesystem on /dev/sdb to 23068672 (4k) blocks.
The filesystem on /dev/sdb is now 23068672 blocks long.
查看是否扩容成功
root@ubuntu:~# mount /dev/sdb /opt/kevin/
root@ubuntu:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 57G 4.1G 51G 8% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 988M 4.0K 988M 1% /dev
tmpfs 201M 488K 200M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 1001M 0 1001M 0% /run/shm
none 100M 0 100M 0% /run/user
/dev/sdb 87G 56M 83G 1% /opt/kevin
完成
注意:客户端可以进行格式化磁盘, 格式化后的磁盘会直接影响Server端对应盘的数据
6、扩容磁盘时,server 端将tgt停掉,加磁盘空间后,进行重新分区,重新扩展分区大小,检查文件系统正确性。
客户端
1、重新分区
root@ubuntu:~# fdisk /dev/sdb
root@ubuntu:~# fdisk -l
root@ubuntu:~# resize2fs /dev/sdb1 59G
root@ubuntu:~# e2fsck -f /dev/sdb1
挂载后若出现旧的空间,卸载后重新resize
mount /dev/sdb1 /opt/kevin/
df -h
resize2fs /dev/sdb1 59G
e2fsck -f /dev/sdb1
然后就显示新的空间
root@ubuntu:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 57G 4.1G 51G 8% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 988M 4.0K 988M 1% /dev
tmpfs 201M 484K 200M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 1001M 0 1001M 0% /run/shm
none 100M 0 100M 0% /run/user
/dev/sdb1 58G 52M 56G 1% /opt/kevin
需要 检测文件系统的正确性 root@ubuntu:~# e2fsck -f /dev/sdb1
7、Server端可以同时分享出多个分区空间; clien端都可以分别挂载
root@ubuntu:~# mount /dev/sdb1 /opt/kevin/
root@ubuntu:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 57G 4.1G 51G 8% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 988M 4.0K 988M 1% /dev
tmpfs 201M 496K 200M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 1001M 0 1001M 0% /run/shm
none 100M 0 100M 0% /run/user
/dev/sdd 20G 244M 19G 2% /tmp/zzzzzzzzzzzz
/dev/sdc 976M 202M 708M 23% /tmp/aaa
/dev/sdb1 9.8G 23M 9.2G 1% /opt/kevin