手动模拟attach cinder volume的过程

我们首先启动一台机器，启动的时候attach一个volume

创建一个空的cinder volume

root:~# cinder create --display-name emptyvolume11g 11
+---------------------+--------------------------------------+
|       Property      |                Value                 |
+---------------------+--------------------------------------+
|     attachments     |                  []                  |
|  availability_zone  |                 nova                 |
|       bootable      |                false                 |
|      created_at     |      2014-07-01T22:22:22.802708      |
| display_description |                 None                 |
|     display_name    |            emptyvolume11g            |
|      encrypted      |                False                 |
|          id         | f6ba87f7-d0b6-4fdb-ac82-346371e78c48 |
|       metadata      |                  {}                  |
|         size        |                  11                  |
|     snapshot_id     |                 None                 |
|     source_volid    |                 None                 |
|        status       |               creating               |
|     volume_type     |                 None                 |
+---------------------+--------------------------------------+

root:~# cinder list  
+--------------------------------------+-----------+----------------+------+-------------+----------+-------------+
|                  ID                  |   Status  |  Display Name  | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+----------------+------+-------------+----------+-------------+
| f6ba87f7-d0b6-4fdb-ac82-346371e78c48 | available | emptyvolume11g |  11  |     None    |  false   |             |
+--------------------------------------+-----------+----------------+------+-------------+----------+-------------+

root:~# glance image-list
+--------------------------------------+--------------+-------------+------------------+-----------+--------+
| ID                                   | Name         | Disk Format | Container Format | Size      | Status |
+--------------------------------------+--------------+-------------+------------------+-----------+--------+
| 3d0f6926-3c74-49d2-9465-801e51635a23 | Cirros       | qcow2       | bare             | 14680064  | active |
| 6d9d011c-b007-47df-9044-0729d01d2101 | cirros-0.3.2 | qcow2       | bare             | 13167616  | active |
| d96b0e41-8264-41de-8dbb-6b31ce9bfbfc | ubuntu-12.04 | qcow2       | bare             | 260637184 | active |
| d23d14bb-6156-43b0-b883-926547cfdec4 | ubuntu-14.04 | qcow2       | bare             | 484376576 | active |
+--------------------------------------+--------------+-------------+------------------+-----------+--------+

root:~# nova keypair-list
+-----------+-------------------------------------------------+
| Name      | Fingerprint                                     |
+-----------+-------------------------------------------------+
| openstack | 00:3a:f1:5f:8b:97:d7:fa:0f:9e:4e:be:54:57:02:25 |
+-----------+-------------------------------------------------+

nova --debug boot --flavor 2 --image d96b0e41-8264-41de-8dbb-6b31ce9bfbfc --key-name openstack --security-groups default --block-device source=volume,id=f6ba87f7-d0b6-4fdb-ac82-346371e78c48,dest=volume,shutdown=preserve myInstanceWithVolume

如果我们ssh到虚拟机里面可以看到attached volume

root@myinstancewithvolume:/home/ubuntu# fdisk -l

Disk /dev/vda: 21.5 GB, 21474836480 bytes
4 heads, 32 sectors/track, 327680 cylinders, total 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000c917a

   Device Boot      Start         End      Blocks   Id  System
/dev/vda1   *        2048    41943039    20970496   83  Linux

Disk /dev/vdb: 11.8 GB, 11811160064 bytes
16 heads, 63 sectors/track, 22885 cylinders, total 23068672 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/vdb doesn't contain a valid partition table

通过virsh dumpxml，也可以看到

<disk type='file' device='disk'>
  <driver name='qemu' type='qcow2' cache='none'/>
  <source file='/var/lib/nova/instances/59ca11ea-0978-4f7d-8385-480649e63a1d/disk'/>
  <target dev='vda' bus='virtio'/>
  <alias name='virtio-disk0'/>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</disk>
<disk type='block' device='disk'>
  <driver name='qemu' type='raw' cache='none'/>
  <source dev='/dev/disk/by-path/ip-16.158.166.197:3260-iscsi-iqn.2010-10.org.openstack:volume-f6ba87f7-d0b6-4fdb-ac82-346371e78c48-lun-1'/>
  <target dev='vdb' bus='virtio'/>
  <serial>f6ba87f7-d0b6-4fdb-ac82-346371e78c48</serial>
  <alias name='virtio-disk1'/>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
</disk>

在cinder volume节点上可以看到Logic volume

root# lvdisplay
  --- Logical volume ---
  LV Path                /dev/cinder-volumes/volume-f6ba87f7-d0b6-4fdb-ac82-346371e78c48
  LV Name                volume-f6ba87f7-d0b6-4fdb-ac82-346371e78c48
  VG Name                cinder-volumes
  LV UUID                PKRR9B-nUCI-Ob1Y-U97W-b1Rh-RhuH-Ow4h5P
  LV Write Access        read/write
  LV Creation host, time escto-bj-hp-z620, 2014-07-02 06:22:23 +0800
  LV snapshot status     source of
                         _snapshot-138e677d-721a-435e-b945-fbd6009f3b2a [active]
  LV Status              available
  # open                 1
  LV Size                11.00 GiB
  Current LE             2816
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           252:2
  --- Logical volume ---
  LV Path                /dev/cinder-volumes/_snapshot-138e677d-721a-435e-b945-fbd6009f3b2a
  LV Name                _snapshot-138e677d-721a-435e-b945-fbd6009f3b2a
  VG Name                cinder-volumes
  LV UUID                uTu3GB-J1Nb-1G6Z-xroP-nUmk-394l-KHVYIk
  LV Write Access        read/write
  LV Creation host, time escto-bj-hp-z620, 2014-07-02 19:14:40 +0800
  LV snapshot status     active destination for volume-f6ba87f7-d0b6-4fdb-ac82-346371e78c48
  LV Status              available
  # open                 0
  LV Size                11.00 GiB
  Current LE             2816
  COW-table size         11.00 GiB
  COW-table LE           2816
  Allocated to snapshot  0.00%
  Snapshot chunk size    4.00 KiB
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           252:3

在cinder volume节点上用tgt查看iscsi target

root# tgtadm --mode target --op show
Target 1: iqn.2010-10.org.openstack:volume-f6ba87f7-d0b6-4fdb-ac82-346371e78c48
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
        I_T nexus: 1
            Initiator: iqn.1993-08.org.debian:01:f7337e3e61e4 alias: escto-bj-hp-z620
            Connection: 0
                IP Address: 16.158.166.197
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: IET     00010000
            SCSI SN: beaf10
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: null
            Backing store path: None
            Backing store flags:
        LUN: 1
            Type: disk
            SCSI ID: IET     00010001
            SCSI SN: beaf11
            Size: 11811 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: rdwr
            Backing store path: /dev/cinder-volumes/volume-f6ba87f7-d0b6-4fdb-ac82-346371e78c48
            Backing store flags:
    Account information:
    ACL information:
        ALL

下面我们手工attach一个volume

在cinder volume上创建一个iscsi target

tgtadm --lld iscsi --op new --mode target --tid 2 -T iqn.2010-10.org.openstack:volume-cliu8-test

Target 2: iqn.2010-10.org.openstack:volume-cliu8-test
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: IET     00020000
            SCSI SN: beaf20
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: null
            Backing store path: None
            Backing store flags:
    Account information:
    ACL information:

创建一个Logic volume

lvcreate -L 14G -n lv_cliu8_test cinder-volumes
  --- Logical volume ---
  LV Path                /dev/cinder-volumes/lv_cliu8_test
  LV Name                lv_cliu8_test
  VG Name                cinder-volumes
  LV UUID                Fg1uLV-Rdfc-p33c-1m03-GMrn-QgJ4-mpMelP
  LV Write Access        read/write
  LV Creation host, time escto-bj-hp-z620, 2014-07-02 23:20:27 +0800
  LV Status              available
  # open                 0
  LV Size                14.00 GiB
  Current LE             3584
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           252:6

将Logic Volume加入刚才创建的iscsi target

tgtadm --lld iscsi --op new --mode logicalunit --tid 2 --lun 1 -b /dev/cinder-volumes/lv_cliu8_test

Target 2: iqn.2010-10.org.openstack:volume-cliu8-test
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: IET     00020000
            SCSI SN: beaf20
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: null
            Backing store path: None
            Backing store flags:
        LUN: 1
            Type: disk
            SCSI ID: IET     00020001
            SCSI SN: beaf21
            Size: 15032 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: rdwr
            Backing store path: /dev/cinder-volumes/lv_cliu8_test
            Backing store flags:
    Account information:
    ACL information:

配置iscsi target监听链接

tgtadm --lld iscsi --op bind --mode target --tid 2 -I ALL

root@escto-bj-hp-z620:/var/lib/cinder/volumes# netstat -tulpn | grep 3260
tcp        0      0 0.0.0.0:3260            0.0.0.0:*               LISTEN      22569/tgtd     
tcp6       0      0 :::3260                 :::*                    LISTEN      22569/tgtd 

在虚拟机运行的节点上discover这个iscsi target

iscsiadm --mode discovery --type sendtargets --portal 16.158.166.197

root# iscsiadm --mode discovery --type sendtargets --portal 16.158.166.197
16.158.166.197:3260,1 iqn.2010-10.org.openstack:volume-f6ba87f7-d0b6-4fdb-ac82-346371e78c48
16.158.166.197:3260,1 iqn.2010-10.org.openstack:volume-cliu8-test

连接那个iscsi target

root# iscsiadm --mode node --targetname iqn.2010-10.org.openstack:volume-cliu8-test --portal 16.158.166.197:3260 --login
Logging in to [iface: default, target: iqn.2010-10.org.openstack:volume-cliu8-test, portal: 16.158.166.197,3260] (multiple)
Login to [iface: default, target: iqn.2010-10.org.openstack:volume-cliu8-test, portal: 16.158.166.197,3260] successful.

这个时候fdisk可以看到disk已经出现在host上了

Disk /dev/sdf: 15.0 GB, 15032385536 bytes
64 heads, 32 sectors/track, 14336 cylinders, total 29360128 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/sdf doesn't contain a valid partition table

将这个disk attach到虚拟机上

root:~# virsh attach-disk instance-0000000b --source /dev/mapper/cinder--volumes-lv_cliu8_test --target vdc
Disk attached successfully

ssh到虚拟机里面，发现又多了一个disk

root@myinstancewithvolume:/home/ubuntu# fdisk -l

Disk /dev/vda: 21.5 GB, 21474836480 bytes
4 heads, 32 sectors/track, 327680 cylinders, total 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000c917a

   Device Boot      Start         End      Blocks   Id  System
/dev/vda1   *        2048    41943039    20970496   83  Linux

Disk /dev/vdb: 11.8 GB, 11811160064 bytes
10 heads, 25 sectors/track, 92274 cylinders, total 23068672 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x72ab3546

   Device Boot      Start         End      Blocks   Id  System
/dev/vdb1            2048    23068671    11533312   83  Linux

Disk /dev/vdc: 15.0 GB, 15032385536 bytes
16 heads, 63 sectors/track, 29127 cylinders, total 29360128 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/vdc doesn't contain a valid partition table