install
[root@centos8 ~]#virt-install --virt-type kvm --name centos7 --ram 1024 --vcpus 2 --cdrom=/data/isos/CentOS-7-x86_64-Minimal-1511.iso --disk path=/var/lib/libvirt/images/centos7.qcow2 --network network=default --graphics vnc,listen=0.0.0.0 --noautoconsole --os-variant=centos7.0
Starting install...
Domain installation still in progress. You can reconnect to
the console to complete the installation process.
#翻译:域安装仍在进行中。你可以重新连接完成控制台的安装过程。
#可以看到虚拟机的磁盘文件大小正在增长
[root@centos8 ~]#ll /var/lib/libvirt/images/centos7.qcow2
-rw-r--r-- 1 qemu qemu 1088290816 Oct 20 13:05 /var/lib/libvirt/images/centos7.qcow2
[root@centos8 ~]#ll /var/lib/libvirt/images/centos7.qcow2
-rw-r--r-- 1 qemu qemu 1101725696 Oct 20 13:05 /var/lib/libvirt/images/centos7.qcow2
kvm使用kickstart安装
配置kickstart应答文件
[root@centos8 ~]#yum install httpd -y && systemctl enable --now httpd
#挂载光盘镜像
[root@centos8 ~]#mount /data/isos/CentOS-7-x86_64-Minimal-1511.iso /var/www/html/centos/7/os/x86_64/
mount: /var/www/html/centos/7/os/x86_64: WARNING: device write-protected, mounted read-only.
[root@centos8 ~]#mount /data/isos/CentOS-8.4.2105-x86_64-dvd1.iso /var/www/html/centos/8/os/x86_64/
mount: /var/www/html/centos/8/os/x86_64: WARNING: device write-protected, mounted read-only.
[root@centos8 ~]#tree /var/www/html/ -d
/var/www/html/
├── centos
│ ├── 7
│ │ └── os
│ │ └── x86_64
│ │ ├── EFI
│ │ │ └── BOOT
│ │ │ └── fonts
│ │ ├── images
│ │ │ └── pxeboot
│ │ ├── isolinux
│ │ ├── LiveOS
│ │ ├── Packages
│ │ └── repodata
│ └── 8
│ └── os
│ └── x86_64
└── ks
19 directories
[root@centos8 ~]#tree /var/www/html/ks/
/var/www/html/ks/
├── centos7.cfg
└── ks8.cfg
0 directories, 2 files
[root@centos8 ~]#qemu-img create -f qcow2 /var/lib/libvirt/images/centos8.qcow2 20G
Formatting '/var/lib/libvirt/images/centos8.qcow2', fmt=qcow2 size=21474836480 cluster_size=65536 lazy_refcounts=off refcount_bits=16
[root@centos8 ~]#ll /var/lib/libvirt/images/centos8.qcow2 -h
-rw-r--r-- 1 root root 193K Oct 20 13:12 /var/lib/libvirt/images/centos8.qcow2
测试httpd服务正常
[root@centos8 ~]#curl -I 10.0.0.5/ks/ks8.cfg
HTTP/1.1 200 OK
Date: Wed, 20 Oct 2021 05:24:05 GMT
Server: Apache/2.4.37 (centos)
Last-Modified: Sat, 28 Aug 2021 01:57:46 GMT
ETag: "5f8-5ca94ed22ba80"
Accept-Ranges: bytes
Content-Length: 1528
Content-Type: text/plain; charset=UTF-8
创建kvm虚拟机磁盘文件
[root@centos8 images]#qemu-img create -f qcow2 /var/lib/libvirt/images/centos8.qcow2 40G
Formatting '/var/lib/libvirt/images/centos8.qcow2', fmt=qcow2 size=42949672960 cluster_size=65536 lazy_refcounts=off refcount_bits=16
创建kvm虚拟机
[root@centos8 images]#virt-install --virt-type kvm --name centos8-vm1 --ram 2048 --vcpus 2 --disk path=/var/lib/libvirt/images/centos8.qcow2 --network network=default --graphics vnc,listen=0.0.0.0 --location=/data/isos/CentOS-8.4.2105-x86_64-dvd1.iso --extra-args="ks=http://10.0.0.5/ks/ks8.cfg"
WARNING No operating system detected, VM performance may suffer. Specify an OS with --os-variant for optimal results.
Starting install...
Retrieving file vmlinuz... | 9.6 MB 00:00:00
Retrieving file initrd.img... | 72 MB 00:00:00
(virt-viewer:11777): gtk-vnc-WARNING **: 22:17:20.531: Unknown X11 keycode mapping '<null>'.
Please report to gtk-vnc-list@gnome.org
including the following information:
配置kickstart应答文件地址
使用VNC连接kvm虚拟机需要安装openssh-askpass
[root@centos8 ~]#yum info openssh-askpass
Last metadata expiration check: 1:38:10 ago on Sat 23 Oct 2021 03:45:29 PM CST.
Available Packages
Name : openssh-askpass
Version : 8.0p1
Release : 6.el8_4.2
Architecture : x86_64
Size : 92 k
Source : openssh-8.0p1-6.el8_4.2.src.rpm
Repository : AppStream
Summary : A passphrase dialog for OpenSSH and X
URL : http://www.openssh.com/portable.html
License : BSD
Description : OpenSSH is a free version of SSH (Secure SHell), a program for logging
: into and executing commands on a remote machine. This package contains
: an X11 passphrase dialog for OpenSSH.
后备差异虚拟磁盘
[root@centos8 images]#qemu-img info centos7.qcow2
image: centos7.qcow2
file format: qcow2
virtual size: 20 GiB (21474836480 bytes)
disk size: 1.05 GiB
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
corrupt: false
[root@centos8 images]#qemu-img create -f qcow2 -o backing_file=centos7.qcow2 centos7-test1.qcow2
Formatting 'centos7-test1.qcow2', fmt=qcow2 size=21474836480 backing_file=centos7.qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16
[root@centos8 images]#qemu-img info centos7-test1.qcow2
image: centos7-test1.qcow2
file format: qcow2
virtual size: 20 GiB (21474836480 bytes)
disk size: 196 KiB
cluster_size: 65536
backing file: centos7.qcow2
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
corrupt: false
[root@centos8 images]#virt-install --import --name=centos7-test1 --vcpus=2 --ram=2048 --disk path=/var/lib/libvirt/images/centos7-test1.qcow2 --network network=default --graphics vnc,listen=0.0.0.0 --os-type=linux --os-variant=centos7
ERROR Unknown OS name 'centos7'. See `osinfo-query os` for valid values.
报错提示os variant格式写错了
解决:
[root@centos8 images]#osinfo-query os | grep centos
centos-stream8 | CentOS Stream 8 | 8 | http://centos.org/centos-stream/8
centos5.0 | CentOS 5.0 | 5.0 | http://centos.org/centos/5.0
centos5.1 | CentOS 5.1 | 5.1 | http://centos.org/centos/5.1
centos5.10 | CentOS 5.10 | 5.10 | http://centos.org/centos/5.10
centos5.11 | CentOS 5.11 | 5.11 | http://centos.org/centos/5.11
centos5.2 | CentOS 5.2 | 5.2 | http://centos.org/centos/5.2
centos5.3 | CentOS 5.3 | 5.3 | http://centos.org/centos/5.3
centos5.4 | CentOS 5.4 | 5.4 | http://centos.org/centos/5.4
centos5.5 | CentOS 5.5 | 5.5 | http://centos.org/centos/5.5
centos5.6 | CentOS 5.6 | 5.6 | http://centos.org/centos/5.6
centos5.7 | CentOS 5.7 | 5.7 | http://centos.org/centos/5.7
centos5.8 | CentOS 5.8 | 5.8 | http://centos.org/centos/5.8
centos5.9 | CentOS 5.9 | 5.9 | http://centos.org/centos/5.9
centos6.0 | CentOS 6.0 | 6.0 | http://centos.org/centos/6.0
centos6.1 | CentOS 6.1 | 6.1 | http://centos.org/centos/6.1
centos6.10 | CentOS 6.10 | 6.10 | http://centos.org/centos/6.10
centos6.2 | CentOS 6.2 | 6.2 | http://centos.org/centos/6.2
centos6.3 | CentOS 6.3 | 6.3 | http://centos.org/centos/6.3
centos6.4 | CentOS 6.4 | 6.4 | http://centos.org/centos/6.4
centos6.5 | CentOS 6.5 | 6.5 | http://centos.org/centos/6.5
centos6.6 | CentOS 6.6 | 6.6 | http://centos.org/centos/6.6
centos6.7 | CentOS 6.7 | 6.7 | http://centos.org/centos/6.7
centos6.8 | CentOS 6.8 | 6.8 | http://centos.org/centos/6.8
centos6.9 | CentOS 6.9 | 6.9 | http://centos.org/centos/6.9
centos7.0 | CentOS 7 | 7 | http://centos.org/centos/7.0
centos8 | CentOS 8 | 8 | http://centos.org/centos/8
[root@centos8 images]#virt-install --import --name=centos7-test1 --vcpus=2 --ram=2048 --disk path=/var/lib/libvirt/images/centos7-test1.qcow2 --network network=default --graphics vnc,listen=0.0.0.0 --os-type=linux --os-variant=centos7.0
Starting install...
(virt-viewer:13763): gtk-vnc-WARNING **: 17:32:04.854: Unknown X11 keycode mapping '<null>'.
Please report to gtk-vnc-list@gnome.org
including the following information:
- Operating system
- GDK build
- X11 Server
- xprop -root
- xdpyinfo
测试后备差异虚拟磁盘依赖关系
说明:上面的后备差异虚拟磁盘基于centos7.qcow2
磁盘文件
[root@centos8 images]#virsh list --all
Id Name State
--------------------------------
1 centos7-test1 running
- centos7 shut off
- centos7-2 shut off
- centos8 shut off
- centos8-2 shut off
[root@centos8 images]#virsh destroy 1
Domain 1 destroyed
修改基础镜像文件
[root@centos8 images]#mv centos7.qcow2 centos7.qcow2.bak
再次开机
[root@centos8 images]#virsh start centos7-test1
error: Failed to start domain centos7-test1
error: Cannot access backing file '/var/lib/libvirt/images/centos7.qcow2' of storage file '/var/lib/libvirt/images/centos7-test1.qcow2' (as uid:107, gid:107): No such file or directory
回复基础镜像文件,再次启动ok
[root@centos8 images]#mv centos7.qcow2.bak centos7.qcow2
[root@centos8 images]#virsh start centos7-test1
Domain centos7-test1 started
kvm克隆lvm卷提示空间不足
[root@centos8 ~]#virsh vol-clone lvvol1 lvvol1_clone vm_images_lvm
error: Failed to clone vol from lvvol1
error: internal error: Child process (/usr/sbin/lvcreate --name lvvol1_clone -L 10485760K vm_images_lvm) unexpected exit status 5: Volume group "vm_images_lvm" has insufficient free space (2559 extents): 2560 required.
清空磁盘的前100兆空间
用于删除掉磁盘经过使用后留下的一些看不到的数据,造成kvm挂载lvm卷组失败
dd if=/dev/zero of=/dev/sdb bs=1M count=100
网桥
系统版本信息
[root@localhost network-scripts]#cat /etc/redhat-release
CentOS Linux release 7.9.2009 (Core)
[root@localhost network-scripts]#uname -r
3.10.0-1160.el7.x86_64
创建网桥并加入物理网卡
[root@localhost network-scripts]#nmcli con add type bridge con-name virbr1 ifname virbr1 ipv4.method manual ipv4.addresses 10.0.0.182/24 ipv4.gateway 10.0.0.2
Connection 'virbr1' (b4383b56-8623-41ea-bae9-96cf804d2459) successfully added.
[root@localhost network-scripts]#nmcli con
NAME UUID TYPE DEVICE
ens33 554f0ed4-9978-4844-85ee-97e4d68b5401 ethernet ens33
virbr0 ae58c4bd-50c8-4a41-bbf2-c792e99e5a9a bridge virbr0
virbr1 b4383b56-8623-41ea-bae9-96cf804d2459 bridge virbr1
[root@localhost network-scripts]#nmcli con add con-name virbr1-port0 ifname ens33 type bridge-slave master virbr1
Connection 'virbr1-port0' (56aa090d-e4a1-4db3-b475-d7e817a5d8aa) successfully added.
[root@localhost network-scripts]#nmcli con
NAME UUID TYPE DEVICE
ens33 554f0ed4-9978-4844-85ee-97e4d68b5401 ethernet ens33
virbr0 ae58c4bd-50c8-4a41-bbf2-c792e99e5a9a bridge virbr0
virbr1 b4383b56-8623-41ea-bae9-96cf804d2459 bridge virbr1
virbr1-port0 56aa090d-e4a1-4db3-b475-d7e817a5d8aa ethernet --
[root@localhost network-scripts]#nmcli con reload
[root@localhost network-scripts]#nmcli con up virbr1
Connection successfully activated (master waiting for slaves) (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/6)
[root@localhost network-scripts]#nmcli con up virbr1-port0
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/7)
网桥配置文件
[root@localhost network-scripts]#cat ifcfg-virbr1
STP=yes
BRIDGING_OPTS=priority=32768
TYPE=Bridge
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
IPADDR=10.0.0.182
PREFIX=24
GATEWAY=10.0.0.2
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=virbr1
UUID=b4383b56-8623-41ea-bae9-96cf804d2459
DEVICE=virbr1
ONBOOT=yes
[root@localhost network-scripts]#cat ifcfg-virbr1-port0
TYPE=Ethernet
NAME=virbr1-port0
UUID=56aa090d-e4a1-4db3-b475-d7e817a5d8aa
DEVICE=ens33
ONBOOT=yes
BRIDGE=virbr1
nfs相关
yum install nfs-utils -y
systemctl enable --now nfs-server
#查看当前共享的文件夹详细信息
[root@centos8 ~]#exportfs -v
/data/kvmdata <world>(sync,wdelay,hide,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash)
[root@centos8 ~]#cat /etc/exports
/data/kvmdata *(rw,no_root_squash)
案例:连接NAS共享存储实现虚拟机实时迁移
环境:三台主机
两台kvm宿主机
- 一台centos8:10.0.0.5
- 一台centos7:10.0.0.182
一台NFS存储服务器
- 10.0.0.7
注意:
两台宿主机的虚拟机都要一样的网卡配置,即:桥接到virbr1网卡
要迁移的虚拟机必须是运行状态才可以迁移
这里是低版本向高版本进行迁移,即centos7的虚拟机向centos8中迁移
两台宿主机的selinux都处于disabled
状态
配置NFS共享存储
[root@centos8 ~]#dnf install -y nfs-utils
[root@centos8 ~]#mkdir /data/kvmdata
[root@centos8 ~]#cat /etc/exports
[root@centos8 ~]#echo "/data/kvmdata *(rw,no_root_squash)" >> /etc/exports
[root@centos8 ~]#cat /etc/exports
/data/kvmdata *(rw,no_root_squash)
[root@centos8 ~]#systemctl enable --now nfs-service
[root@centos8 ~]#exportfs -v
/data/kvmdata <world>(sync,wdelay,hide,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash)
[root@centos8 ~]#ll /data/kvmdata/
total 0
两台宿主机网络配置:
centos8:
[root@centos8 ~]#ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master virbr1 state UP group default qlen 1000
link/ether 00:0c:29:c5:70:44 brd ff:ff:ff:ff:ff:ff
3: ens36: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:0c:29:c5:70:4e brd ff:ff:ff:ff:ff:ff
inet 192.168.10.5/24 brd 192.168.10.255 scope global noprefixroute ens36
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fec5:704e/64 scope link
valid_lft forever preferred_lft forever
4: virbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:0c:29:c5:70:44 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.5/24 brd 10.0.0.255 scope global noprefixroute virbr1
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fec5:7044/64 scope link
valid_lft forever preferred_lft forever
5: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
link/ether 52:54:00:86:ef:50 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
valid_lft forever preferred_lft forever
6: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master virbr0 state DOWN group default qlen 1000
link/ether 52:54:00:86:ef:50 brd ff:ff:ff:ff:ff:ff
cnetos7:
[root@localhost ~]#ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master virbr1 state UP group default qlen 1000
link/ether 00:0c:29:f7:9a:d1 brd ff:ff:ff:ff:ff:ff
3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
link/ether 52:54:00:0b:47:f0 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
valid_lft forever preferred_lft forever
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN group default qlen 1000
link/ether 52:54:00:0b:47:f0 brd ff:ff:ff:ff:ff:ff
5: virbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:0c:29:f7:9a:d1 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.182/24 brd 10.0.0.255 scope global noprefixroute virbr1
valid_lft forever preferred_lft forever
inet6 fe80::f283:2dbb:dd68:c0a9/64 scope link noprefixroute
valid_lft forever preferred_lft forever
虚拟机关机状态下把要进行迁移的虚拟机磁盘文件拷贝到NFS共享存储
[root@centos8 ~]#scp /var/lib/libvirt/images/centos8-pxe.qcow2 10.0.0.7:/data/kvmdata/
root@10.0.0.7's password:
centos8-pxe.qcow2 100% 2054MB 202.8MB/s 00:10
[root@localhost ~]#ip addr show virbr1
5: virbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:0c:29:f7:9a:d1 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.182/24 brd 10.0.0.255 scope global noprefixroute virbr1
valid_lft forever preferred_lft forever
inet6 fe80::f283:2dbb:dd68:c0a9/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@localhost ~]#scp /var/lib/libvirt/images/centos8.qcow2 10.0.0.7:/data/kvmdata/
The authenticity of host '10.0.0.7 (10.0.0.7)' can't be established.
ECDSA key fingerprint is SHA256:qCLXNgu1TNZWaOLs9l5oEJeY/ZFJlAXDLrd3Korzops.
ECDSA key fingerprint is MD5:30:58:31:21:39:99:3c:fc:65:1e:42:50:79:ad:7e:ac.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.0.0.7' (ECDSA) to the list of known hosts.
root@10.0.0.7's password:
centos8.qcow2 100% 2058MB 161.7MB/s 00:12
查看NFS共享存储器中的内容
[root@centos8 ~]#hostname -I
10.0.0.7
[root@centos8 ~]#ls /data/kvmdata/centos8.qcow2 -lh
-rw-r--r-- 1 root root 2.1G Nov 14 22:32 /data/kvmdata/centos8.qcow2
将两台宿主机下的其他虚拟机文件复制到临时目录,原因:挂载NFS存储时会将虚拟机的默认存放磁盘文件的文件夹覆盖
[root@centos8 ~]#mv /var/lib/libvirt/images/* /opt/
[root@localhost ~]#mv /var/lib/libvirt/images/* /opt/
挂载NFS存储池
命令行查看挂载情况
[root@centos8 ~]#ll /var/lib/libvirt/images/
total 4211200
-rw-r--r-- 1 root root 2154168320 Nov 14 2021 centos8-pxe.qcow2
-rw-r--r-- 1 root root 2158100480 Nov 14 2021 centos8.qcow2
[root@centos8 ~]#df
Filesystem 1K-blocks Used Available Use% Mounted on
devtmpfs 4033052 0 4033052 0% /dev
tmpfs 4062464 0 4062464 0% /dev/shm
tmpfs 4062464 9668 4052796 1% /run
tmpfs 4062464 0 4062464 0% /sys/fs/cgroup
/dev/sda1 104806400 41170004 63636396 40% /
/dev/sda2 52403200 31695260 20707940 61% /data
/dev/sda5 999320 189500 741008 21% /boot
10.0.0.7:/data/kvmdata 52403200 4613120 47790080 9% /var/lib/libvirt/images
[root@localhost ~]#df
Filesystem 1K-blocks Used Available Use% Mounted on
devtmpfs 3984020 0 3984020 0% /dev
tmpfs 3995032 0 3995032 0% /dev/shm
tmpfs 3995032 11936 3983096 1% /run
tmpfs 3995032 0 3995032 0% /sys/fs/cgroup
/dev/sda2 104806400 4406356 100400044 5% /
/dev/sda3 48803552 32992 48770560 1% /data
/dev/sda1 944120 115080 763864 14% /boot
10.0.0.7:/data/kvmdata 52403200 8092416 44310784 16% /var/lib/libvirt/images/kvmdata
tmpfs 799008 4 799004 1% /run/user/0
将缓存目录中的文件移动至虚拟机默认存放目录
[root@localhost ~]#mv /opt/* /var/lib/libvirt/images/
[root@centos8 ~]#mv /opt/* /var/lib/libvirt/images/
两台宿主机启动虚拟机
[root@centos8 ~]#virsh start centos8pxe
[root@centos8 ~]#virsh list --all
Id Name State
--------------------------------
1 centos8pxe running
- centos7 shut off
- centos7-2 shut off
- centos7-test1 shut off
- centos8-3 shut off
[root@localhost ~]#virsh start centos8
virt-manager新建连接
提示要经过ssh验证连接
安装openssh-askpass
yum install openssh-askpass -y
开始迁移 selinux如果未关闭,这里会提示你selinux权限不允许
迁移后的界面如图
再迁移回来
迁移过程中可以用物理机ping虚拟机,检测迁移过程中网络状况