使用CBSD克隆复制Ubuntu bhyve子系统环境（未成功）@FreeBSD

创建bhyve Ubuntu虚拟机子系统见：CBSD创建和管理bhyve容器Ubuntu@FreeBSD_bhyve管理器-CSDN博客

CBSD官网：CBSD — FreeBSD Jail and Bhyve Management Tools

想快速复制Ubuntu系统，在一台FreeBSD机器有了bhyve Ubuntu子系统后，想把这个子系统复制到另外一台服务器。发现可以使用cbsd bexport和bimport导出和导入。

导出子系统

首先看一下有哪些bhyve虚拟子系统

先用cbsd bls看看有哪些bhyve虚拟子系统

cbsd bls
JNAME  JID   VM_RAM  VM_CURMEM  VM_CPUS  PCPU  VM_OS_TYPE  IP4_ADDR      STATUS  VNC
ub12   1280  2048    0          2        0     linux       192.168.1.12  On      0.0.0.0:5900  

导出bhyve虚拟子系统

使用cbsd bexport命令导出

注意，联机时不可以导出。所以如果没有关机，需要首先关闭子系统

关闭子系统

cbsd bstop jname=ub12
Send SIGTERM to ub12. Soft timeout is 30 sec. 1 seconds left [............................Send SIGTERM to ub12. Soft timeout is 30 sec. 0 seconds left [..............................]
bstop done in 10 seconds

导出子系统

cbsd bexport jname=ub12
Exporting (with compress level:6), please stand by: ub12

不过因为原来的系统异乎寻常的慢，所以导出子系统也是特别慢

总共用了13分钟，还可以接受

cbsd bexport jname=ub12
Exporting (with compress level:6), please stand by: ub12
WIP: [0%...100%]

20480+1 records in
20480+1 records out
21474967552 bytes transferred in 816.853627 secs (26289860 bytes/sec)
environment flat size: 20g, images size ub12.img: 6g
exported image file: /usr/jails/export/ub12.img
jexport done in 13 minutes and 42 seconds

 将导出的系统cp到新系统

前面导出到/usr/jails/export/ub12.img ，在新系统中scp过来

scp skywalk@192.168.1.5:/usr/jails/export/ub12.img .

导入子系统

在另一台机器使用cbsd bimport导入子系统

cbsd bimport jname=/tmp/debian1.img

可以看到只需要3分钟，速度还是挺快的

root@fb14:/usr/jails/export # cbsd bimport ub12.img
Importing image, please stand by: ub12
CBSD Image, version: 4
Image was created on node: fb5
Image was created at: 20240815
Image size: 6g
Environment flat size after extracting: 20g
Compress level: 6
WIP: [0%...100%]
ZVOLs list: dsk1
 * dsk1 (20g): WIP: [0%...13%...21%...35%...100%]
Hostname sets to: ub12.my.domain
jimport done in 3 minutes and 41 seconds

启动子系统

cbsd bstart ub12
init_systap: waiting for link: vmx0
VRDP is enabled. VNC bind/port: 192.168.1.250:5900
For attach VM console, use: vncviewer 192.168.1.250:5900
Resolution: 800x600.
VNC pass: cbsd

Warning!!! You are running a system with open VNC port to the world wich is not secure
Please use IP filter or balancer with password to restrict VNC port access
Or change vnc_bind params to 127.0.0.1 and reboot VM after maintenance work

bhyve renice: 1
Waiting for PID..........
PID: 0
Thu Aug 15 21:11:27 CST 2024
cmd: env LIB9P_LOGGING=/usr/jails/jails-system/ub12/cbsd_lib9p.log /usr/bin/nice -n 1 /usr/sbin/bhyve  -c cpus=2,sockets=2,cores=1,threads=1 -m 2147483648 -H -A -U 6b143aa7-4afb-11ef-935b-00e2691367f6 -s 0,hostbridge  -s 1,ahci-cd,/usr/local/cbsd/upgrade/patch/efirefd.fd,ro -s 7,virtio-blk,/usr/jails/jails-data/ub12-data/dsk1.vhd,sectorsize=512/4096    -s 5,virtio-net,tap2,mtu=1500,mac=00:a0:98:31:84:3b  -s 6,virtio-rnd  -s 9,fbuf,tcp=0.0.0.0:5900,w=800,h=600,password=cbsd -s 30,xhci,tablet  -s 31,lpc -l com1,stdio -l bootrom,/usr/local/cbsd/upgrade/patch/efi.fd,/usr/jails/jails-system/ub12/BHYVE_UEFI_VARS.fd   ub12
-----
vm_create: Device not configured
Please use for debug: /usr/local/cbsd/share/bhyverun.sh -c /usr/jails/jails-system/ub12/bhyve.conf
bstart done in 15 seconds
root@fb14:/usr/jails/export # cbsd bls
JNAME  JID  VM_RAM  VM_CURMEM  VM_CPUS  PCPU  VM_OS_TYPE  IP4_ADDR      STATUS  VNC
ub12   0    2048    0          2        0     linux       192.168.1.12  Off     0.0.0.0:5900  

没有启动起来，还在查找原因中。 

设置pf防火墙

需要设置防火墙，放开5900端口

先cp配置文件

cp /usr/share/examples/pf/pf.conf /etc/pf.conf

编辑/etc/pf.conf文件，加入：

pass out all keep state
tcp_services = "{ ssh, smtp, domain, www, pop3, auth, pop3s, 8080, 5900 }"
block in all
pass in proto tcp to any port $tcp_services keep state

 

 

调试

创建bhyve的时候报错vmm没有load

cbsd bconstruct-tui
Warning: Your current environment is not physical. This is a requirement for a working bhyve
Virtual engine detected: vmware
Please refer to this page for more information: https://wiki.freebsd.org/bhyve
Try to run bhyve anyway...
To disable this warning, please set in /usr/jails/etc/bhyve-default-default.conf: skip_bhyve_init_virtual_warning to: 1
Pause for 10 seconds
No kldloaded module: vmm
Please add vmm_load="YES" to /boot/loader.conf and
put kld_list="vmm if_tuntap if_bridge nmdm" into your /etc/rc.conf then reboot the host. Or, for example, run:
 sysrc kld_list+="vmm if_tuntap if_bridge nmdm" && service kld restart
root@fb14:~ # kldstat |grep vmm
 2    1 0xffffffff81f35000   37c540 vmm.ko
 

说没有vmm，但是kldstat里面显示有vmm啊！

 按照提示，把skip_bhyve_init_virtual_warning=1 写入/usr/jails/etc/bhyve-default-default.conf ，vmm的问题解决。

创建bhyve提示Your current environment is not physical.

cbsd bconstruct-tui
Warning: Your current environment is not physical. This is a requirement for a working bhyve
Virtual engine detected: vmware
Please refer to this page for more information: https://wiki.freebsd.org/bhyve
Try to run bhyve anyway...
To disable this warning, please set in /usr/jails/etc/bhyve-default-default.conf: skip_bhyve_init_virtual_warning to: 1

  按照提示，把skip_bhyve_init_virtual_warning=1 写入/usr/jails/etc/bhyve-default-default.conf ，问题解决。

vnc也设置了，pf也打开了，但是5900连不上

发现主机的5900端口根本就没开

原来是虚拟机没有启动成功

cbsd bstart ub12虚拟机启动报错

不管是复制的ub12,还是新create的ub22,都是报错

cmd: env LIB9P_LOGGING=/usr/jails/jails-system/ub22/cbsd_lib9p.log /usr/bin/nice -n 1 /usr/sbin/bhyve  -c cpus=6,sockets=6,cores=1,threads=1 -m 4294967296 -H -A -U ceae361c-5b0d-11ef-a876-000c29ff148f -s 0,hostbridge   -s 4,virtio-blk,/usr/jails/vm/ub22/dsk1.vhd,sectorsize=512/4096   -s 2,ahci-cd,/usr/jails/src/iso/cbsd-iso-ubuntu-22.04.3-live-server-amd64.iso,ro  -s 5,virtio-net,tap2,mtu=1500,mac=00:a0:98:68:0a:de  -s 6,virtio-rnd  -s 7,fbuf,tcp=127.0.0.1:5901,w=800,h=600,wait,password=cbsd -s 30,xhci,tablet  -s 31,lpc -l com1,stdio -l bootrom,/usr/local/cbsd/upgrade/patch/efi.fd,/usr/jails/jails-system/ub22/BHYVE_UEFI_VARS.fd   ub22