本篇文章手把手搭建一下达梦 8 中的 RAC 集群,也就是 DSC 集群。
主要涉及以下步骤:
1 环境说明
此次搭建使用了两个节点,分别为DSC1,DSC2。通过vmware workstation部署两台CentOS 7.6。
其中存储部分,操作系统安装规划了50G空间,另外的两个1G和两个10G的盘,作为vote,dcr,dmdata,dmlog。
2 安装软件
这部分两台主机只安装软件即可,不用初始化数据库。具体安装软件的步骤可以参考:
达梦 8 数据库的安装 —— 适配生产环境 Linux 平台_达梦数据库8.1-CSDN博客
3 配置存储
3.1 创建虚拟磁盘
3.1.1 找到vmware-vdiskmanager.exe程序文件
进入到安装vmware的目录,找到vmware-vdiskmanager.exe程序文件。
windows 中按 windows 键 + R 输入cmd,可以调出doc运行界面。
比如我的电脑中 vmware-vdiskmanager.exe 在 c://【soft】目录,可以在 doc 中输入:
切换目录到c盘:
C:\Users\Administrator>cd c://
切换到【soft】,由于带特殊符号【】所以要加双引号。
如果没有特殊符号,直接写目录即可,比如:c:\>cd ESD
c:\>cd "【soft】"
查看当前目录下的文件
c:\【soft】>dir
3.1.2 创建虚拟磁盘
c:\【soft】>vmware-vdiskmanager.exe -c -s 1Gb -a lsilogic -t 2 "d:\dscshare\dscshare_1G_1.vmdk"
Creating disk 'd:\dscshare\dscshare_1G_1.vmdk'
Create: 100% done.
Virtual disk creation successful.
c:\【soft】>vmware-vdiskmanager.exe -c -s 1Gb -a lsilogic -t 2 "d:\dscshare\dscshare_1G_2.vmdk"
Creating disk 'd:\dscshare\dscshare_1G_2.vmdk'
Create: 100% done.
Virtual disk creation successful.
c:\【soft】>vmware-vdiskmanager.exe -c -s 10Gb -a lsilogic -t 2 "d:\dscshare\dscshare_10G_1.vmdk"
Creating disk 'd:\dscshare\dscshare_10G_1.vmdk'
Create: 100% done.
Virtual disk creation successful.
c:\【soft】>vmware-vdiskmanager.exe -c -s 10Gb -a lsilogic -t 2 "d:\dscshare\dscshare_10G_2.vmdk"
Creating disk 'd:\dscshare\dscshare_10G_2.vmdk'
Create: 100% done.
Virtual disk creation successful.
3.2 修改虚拟机配置文件
3.2.1 添加已有磁盘
关闭虚拟机,分别将以上创建的四块盘按照已有磁盘添加到虚拟机中。
添加之后,每块盘的scsi号可以通过虚拟机设置---》硬盘---》高级选项查看。
3.2.2 修改虚拟机配置文件
分别修改两台主机的虚拟机vmx配置文件做如下操作,注意每块盘的scsi号:
scsi0:1.mode = "independent-persistent"
scsi0:1.SharedBus = "Virtual"
scsi0:2.mode = "independent-persistent"
scsi0:2.SharedBus = "Virtual"
scsi0:3.mode = "independent-persistent"
scsi0:3.SharedBus = "Virtual"
scsi0:4.mode = "independent-persistent"
scsi0:4.SharedBus = "Virtual"
disk.locking = "FALSE"
disk.EnableUUID = "TRUE"
3.2.3 查看挂载后的磁盘
两台主机上均操作。
[root@dm1 ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 80G 0 disk
├─sda1 8:1 0 4G 0 part [SWAP]
└─sda2 8:2 0 76G 0 part /
sdb 8:16 0 1G 0 disk
sdc 8:32 0 1G 0 disk
sdd 8:48 0 10G 0 disk
sde 8:64 0 10G 0 disk
sr0 11:0 1 4.3G 0 rom
3.2.4 配置共享存储绑定UDEV
两台主机均操作,查看设备的scsi_id:
[root@dm1 ~]# /usr/lib/udev/scsi_id -g -u /dev/sdb
36000c293c6b32ba6f6095100224039d5
[root@dm1 ~]# /usr/lib/udev/scsi_id -g -u /dev/sdc
36000c296400b243289a2b83558164369
[root@dm1 ~]# /usr/lib/udev/scsi_id -g -u /dev/sdd
36000c29632fed860dc461f8204756dc9
[root@dm1 ~]# /usr/lib/udev/scsi_id -g -u /dev/sde
36000c299c2adce57e7d2845fdadd8f5b
[root@dm1 ~]#
3.2.5 修改配置文件:
[root@dm1 ~]# cd /etc/udev/rules.d/
[root@dm1 rules.d]# vi 66-dmdevices.rules
## DCR磁盘配置,且在软链接之前创建文件夹 /dev_DSC
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name",RESULT=="36000c293c6b32ba6f6095100224039d5",SYMLINK+="DCR", OWNER="dmdba", GROUP="dinstall", MODE="0660", RUN+="/bin/sh -c 'chown dmdba:dinstall /dev/$name;mkdir -p /dev_DSC; ln -s /dev/DCR /dev_DSC/DCR'"
## VOTE 磁盘配置
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name",RESULT=="36000c296400b243289a2b83558164369",SYMLINK+="VOTE", OWNER="dmdba", GROUP="dinstall", MODE="0660", RUN+="/bin/sh -c 'chown dmdba:dinstall /dev/$name; ln -s /dev/VOTE /dev_DSC/VOTE'"
## DMDATA 磁盘配置
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name",RESULT=="36000c29632fed860dc461f8204756dc9",SYMLINK+="DMDATA", OWNER="dmdba", GROUP="dinstall", MODE="0660", RUN+="/bin/sh -c 'chown dmdba:dinstall /dev/$name; ln -s /dev/DMDATA /dev_DSC/DMDATA'"
## DMLOG 磁盘配置,且在搭建完成之后,将权限直接赋予 dmdba组的dmdba用户
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name",RESULT=="36000c299c2adce57e7d2845fdadd8f5b",SYMLINK+="DMLOG", OWNER="dmdba", GROUP="dinstall", MODE="0660", RUN+="/bin/sh -c 'chown dmdba:dinstall /dev/$name; ln -s /dev/DMLOG /dev_DSC/DMLOG ; chown -R dmdba:dinstall /dev_DSC'"
3.2.6 重启服务生效
[root@dm1 rules.d]# systemctl restart systemd-udev-trigger.service
[root@dm1 dev]# ls -lth /dev_DSC
total 0
[root@dm1 dev]# ll /dev_DSC
total 0
lrwxrwxrwx. 1 dmdba dinstall 8 Jul 1 16:50 DCR -> /dev/DCR
lrwxrwxrwx. 1 dmdba dinstall 11 Jul 1 16:50 DMDATA -> /dev/DMDATA
lrwxrwxrwx. 1 dmdba dinstall 10 Jul 1 16:50 DMLOG -> /dev/DMLOG
lrwxrwxrwx. 1 dmdba dinstall 9 Jul 1 16:58 VOTE -> /dev/VOTE
4 dmdcr_cfg.ini配置文件配置
dmdcr_cfg.ini存放于服务器DM1的/home/dmdba/dmdsc/data/DSC1
DCR_N_GRP= 3
DCR_VTD_PATH=/dev_DSC/VOTE
DCR_OGUID= 1234
[GRP]
DCR_GRP_TYPE = CSS
DCR_GRP_NAME = GRP_CSS
DCR_GRP_N_EP = 2
DCR_GRP_DSKCHK_CNT = 60
[GRP_CSS]
DCR_EP_NAME = CSS1
DCR_EP_HOST = 192.168.6.224
DCR_EP_PORT = 5230
[GRP_CSS]
DCR_EP_NAME = CSS2
DCR_EP_HOST = 192.168.6.225
DCR_EP_PORT = 5230
[GRP]
DCR_GRP_TYPE= ASM
DCR_GRP_NAME= GRP_ASM
DCR_GRP_N_EP= 2
DCR_GRP_DSKCHK_CNT= 60
[GRP_ASM]
DCR_EP_NAME= ASM1
DCR_EP_SHM_KEY= 64735
DCR_EP_SHM_SIZE= 512
DCR_EP_HOST= 192.168.6.224
DCR_EP_PORT= 5231
DCR_EP_ASM_LOAD_PATH= /dev_DSC
[GRP_ASM]
DCR_EP_NAME= ASM2
DCR_EP_SHM_KEY= 54736
DCR_EP_SHM_SIZE= 512
DCR_EP_HOST= 192.168.6.225
DCR_EP_PORT= 5231
DCR_EP_ASM_LOAD_PATH= /dev_DSC
[GRP]
DCR_GRP_TYPE= DB
DCR_GRP_NAME= GRP_DSC
DCR_GRP_N_EP= 2
DCR_GRP_DSKCHK_CNT= 60
[GRP_DSC]
DCR_EP_NAME= DSC1
DCR_EP_SEQNO= 0
DCR_EP_PORT= 5232
[GRP_DSC]
DCR_EP_NAME= DSC2
DCR_EP_SEQNO= 1
DCR_EP_PORT= 5232
5 在DM1上使用dmasmcmd工具初始化磁盘
注:不要加分号,否则报语法错误
dmasmcmd
ASM>create dcrdisk '/dev_DSC/DCR' 'DCR'
[TRACE]The ASM initialize dcrdisk /dev_DSC/DCR to name DMASMDCR
Used time: 49.467(ms).
ASM>create votedisk '/dev_DSC/VOTE' 'VOTE'
[TRACE]The ASM initialize votedisk /dev_DSC/VOTE to name DMASMVOTE
Used time: 17.320(ms).
ASM>create asmdisk '/dev_DSC/DMDATA' 'DMDATA'
[TRACE]The ASM initialize asmdisk /dev_DSC/DMDATA to name DMASMDMDATA
Used time: 18.273(ms).
ASM>create asmdisk '/dev_DSC/DMLOG' 'DMLOG'
[TRACE]The ASM initialize asmdisk /dev_DSC/DMLOG to name DMASMDMLOG
Used time: 16.435(ms).
ASM>init dcrdisk '/dev_DSC/DCR' from '/home/dmdba/dmdsc/data/DSC1/dmdcr_cfg.ini' identified by 'SYSDBA'
[TRACE]DG 126 alloc extent for inode (0, 0, 1)
[TRACE]DG 126 alloc 4 extents for 0xfe000002 (0, 0, 2)->(0, 0, 5)
Used time: 62.422(ms).
ASM>init votedisk '/dev_DSC/VOTE' from '/home/dmdba/dmdsc/data/DSC1/dmdcr_cfg.ini'
[TRACE]DG 125 alloc extent for inode (0, 0, 1)
[TRACE]DG 125 alloc 4 extents for 0xfd000002 (0, 0, 2)->(0, 0, 5)
Used time: 44.554(ms).
6 配置dmasvrmal.ini文件
保存在/home/dmdba/dmdsc/data/DSC1下,DM1和DM2的配置一致。
DM1和DM2的配置一样
[dmdba@dm1 DSC1]$ cat dmasvrmal.ini
[MAL_INST1]
MAL_INST_NAME=ASM1
MAL_HOST=192.168.6.224
MAL_PORT=5233
[MAL_INST2]
MAL_INST_NAME= ASM2
MAL_HOST=192.168.6.225
MAL_PORT= 5233
7 DMDCR.INI配置
DM1和DM2的配置仅DMDCR_SEQNO有区别,当前为手动拉起两个节点的ASM和DMSERVER服务。待集群配置完成后,再去掉注释符号##,修改为自动拉起
7.1 DM1节点配置
[dmdba@dm1 DSC1]$ cat dmdcr.ini
DMDCR_PATH=/dev_DSC/DCR
DMDCR_MAL_PATH=/home/dmdba/dmdsc/data/DSC1/dmasvrmal.ini
DMDCR_SEQNO=0
DMDCR_AUTO_OPEN_CHECK=111
DMDCR_ASM_TRACE_LEVEL =2
#DMDCR_ASM_RESTART_INTERVAL = 30 #CSS 认定 ASM 故障重启的时间
#DMDCR_ASM_STARTUP_CMD=/home/dmdba/dmdsc/bin/DmASMSvrServiceASM start
#DMDCR_DB_RESTART_INTERVAL= 60##CSS认定DSC 故障重启的时间,设置为 0 不自动拉起
#DMDCR_DB_STARTUP_CMD=/home/dmdba/dmdsc/bin/DmServiceDSC start
7.2 DM2节点配置
[dmdba@dm2 DSC2]$ cat dmdcr.ini
DMDCR_PATH=/dev_DSC/DCR
DMDCR_MAL_PATH=/home/dmdba/dmdsc/data/DSC2/dmasvrmal.ini
DMDCR_SEQNO=1
DMDCR_AUTO_OPEN_CHECK=111
DMDCR_ASM_TRACE_LEVEL =2
#DMDCR_ASM_RESTART_INTERVAL = 30 #CSS 认定 ASM 故障重启的时间
#DMDCR_ASM_STARTUP_CMD=/home/dmdba/dmdsc/bin/DmASMSvrServiceASM start
#DMDCR_DB_RESTART_INTERVAL= 60##CSS认定DSC 故障重启的时间,设置为 0 不自动拉起
#DMDCR_DB_STARTUP_CMD=/home/dmdba/dmdsc/bin/DmServiceDSC start
7.3 DM1和DM2两台机器分别启动dmcss
[dmdba@dm1 ~]$ dmcss dcr_ini=/home/dmdba/dmdsc/data/DSC1/dmdcr.ini
[dmdba@dm2 ~]$ dmcss dcr_ini=/home/dmdba/dmdsc/data/DSC2/dmdcr.ini
7.4 DM1和DM2两台机器分别启动asm
[dmdba@dm1 ~]$ dmasmsvr dcr_ini=/home/dmdba/dmdsc/data/DSC1/dmdcr.ini
[dmdba@dm2 ~]$ dmasmsvr dcr_ini=/home/dmdba/dmdsc/data/DSC2/dmdcr.ini
启动asm之后,可以在css看到asm服务启动的信息:
8 DM1创建ASM磁盘组
[dmdba@dm1 ~]$ dmasmtool dcr_ini=/home/dmdba/dmdsc/data/DSC1/dmdcr.ini
DMASMTOOL V8
[code : -11041] ASM connection exception
如上报错,解决方法是关闭DM1和DM2主机上的防火墙和SELINUX。
再次尝试DM1创建磁盘组:注:不能加分号,否则提示语法错误。
[dmdba@dm1 DSC1]$ ll /dev_DSC/DCR
lrwxrwxrwx. 1 dmdba dinstall 8 Jul 1 16:50 /dev_DSC/DCR -> /dev/DCR
[dmdba@dm1 DSC1]$ dmasmtool dcr_ini=/home/dmdba/dmdsc/data/DSC1/dmdcr.ini
DMASMTOOL V8
ASM>create diskgroup 'DMDATA' asmdisk '/dev_DSC/DMDATA'
Used time: 31.205(ms).
ASM>create diskgroup 'DMLOG' asmdisk '/dev_DSC/DMLOG'
Used time: 35.844(ms).
9 初始化数据库
9.1 DM1初始化
创建 dminit.ini 文件保存在/home/dmdba/data/DSC1上
/home/dmdba/dmdsc/data/DSC1
Dminit.ini内容:
DB_NAME= dsc
SYSDBA_PWD= SYSDBA_PWD
SYSTEM_PATH= +DMDATA/data
SYSTEM= +DMDATA/data/dsc/SYSTEM.dbf
SYSTEM_SIZE= 512
ROLL= +DMDATA/data/dsc/ROLL.dbf
ROLL_SIZE= 512
MAIN= +DMDATA/data/dsc/MAIN.dbf
MAIN_SIZE= 512
CTL_PATH= +DMDATA/data/dsc/dm.ctl
CTL_SIZE= 8
LOG_SIZE= 512
DCR_PATH= /dev_DSC/DCR
DCR_SEQNO= 0
AUTO_OVERWRITE= 2
PAGE_SIZE= 32
EXTENT_SIZE = 16
BLANK_PAD_MODE= 1
[DSC1]
CONFIG_PATH= /home/dmdba/dmdsc/data/DSC1
PORT_NUM= 5232
MAL_HOST= 192.168.6.224
MAL_PORT= 5235
LOG_PATH= +DMLOG/DSC1_LOG01.log
LOG_PATH= +DMLOG/DSC1_LOG02.log
[DSC2]
CONFIG_PATH= /home/dmdba/dmdsc/data/DSC2
PORT_NUM= 5232
MAL_HOST= 192.168.6.225
MAL_PORT= 5235
LOG_PATH= +DMLOG/DSC2_LOG01.log
LOG_PATH= +DMLOG/DSC2_LOG02.log
执行初始化:
[dmdba@dm1 DSC1]$ dminit control=/home/dmdba/dmdsc/data/DSC1/dminit.ini
initdb V8
db version: 0x7000c
file dm.key not found, use default license!
License will expire on 2025-03-21
Normal of FAST
Normal of DEFAULT
Normal of RECYCLE
Normal of KEEP
Normal of ROLL
log file path: +DMLOG/DSC1_LOG01.log
log file path: +DMLOG/DSC1_LOG02.log
log file path: +DMLOG/DSC2_LOG01.log
log file path: +DMLOG/DSC2_LOG02.log
write to dir [+DMDATA/data/dsc].
create dm database success. 2024-07-02 10:53:10
9.2 拷贝DM1生成的DSC2配置文件到DM2
把DM1主机中生成的DSC2的配置文件拷贝到DM2的DSC2目录
[dmdba@dm1 DSC2]$ scp /home/dmdba/dmdsc/data/DSC2/* 192.168.6.225:/home/dmdba/dmdsc/data/DSC2/
dmdba@192.168.6.225's password:
dm.ini 100% 76KB 26.3MB/s 00:00
dmmal.ini 100% 206 203.8KB/s 00:00
sqllog.ini 100% 714 783.9KB/s 00:00
[dmdba@dm1 DSC2]$ pwd
9.3 启动dm1和dm2
[dmdba@dm1 ~]$ dmserver /home/dmdba/dmdsc/data/DSC1/dm.ini dcr_ini=/home/dmdba/dmdsc/data/DSC1/dmdcr.ini
[dmdba@dm2 DSC2]$ dmserver /home/dmdba/dmdsc/data/DSC2/dm.ini dcr_ini=/home/dmdba/dmdsc/data/DSC2/dmdcr.ini
10 注册服务
10.1 DM1
[root@dm1 root]# ./dm_service_installer.sh -t dmcss -dcr_ini /home/dmdba/dmdsc/data/DSC1/dmdcr.ini -p CSS
Created symlink from /etc/systemd/system/multi-user.target.wants/DmCSSServiceCSS.service to /usr/lib/systemd/system/DmCSSServiceCSS.service.
Finished to create the service (DmCSSServiceCSS)
[root@dm1 root]# ./dm_service_installer.sh -t dmasmsvr -dcr_ini /home/dmdba/dmdsc/data/DSC1/dmdcr.ini -y DmCSSServiceCSS.service -p ASM
Created symlink from /etc/systemd/system/multi-user.target.wants/DmASMSvrServiceASM.service to /usr/lib/systemd/system/DmASMSvrServiceASM.service.
Finished to create the service (DmASMSvrServiceASM)
[root@dm1 root]# ./dm_service_installer.sh -t dmserver -dm_ini /home/dmdba/dmdsc/data/DSC1/dm.ini -dcr_ini /home/dmdba/dmdsc/data/DSC1/dmdcr.ini -y DmASMSvrServiceASM.service -p DSC
Created symlink from /etc/systemd/system/multi-user.target.wants/DmServiceDSC.service to /usr/lib/systemd/system/DmServiceDSC.service.
Finished to create the service (DmServiceDSC)
10.2 DM2
DM2:
[root@dm2 root]# ./dm_service_installer.sh -t dmcss -dcr_ini /home/dmdba/dmdsc/data/DSC2/dmdcr.ini -p CSS
Created symlink from /etc/systemd/system/multi-user.target.wants/DmCSSServiceCSS.service to /usr/lib/systemd/system/DmCSSServiceCSS.service.
Finished to create the service (DmCSSServiceCSS)
[root@dm2 root]# ./dm_service_installer.sh -t dmasmsvr -dcr_ini /home/dmdba/dmdsc/data/DSC2/dmdcr.ini -y DmCSSServiceCSS.service -p ASM
Created symlink from /etc/systemd/system/multi-user.target.wants/DmASMSvrServiceASM.service to /usr/lib/systemd/system/DmASMSvrServiceASM.service.
Finished to create the service (DmASMSvrServiceASM)
[root@dm2 root]# ./dm_service_installer.sh -t dmserver -dm_ini /home/dmdba/dmdsc/data/DSC2/dm.ini -dcr_ini /home/dmdba/dmdsc/data/DSC2/dmdcr.ini -y DmASMSvrServiceASM.service -p DSC
Created symlink from /etc/systemd/system/multi-user.target.wants/DmServiceDSC.service to /usr/lib/systemd/system/DmServiceDSC.service.
Finished to create the service (DmServiceDSC)
11 配置监听器
11.1 DM1主机中配置
注:在任意主机进行配置即可。
[dmdba@dm1 DSC1]$ cat dmcssm.ini
CSSM_OGUID = 1234
CSSM_CSS_IP = 192.168.6.224:5230
CSSM_CSS_IP = 192.168.6.225:5230
CSSM_LOG_PATH = /home/dmdba/dmdsc/log
CSSM_LOG_FILE_SIZE = 512
CSSM_LOG_SPACE_LIMIT = 2048
[dmdba@dm1 DSC1]$ pwd
/home/dmdba/dmdsc/data/DSC1
11.2 启动监视器cssm:
[dmdba@dm1 DSC1]$ dmcssm ini_path=/home/dmdba/dmdsc/data/DSC1/dmcssm.ini
[monitor] 2024-07-02 13:33:47: CSS MONITOR V8
[monitor] 2024-07-02 13:33:47: CSS MONITOR SYSTEM IS READY.
[monitor] 2024-07-02 13:33:47: Wait CSS Control Node choosed...
[monitor] 2024-07-02 13:33:48: Wait CSS Control Node choosed succeed.
Show查看集群状态: