一、环境信息
1.1 基础信息
深信服超融合平台
操作系统:银河麒麟v10 sp2
cpu :海光C86
数据库:DM8
1.2 网络规划
数据库服务器名称 | DSC1 | DSC2 |
业务IP | 10.255.250.55 | 10.255.250.56 |
心跳IP | 192.168.188.1 | 192.168.188.2 |
1.3 端口规划
参数 | DSC1 | DSC2 | 作用 | |
CSS | DCR_EP_PORT | 11286 | 11286 | CSS之间通讯 |
ASM | DCR_EP_PORT | 11266 | 11266 | ASM |
MAL_PORT | 7236 | 7236 | MAL | |
DB | DCR_CHECK_PORT | 11256 | 11256 | 与CSS通讯 |
PORT_NUM | 5236 | 5236 | 实例端口 |
二、安装目录规划
2.1 软件目录规划
节点1 | 节点二 | |
软件安装目录 | /home/dmdba/dmdbms | /home/dmdba/dmdbms |
集群配置文件目录 | /home/dmdba/dsc_config | /home/dmdba/dsc_config |
备份目录 | /dmbak | /dmbak |
sql日志目录 | /dmbak/dmlog | /dmbak/dmlog |
2.2 共享磁盘规划
大小 | |||
配置盘 | 5G | ||
表决盘 | 5G | ||
数据盘 | 500G*2 | ||
在线日志盘 | 30G | ||
归档日志盘 | 200G*2 |
三、开始安装
3.1 操作系统参数设置
limit.conf参数设置 dmdba soft nofile 65535 dmdba hard nofile 65535 dmdba soft nproc 65535 dmdba hard nproc 65535 dmdba soft core unlimited dmdba hard core unlimited vi /etc/sysctl.conf vm.swappiness = 1 vm.dirty_background_ratio = 10 vm.dirty_ratio = 30 vm.dirty_writeback_centisecs = 500 vm.dirty_expire_centisecs = 3000 关闭防火墙(两节点) [root@dmdb1 ~]# systemctl stop firewalld [root@dmdb1 ~]# systemctl disable firewalld Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service. Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. |
3.2 绑定磁盘
安装multipah
yum install multipath-tools-devel.x86_64
加载dm模块
modprobe dm-multipath
modprobe dm-round-robin
启动服务
systemctl enable multipathd
systemctl start multipathd
3.3 绑定wwn
multipath -F
multipath -v2
multipath -ll
编辑配置文件
[root@dsc1 etc]$ cat multipath.conf # Blacklist all devices by default. Remove this to enable multipathing # on the default devices. blacklist { devnode "vda" devnode "vdb" } ## Use user friendly names, instead of using WWIDs as names. defaults { user_friendly_names yes } multipaths { multipath { wwid 36b370960e041f80b5730fec26c4bad12 alias vote path_grouping_policy multibus path_selector "round-robin 0" failback immediate rr_weight uniform no_path_retry 12 rr_min_io 100 } multipath { wwid 36e0008188043e409bef033e23de00f21 alias dcr path_grouping_policy multibus path_selector "round-robin 0" failback immediate rr_weight uniform no_path_retry 12 rr_min_io 100 } multipath { wwid 3699e0a8d60430f0aa81004cd0946d2e3 alias data01 path_grouping_policy multibus path_selector "round-robin 0" failback immediate rr_weight uniform no_path_retry 12 rr_min_io 100 } multipath { wwid 36d4e0a47c049d30a34302b3e222deef8 alias data02 path_grouping_policy multibus path_selector "round-robin 0" failback immediate rr_weight uniform no_path_retry 12 rr_min_io 100 } multipath { wwid 36b4c0d8360419e0a4ee0af6c9ceddf9c alias redo path_grouping_policy multibus path_selector "round-robin 0" failback immediate rr_weight uniform no_path_retry 12 rr_min_io 100 } multipath { wwid 36c060810d04f7b0891302ac9a0e48682 alias archlog1 path_grouping_policy multibus path_selector "round-robin 0" failback immediate rr_weight uniform no_path_retry 12 rr_min_io 100 } multipath { wwid 36a5408935044440b42f02528bc78a48b alias archlog2 path_grouping_policy multibus path_selector "round-robin 0" failback immediate rr_weight uniform no_path_retry 12 rr_min_io 100 } } |
3.4 udev绑定授权
因为后续一些管理维护,建议给共享磁盘单独配置一个软连接目录。避免全部在/dev/mapper下
SYMLINK指定生成在/dev/dm下,这里根据multipath绑定的别名配置
[root@dsc1 rules.d]$ cat 15-permssions.rules ENV{DM_NAME}=="dcr",SYMLINK+="dm/asm-dmdcr",OWNER="dmdba",GROUP="dinstall",MODE="660" ENV{DM_NAME}=="data01",SYMLINK+="dm/asm-data01",OWNER="dmdba",GROUP="dinstall",MODE="660" ENV{DM_NAME}=="data02",SYMLINK+="dm/asm-data02",OWNER="dmdba",GROUP="dinstall",MODE="660" ENV{DM_NAME}=="redo",SYMLINK+="dm/asm-redolog",OWNER="dmdba",GROUP="dinstall",MODE="660" ENV{DM_NAME}=="vote",SYMLINK+="dm/asm-vote",OWNER="dmdba",GROUP="dinstall",MODE="660" ENV{DM_NAME}=="archlog1",SYMLINK+="dm/asm-arch1",OWNER="dmdba",GROUP="dinstall",MODE="660" ENV{DM_NAME}=="archlog2",SYMLINK+="dm/asm-arch2",OWNER="dmdba",GROUP="dinstall",MODE="660" |
绑定后查看
查看软连接目录,这里权限是root不用关注,/dev下权限正确即可
四、数据库实例初始化
4.1 配置文件准备
dmdcr_cfg.ini dmasvrmal.ini两节点相同
表决盘 集群配置盘配置
[dmdba@dsc1 dsc_config]$ cat dmdcr_cfg.ini DCR_N_GRP = 3 #集群环境有多少个 GROUP,范围:1~16 DCR_VTD_PATH = /dev/dm/asm-vote #规划为 vote 的磁盘 DCR_OGUID = 230714 ## 消息标识,一个组里面只有一个。 [GRP] #新建一个 GROUP DCR_GRP_TYPE = CSS #组类型(CSS/ASM/DB) DCR_GRP_NAME = GRP_CSS #组名 DCR_GRP_N_EP = 2 #组内节点个数 DCR_GRP_DSKCHK_CNT = 60 #磁盘心跳容错时间,单位:秒 [GRP_CSS] DCR_EP_NAME = CSS0 #CSS 节点名 DCR_EP_HOST = 192.168.188.1 #心跳地址 DCR_EP_PORT = 11286 #CSS 端口 [GRP_CSS] DCR_EP_NAME = CSS1 DCR_EP_HOST = 192.168.188.2 DCR_EP_PORT = 11286 [GRP] DCR_GRP_TYPE = ASM DCR_GRP_NAME = GRP_ASM DCR_GRP_N_EP = 2 DCR_GRP_DSKCHK_CNT = 60 [GRP_ASM] DCR_EP_NAME = ASM0 #ASM 节点名,和 dmasvrmal 的 MAL_INST_NAME 一致 DCR_EP_SHM_KEY = 42424 #共享内存标识 DCR_EP_SHM_SIZE = 512 #共享内存大小 DCR_EP_HOST = 192.168.188.1 #心跳地址 DCR_EP_PORT = 11266 #ASM 端口 DCR_EP_ASM_LOAD_PATH = /dev/dm [GRP_ASM] DCR_EP_NAME = ASM1 DCR_EP_SHM_KEY = 42425 DCR_EP_SHM_SIZE = 512 DCR_EP_HOST = 192.168.188.2 DCR_EP_PORT = 11266 DCR_EP_ASM_LOAD_PATH = /dev/dm [GRP] DCR_GRP_TYPE = DB DCR_GRP_NAME = GRP_DSC DCR_GRP_N_EP = 2 DCR_GRP_DSKCHK_CNT = 60 [GRP_DSC] DCR_EP_NAME = DSC0 #实例名,和 dm.ini 的 INSTANCE_NAME 一致 DCR_EP_SEQNO = 0 #组内序号,不能重复 DCR_EP_PORT = 5236#实例端口,和 dm.ini 的 PORT_NUM 一致 DCR_CHECK_PORT = 11256 #DCR 检查端口 [GRP_DSC] DCR_EP_NAME = DSC1 DCR_EP_SEQNO = 1 DCR_EP_PORT = 5236 DCR_CHECK_PORT = 11257 |
asm通讯配置
[dmdba@dsc1 dsc_config]$ cat dmasvrmal.ini [MAL_INST0] MAL_INST_NAME = ASM0 MAL_HOST = 192.168.188.1 #心跳地址 MAL_PORT = 7236 #MAL 监听端口 [MAL_INST1] MAL_INST_NAME = ASM1 MAL_HOST = 192.168.188.2 MAL_PORT = 7236 |
两节点不同,dmdcr.ini,集群启动配置文件
节点1 [dmdba@dsc1 dsc_config]$ cat dmdcr.ini DMDCR_PATH = /dev/dm/asm-dmdcr DMDCR_MAL_PATH = /home/dmdba/dsc_config/dmasvrmal.ini DMDCR_SEQNO = 0 DMDCR_AUTO_OPEN_CHECK = 60 DMDCR_ASM_TRACE_LEVEL = 2 DMDCR_ASM_RESTART_INTERVAL = 60 #CSS 认定 ASM 故障重启的时间 DMDCR_ASM_STARTUP_CMD = /home/dmdba/dmdbms/bin/DmASMSvrServiceASM start DMDCR_DB_RESTART_INTERVAL = 60 ##CSS 认定 DSC 故障重启的时间,设置为0不开启 DMDCR_DB_STARTUP_CMD = /home/dmdba/dmdbms/bin/DmServiceDSC start节点2 [dmdba@dsc2 dsc_config]$ cat dmdcr.ini DMDCR_PATH = /dev/dm/asm-dmdcr DMDCR_MAL_PATH = /home/dmdba/dsc_config/dmasvrmal.ini DMDCR_SEQNO = 1 DMDCR_AUTO_OPEN_CHECK = 60 DMDCR_ASM_TRACE_LEVEL = 2 DMDCR_ASM_RESTART_INTERVAL = 60 #CSS 认定 ASM 故障重启的时间 DMDCR_ASM_STARTUP_CMD = /home/dmdba/dmdbms/bin/DmASMSvrServiceASM start DMDCR_DB_RESTART_INTERVAL = 60 ##CSS 认定 DSC 故障重启的时间,设置为0不开启 DMDCR_DB_STARTUP_CMD = /home/dmdba/dmdbms/bin/DmServiceDSC start |
4.2 初始化dcr vote以及磁盘格式化为asm
create dcrdisk '/dev/mapper/dcr' 'dcr' create votedisk '/dev/mapper/vote' 'vote' init dcrdisk '/dev/mapper/dcr' from '/home/dmdba/dsc_config/dmdcr_cfg.ini' identified by 'dameng123' init votedisk '/dev/mapper/vote' from '/home/dmdba/dsc_config/dmdcr_cfg.ini' create asmdisk '/dev/mapper/data01' 'DATA01' create asmdisk '/dev/mapper/data02' 'DATA02' create asmdisk '/dev/mapper/redo' 'REDOLOG' create asmdisk '/dev/mapper/archlog1' 'ARCH1' create asmdisk '/dev/mapper/archlog2' 'ARCH2' |
4.3 创建磁盘组
启动asm后方可创建磁盘组,暂时前台启动
节点1 dmcss DCR_INI=/home/dmdba/dmdbms/dmdcr.ini Dmasmsvr DCR_INI=/home/dmdba/dmdbms/dmdcr.ini 节点2 dmcss DCR_INI=/home/dmdba/dmdbms/dmdcr.ini dmasmsvr DCR_INI=/home/dmdba/dmdbms/dmdcr.ini |
create diskgroup 'REDOLOG' asmdisk '/dev/mapper/redo' create diskgroup 'DMDATA' asmdisk '/dev/mapper/data01' alter diskgroup 'DMDATA' add asmdisk '/dev/mapper/data02' create diskgroup 'DMARCH' asmdisk '/dev/mapper/archlog1' alter diskgroup 'DMARCH' add asmdisk '/dev/mapper/archlog2' |
4.4 初始化数据库实例
创建dminit文件,两节点一致 [dmdba@dsc1 dsc_config]$ cat dminit.ini DB_NAME = DSC SYSTEM_PATH = +DMDATA/data SYSTEM = +DMDATA/data/SYSTEM.dbf SYSTEM_SIZE = 1024 ROLL = +DMDATA/data/ROLL.dbf ROLL_SIZE = 20000 MAIN = +DMDATA/data/MAIN.dbf MAIN_SIZE = 1024 CTL_PATH = +DMDATA/data/dm.ctl CTL_SIZE = 8 LOG_SIZE = 2048 DCR_PATH = /dev/dm/asm-dmdcr DCR_SEQNO = 0 AUTO_OVERWRITE = 1 PAGE_SIZE = 32 EXTENT_SIZE = 32 BLANK_PAD_MODE = 1 [DSC0] CONFIG_PATH = /home/dmdba/dsc_config/DSC0 PORT_NUM = 5236 MAL_HOST = 192.168.188.1 MAL_PORT = 11246 LOG_PATH = +REDOLOG/DSC0_LOG01.log LOG_PATH = +REDOLOG/DSC0_LOG02.log [DSC1] CONFIG_PATH = /home/dmdba/dsc_config/DSC1 PORT_NUM = 5236 MAL_HOST = 192.168.188.2 MAL_PORT = 11246 LOG_PATH = +REDOLOG/DSC1_LOG01.log LOG_PATH = +REDOLOG/DSC1_LOG02.log |
开始初始化实例
在节点1初始化 /home/dmdba/dmdbms/bin/dminit control=/dm/dmdbms/dsc_config/init/dminit.ini 拷贝文件到节点2 [dmdba@~]# scp -r DSC1 192.168.188.2:/home/dmdba/dsc_config/ |
节点1 [dmdba@dsc1 DSC0]$ cat dmarch.ini ARCH_WAIT_APPLY = 0 ARCH_LOCAL_SHARE = 1 ARCH_LOCAL_SHARE_CHECK=0 [ARCHIVE_LOCAL1] ARCH_TYPE = LOCAL ARCH_DEST = +DMARCH/ARCH/DSC0/arch ARCH_FILE_SIZE = 2048 ARCH_SPACE_LIMIT = 102400 [ARCHIVE_REMOTE1] ARCH_TYPE = REMOTE ARCH_DEST = DSC1 ARCH_INCOMING_PATH = +DMARCH/ARCH/DSC1/arch ARCH_FILE_SIZE = 2048 ARCH_SPACE_LIMIT = 102400 节点2 [dmdba@dsc2 DSC1]$ cat dmarch.ini ARCH_WAIT_APPLY = 0 ARCH_LOCAL_SHARE = 1 ARCH_LOCAL_SHARE_CHECK=0 [ARCHIVE_LOCAL1] ARCH_TYPE = LOCAL ARCH_DEST = +DMARCH/ARCH/DSC1/arch ARCH_FILE_SIZE = 2048 ARCH_SPACE_LIMIT = 102400 [ARCHIVE_REMOTE1] ARCH_TYPE = REMOTE ARCH_DEST = DSC0 ARCH_INCOMING_PATH = +DMARCH/ARCH/DSC0/arch ARCH_FILE_SIZE = 2048 ARCH_SPACE_LIMIT = 102400 |
启动实例,启动后没有问题关闭注册服务后台启动
节点1 [dmdba@]# /home/dmdba/dmdbms/bin/dmserver /home/dmdba/dmdbms/dsc_config/DSC0/dm.ini dcr_ini=/dm/dmdbms/dsc_config/dmdcr.ini 节点2 [dmdba@~]# /home/dmdba/dmdbms/bin/dmserver /dm/dmdbms/dsc_config/DSC1/dm.ini dcr_ini= /home/dmdba/dmdbms/dsc_config/dmdcr.ini |
五、注册服务
两节点执行
/home/dmdba/dmdbms/script/root/dm_service_installer.sh -t dmcss -dcr_ini /home/dmdba/dsc_config/dmdcr.ini -p CSS /home/dmdba/dmdbms/script/root/dm_service_installer.sh -t dmasmsvr -dcr_ini /home/dmdba/dsc_config/dmdcr.ini -y DmCSSServiceCSS.service -p ASM /home/dmdba/dmdbms/script/root/dm_service_installer.sh -t dmserver -dm_ini /home/dmdba/dsc_config/DSC0/dm.ini -dcr_ini /home/dmdba/dsc_config/dmdcr.ini -y DmASMSvrServiceASM.service -p DSC |