Oracle Rac 使用nfs搭建各自节点归档目录
Oracle Rac 使用nfs搭建各自节点归档目录
本次是在Oracle Rac环境下已有了ASM共享存储的前提下没有把归档目录设置在ASM存储里，而是使用了nfs协议两节点互相能识别各自的归档目录，这种环境设置在生产服务器的话在nfs相关的mount方面需特别注意，不然会导致各种异常问题，不建议在生产端使用。生产端建议备份至ASM共享存储里，以下是这次试验的过程和踩坑，最终结果为两节点能正常归档、互相识别并且单节点能正常运行。
背景
操作系统：redhat7.7
[root@rac1 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.7 (Maipo)
Grid和Oracle版本：19C
[grid@rac1 ~]$ crsctl query crs releaseversion
Oracle High Availability Services release version on the local node is [19.0.0.0.0]
[grid@rac1 ~]$ crsctl query crs softwareversion
Oracle Clusterware version on node [rac1] is [19.0.0.0.0]
[grid@rac1 ~]$ crsctl query crs activeversion
Oracle Clusterware active version on the cluster is [19.0.0.0.0]
[oracle@rac1 ~]$ sqlplus -v
SQL*Plus: Release 19.0.0.0.0 - Production
Version 19.3.0.0.0
数据库归档情况：未开启
SQL> archive log list
Database log mode No Archive Mode //非归档模式
Automatic archival Disabled //禁用
Archive destination /u01/app/oracle/product/19.3.0/db_1/dbs/arch
Oldest online log sequence 22
Current log sequence 23
部署过程
-
两节点创建各自的lvm归档目录(两节点分别做)
两节点分别增加50G硬盘作为归档目录用途。
[root@rac1 ~]# fdisk -l //增加硬盘后查看增加的硬盘 ----------------------- Disk /dev/sdh: 53.7 GB, 53687091200 bytes, 104857600 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x15927720 [root@rac1 ~]# fdisk /dev/sdh //对增加的sdb硬盘做修改 Welcome to fdisk (util-linux 2.23.2). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. Command (m for help): m--> n--> //增加分区 m--> p //查看sdb盘的分区情况 m--> t--> l--> 8e //修改分区1的格式为lvm m--> w //保存 [root@rac1 ~]# pvcreate /dev/sdh1 //创建PV [root@rac1 ~]# pvs PV VG Fmt Attr PSize PFree /dev/sda2 rhel lvm2 a-- 48.00g 4.00m /dev/sdh1 lvm2 a-- <50.00g 1020.00m [root@rac1 ~]# vgcreate vg_arch1 /dev/sdh1 //创建VG [root@rac1 ~]# lvcreate -L +49G -n lv_arch1 vg_arch1 //创建LV [root@rac1 ~]# lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert root rhel -wi-ao---- 40.00g swap rhel -wi-ao---- 8.00g lv_arch1 vg_arch1 -wi-ao---- 49.00g [root@rac1 ~]# mkfs.xfs /dev/vg_arch1/lv_arch1 [root@rac1 ~]# mkdir /arch1 [root@rac1 ~]# mount /dev/vg_arch1/lv_arch1 /arch1 [root@rac1 ~]# vi /etc/fstab 增加以下内容 /dev/mapper/vg_arch1-lv_arch1 /arch1 xfs defaults 0 0 [root@rac1 ~]# df -h //查看挂载的挂载成功 Filesystem Size Used Avail Use% Mounted on devtmpfs 3.9G 24K 3.9G 1% /dev tmpfs 3.9G 893M 3.0G 23% /dev/shm tmpfs 3.9G 91M 3.8G 3% /run tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup /dev/mapper/rhel-root 40G 29G 12G 71% / /dev/sda1 297M 165M 133M 56% /boot tmpfs 783M 48K 783M 1% /run/user/54322 tmpfs 783M 4.0K 783M 1% /run/user/42 tmpfs 783M 56K 783M 1% /run/user/54321 /dev/sr0 4.2G 4.2G 0 100% /media/cdrom /dev/mapper/vg_arch1-lv_arch1 49G 107M 49G 1% /arch1 tmpfs 783M 0 783M 0% /run/user/0 [root@rac1 ~]# chown oracle:oinstall -R /arch1 -
使用NFS协议分别挂载两节点的归档目录(两节点分别做)
NFS协议是一种用于文件共享的协议，它可以使得主机之间进行文件的共享。客户端可以像在本机上的文件一样操作远程主机的文件。NFS协议最初仅支持UDP协议，目前最新版本的NFS可以支持UDP和TCP协议，不过UDP协议的速度会更快。
NFS协议是一个十分简单的协议，它本身没有提供信息传输的协议和功能。之所以NFS能够让主机之间通过网络进行资料共享，这是因为NFS使用了一些其它的传输协议，主要用到了RPC（Remote Procedure Call）功能。所以在启动NFS服务器的时候需要启动RPC服务。linux7安装需要安装nfs的相关包和portmap
[root@rac1 arch1]# yum install nfs* portmap -y //NFS与RPC同时安装的yum命令 [root@rac1 ~]# systemctl stop firewalld //关闭防火墙 [root@rac1 ~]# vi /etc/sysconfig/selinux SELINUX=disabled //关闭selinux节点1共享/arch1目录
[root@rac1 ~]# cat /etc/exports //节点1(只允许61这台机访问、读写、非授权访问、同步写入到内存和硬盘中、客户机上的任何用户访问该共享目录时都映射成匿名用户，映射成uid和gid为54321) /arch1 10.16.35.61(rw,insecure,sync,all_squash,anonuid=54321,anongid=54321) [root@rac1 ~]# systemctl start nfs.service //启动nfs服务 [root@rac1 ~]# systemctl enable nfs.service //设置开机启动 Created symlink from /etc/systemd/system/multi-user.target.wants/nfs-server.service to /usr/lib/systemd/system/nfs-server.service. [root@rac1 ~]# systemctl is-enabled nfs.service enabled [root@rac1 ~]# systemctl is-enabled rpcbind enabled 如果nfs服务以及启动了，临时增加需要共享的目录只需以下命令 [root@rac1 ~]# exportfs -rv //重新mount /etc/exports中分享出来的目录 [root@rac1 ~]# mkdir /arch2 //创建跟目录/arch2目录 [root@rac1 ~]# mount -t nfs -o rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,vers=3,timeo=600,actimeo=0 10.16.35.61:/arch2 /arch2 //使用oracle建议的方式挂载nfs远程目录节点2共享/arch2目录。
[root@rac2 arch1]# cat /etc/exports //节点2(只允许61这台机访问、读写、非授权访问、同步写入到内存和硬盘中、客户机上的任何用户访问该共享目录时都映射成匿名用户，映射成uid和gid为54321) /arch2 10.16.35.60(rw,insecure,sync,all_squash,anonuid=54321,anongid=54321) [root@rac2 ~]# systemctl start nfs.service //启动nfs服务 [root@rac2 ~]# systemctl enable nfs.service //设置开机启动 Created symlink from /etc/systemd/system/multi-user.target.wants/nfs-server.service to /usr/lib/systemd/system/nfs-server.service. [root@rac2 ~]# systemctl is-enabled nfs.service enabled 如果nfs服务以及启动了，临时增加需要共享的目录只需以下命令 [root@rac2 ~]# exportfs -rv //重新mount /etc/exports中分享出来的目录 [root@rac2 ~]# mkdir /arch1 //创建跟目录/arch2目录 [root@rac2 ~]# mount -t nfs -o rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,vers=3,timeo=600,actimeo=0 10.16.35.60:/arch1 /arch1 //使用oracle建议的方式挂载nfs远程目录 如果是这样指定mount的： mount -t nfs 192.168.0.21:/archive /archive 备份归档时报错： ORA-19625: error identifying file /archive/1_42_682943725.arch ORA-27054: NFS file system where the file is created or resides is not mounted with correct options 需要增加mount参数： mount -t nfs -o rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,vers=3,timeo=600,actimeo=0 10.16.35.60:/arch1 /arch1 指定参数后，归档进程就可以正常的在NFS文件系统上归档了。其他相关系统的mount参数参考资料
http://blog.itpub.net/441019/viewspace-1055565/配置好后的目录情况如下：
[root@rac1 ~]# df -h //rac1节点目录 Filesystem Size Used Avail Use% Mounted on devtmpfs 3.9G 24K 3.9G 1% /dev tmpfs 3.9G 893M 3.0G 23% /dev/shm tmpfs 3.9G 91M 3.8G 3% /run tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup /dev/mapper/rhel-root 40G 29G 12G 71% / /dev/sda1 297M 165M 133M 56% /boot tmpfs 783M 48K 783M 1% /run/user/54322 tmpfs 783M 4.0K 783M 1% /run/user/42 tmpfs 783M 52K 783M 1% /run/user/54321 /dev/sr0 4.2G 4.2G 0 100% /media/cdrom /dev/mapper/vg_arch1-lv_arch1 49G 107M 49G 1% /arch1 10.16.35.61:/arch2 49G 176M 49G 1% /arch2 //nfs目录 tmpfs 783M 0 783M 0% /run/user/0 [root@rac2 arch1]# df -h //RAC2节点目录 Filesystem Size Used Avail Use% Mounted on devtmpfs 3.9G 24K 3.9G 1% /dev tmpfs 3.9G 893M 3.0G 23% /dev/shm tmpfs 3.9G 74M 3.8G 2% /run tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup /dev/mapper/rhel-root 40G 27G 13G 68% / /dev/sda1 297M 165M 133M 56% /boot tmpfs 783M 0 783M 0% /run/user/54322 /dev/mapper/vg_arch2-lv_arch2 49G 177M 49G 1% /arch2 10.16.35.60:/arch1 49G 107M 49G 1% /arch1 //nfs目录 tmpfs 783M 0 783M 0% /run/user/0 -
NFS自动挂载配置(2个节点分别做)
NFS开机挂载的命令不能写入到/etc/fstab 中，由于 NFS 依赖于网络，而/etc/fstab 的引用是在计算机网络尚未启动的时候就开始引导，因此肯定无法挂载成功！我们应该把挂载命令写入到/etc/rc.d/rc.local 中。
节点1：
[root@rac1 ~]# vi /etc/rc.d/rc.local //节点1 #!/bin/bash # THIS FILE IS ADDED FOR COMPATIBILITY PURPOSES # # It is highly advisable to create own systemd services or udev rules # to run scripts during boot instead of using this file. # # In contrast to previous versions due to parallel execution during boot # this script will NOT be run after all other services. # # Please note that you must run 'chmod +x /etc/rc.d/rc.local' to ensure # that this script will be executed during boot. touch /var/lock/subsys/local 增加以下内容 mount -t nfs -o rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,vers=3,timeo=600,actimeo=0 10.16.35.61:/arch2 /arch2节点2：
[root@rac2 ~]# vi /etc/rc.d/rc.local //节点2 #!/bin/bash # THIS FILE IS ADDED FOR COMPATIBILITY PURPOSES # # It is highly advisable to create own systemd services or udev rules # to run scripts during boot instead of using this file. # # In contrast to previous versions due to parallel execution during boot # this script will NOT be run after all other services. # # Please note that you must run 'chmod +x /etc/rc.d/rc.local' to ensure # that this script will be executed during boot. touch /var/lock/subsys/local 增加以下内容 mount -t nfs -o rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,vers=3,timeo=600,actimeo=0 10.16.35.60:/arch1 /arch1 -
数据库配置归档信息和开启归档(1个节点做就OK)
[root@rac1 ~]# su - oracle Last login: Tue May 12 16:34:43 CST 2020 on pts/0 [oracle@rac1 ~]$ sqlplus / as sysdba SQL*Plus: Release 19.0.0.0.0 - Production on Tue May 12 16:35:02 2020 Version 19.3.0.0.0 Copyright (c) 1982, 2019, Oracle. All rights reserved. Connected to: Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production Version 19.3.0.0.0 SQL> show parameter archive NAME TYPE VALUE ------------------------------------ ----------- ------------------------------ archive_lag_target integer 0 log_archive_config string log_archive_dest string log_archive_dest_1 string // 修改此参数 SQL> create pfile='/rac/spfile/orcl.ora' from spfile='+DATA1/orcl/spfileorcl.ora'; //简答备份下spfile SQL> alter system set log_archive_dest_1='location=/arch1' sid='racdb1' scope=spfile; //节点1修改归档路径为/arch1,需重启数据库生效 SQL> alter system set log_archive_dest_1='location=/arch2' sid='racdb2' scope=spfile; //节点2修改归档路径为/arch2，需重启数据库生效 SQL> alter system set log_archive_format="racdb%t_%r_%t_%s.arc" scope=spfile sid='racdb1'; //节点1修改归档格式,需重启数据库生效 SQL> alter system set log_archive_format="racdb%t_%r_%t_%s.arc" scope=spfile sid='racdb2'; //节点2修改归档格式,需重启数据库生效 [oracle@rac1 ~]$ srvctl stop database -d racdb //关闭RAC数据库 [oracle@rac1 ~]$ srvctl start database -d racdb //启动RAC数据库 取值意义 该参数指定Oracle 归档文件的文件格式，由一些字符串和变量组成 只在归档模式下适用 可使用如下变量 %s 日志sequence 号 %S 日志 sequence号，0填充 %t thread号 %T thread 号，0填充 %a activation ID %d 数据库ID %r resetlogs ID 其中大写的字母表示固定长度，不足则用0填充 其中 %s,%t,%r 为必须变量以保证文件名唯一 如果缺少其中一个则会启动时报错 [oracle@rac1 ~]$ srvctl stop database -d racdb //关闭数据库 [oracle@rac1 ~]$ srvctl start database -d racdb //开启数据库 SQL> archive log list Database log mode Archive Mode Automatic archival Enabled Archive destination /arch1 Oldest online log sequence 25 Next log sequence to archive 26 Current log sequence 26 SQL> alter system switch logfile; //手动归档 System altered. SQL> archive log list Database log mode Archive Mode Automatic archival Enabled Archive destination /arch1 Oldest online log sequence 26 Next log sequence to archive 27 Current log sequence 27 [oracle@rac1 ~]$ cd /arch1 [oracle@rac1 arch1]$ ll //查看归档记录，显示正常。 total 154768 drwxr-xr-x 2 oracle oinstall 6 May 11 10:12 bak -rw-r----- 1 oracle asmadmin 2514432 May 12 17:49 racdb1_1038938886_1_26.arc
结尾
至此，开启归档设置完毕。其实在此实验中配置附档参数的时候当时配置多了，导致节点2启动不了报错，后续整理在此次实验后。
3713
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
