ORA-03113

RAC模式下的DG遇到如下报错：

数据库起不来了。

表面现象能看出问题出现在了数据文件有问题：

1、mount下说明参数文件和控制文件加载了

SQL> 
SQL> startup force 
ORACLE instance started.

Total System Global Area  734892032 bytes
Fixed Size                  2256872 bytes
Variable Size             260046872 bytes
Database Buffers          469762048 bytes
Redo Buffers                2826240 bytes
Database mounted.
ORA-03113: end-of-file on communication channel
Process ID: 13107
Session ID: 1 Serial number: 5

看alert日志：

我这个日志截的比较全，而且我没怀疑过我的pfile会有问题：

但事实就是如此EMREP写成了RMREP

ARC1: Archival started
ARC2: Archival started
ARC1: Becoming the 'no FAL' ARCH
ARC1: Becoming the 'no SRL' ARCH
ARC2: Becoming the heartbeat ARCH
ARCH: LGWR is scheduled to archive destination LOG_ARCHIVE_DEST_2 after log switch
Errors in file /u01/app/oracle/diag/rdbms/emrep/EMREP1/trace/EMREP1_ora_13261.trc:
ORA-16038: log 1 sequence# 51 cannot be archived
ORA-00254: error in archive control string ''
ORA-00312: online log 1 thread 1: '+INIFO/emrep/onlinelog/group_1.257.1072841213'
ORA-00312: online log 1 thread 1: '+FBA/emrep/onlinelog/group_1.257.1072841213'
ORA-15173: entry 'rmrep' does not exist in directory '/'
USER (ospid: 13261): terminating the instance due to error 16038
System state dump requested by (instance=1, osid=13261), summary=[abnormal instance termination].
System State dumped to trace file /u01/app/oracle/diag/rdbms/emrep/EMREP1/trace/EMREP1_diag_13207_20210520014942.trc
Dumping diagnostic data in directory=[cdmp_20210520014942], requested by (instance=1, osid=13261), summary=[abnormal instance termination].
Instance terminated by USER, pid = 13261
Thu May 20 01:50:51 2021
Starting ORACLE instance (normal)
************************ Large Pages Information *******************
Per process system memlock (soft) limit = 64 KB
 
Total Shared Global Region in Large Pages = 0 KB (0%)
 
Large Pages used by this instance: 0 (0 KB)
Large Pages unused system wide = 127 (254 MB)
Large Pages configured system wide = 127 (254 MB)
Large Page size = 2048 KB
 
RECOMMENDATION:
  Total System Global Area size is 706 MB. For optimal performance,
  prior to the next instance restart:
  1. Increase the number of unused large pages by 
 at least 226 (page size 2048 KB, total size 452 MB) system wide to
  get 100% of the System Global Area allocated with large pages
  2. Large pages are automatically locked into physical memory.
 Increase the per process memlock (soft) limit to at least 714 MB to lock
 100% System Global Area's large pages into physical memory
********************************************************************
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0

SQL> alter system set log_archive_dest_1='LOCATION=+FBA/EMREP VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=EMREP'
  2  ;

System altered.

SQL> startup force 
ORACLE instance started.

Total System Global Area  734892032 bytes
Fixed Size                  2256872 bytes
Variable Size             260046872 bytes
Database Buffers          469762048 bytes
Redo Buffers                2826240 bytes
Database mounted.
Database opened.
SQL> 

另外我调整了大页内存，毕竟给出了建议：

vm.nr_hugepages = 400*2M=800M

我的系统SGA=703M

800>703这样才能使大页生效

[root@rac-1 ~]# cat /proc/meminfo
MemTotal:        2402972 kB
MemFree:          330112 kB
MemAvailable:     737356 kB
Buffers:             912 kB
Cached:           859384 kB
SwapCached:         2228 kB
Active:          1054576 kB
Inactive:         501952 kB
Active(anon):     682936 kB
Inactive(anon):   347960 kB
Active(file):     371640 kB
Inactive(file):   153992 kB
Unevictable:      332708 kB
Mlocked:          332708 kB
SwapTotal:       4227068 kB
SwapFree:        4189124 kB
Dirty:               596 kB
Writeback:             0 kB
AnonPages:       1026760 kB
Mapped:           241636 kB
Shmem:            263824 kB
Slab:              58292 kB
SReclaimable:      29632 kB
SUnreclaim:        28660 kB
KernelStack:        9440 kB
PageTables:        42756 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     5428552 kB
Committed_AS:    3033524 kB
VmallocTotal:   34359738367 kB
VmallocUsed:       35412 kB
VmallocChunk:   34359692368 kB
HardwareCorrupted:     0 kB
AnonHugePages:    337920 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       79808 kB
DirectMap2M:     2377728 kB
[root@rac-1 ~]# vi  /etc/sysctl.conf

# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586
vm.nr_hugepages=400
kernel.shmmax = 1230321664
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
"/etc/sysctl.conf" 22L, 778C written
[root@rac-1 ~]# sysctl -p
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586
vm.nr_hugepages = 400
kernel.shmmax = 1230321664
[root@rac-1 ~]# vi  /etc/security/limits.conf 

# /etc/security/limits.conf
#
#This file sets the resource limits for the users logged in via PAM.
#It does not affect resource limits of the system services.
#
#Also note that configuration files in /etc/security/limits.d directory,
#which are read in alphabetical order, override the settings in this
#file in case the domain is the same or more specific.
#That means for example that setting a limit for wildcard domain here
#can be overriden with a wildcard setting in a config file in the
#subdirectory, but a user specific setting here can be overriden only
#with a user specific setting in the subdirectory.
#
#Each line describes a limit for a user in the form:
#
#<domain>        <type>  <item>  <value>
#
#Where:
#<domain> can be:
#        - a user name
#        - a group name, with @group syntax
#        - the wildcard *, for default entry
#        - the wildcard %, can be also used with %group syntax,
#                 for maxlogin limit
#
#<type> can have the two values:
#        - "soft" for enforcing the soft limits
#        - "hard" for enforcing hard limits
#
#<item> can be one of the following:
#        - core - limits the core file size (KB)
#        - data - max data size (KB)
#        - fsize - maximum filesize (KB)
#        - memlock - max locked-in-memory address space (KB)
#        - nofile - max number of open file descriptors
#        - rss - max resident set size (KB)
#        - stack - max stack size (KB)
#        - cpu - max CPU time (MIN)
#        - nproc - max number of processes
#        - as - address space limit (KB)
#        - maxlogins - max number of logins for this user
#        - maxsyslogins - max number of logins on the system
#        - priority - the priority to run user process with
#        - locks - max number of file locks the user can hold
#        - sigpending - max number of pending signals
#        - msgqueue - max memory used by POSIX message queues (bytes)
#        - nice - max nice priority allowed to raise to values: [-20, 19]
#        - rtprio - max realtime priority
#
#<domain>      <type>  <item>         <value>
#

#*               soft    core            0
#*               hard    rss             10000
#@student        hard    nproc           20
#@faculty        soft    nproc           20
#@faculty        hard    nproc           50
#ftp             hard    nproc           0
#@student        -       maxlogins       4
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
oracle soft stack 10240
grid soft nproc 2047
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536
grid soft stack 1024
oracle soft memlock unlimited
oracle hard memlock unlimited
"/etc/security/limits.conf" 72L, 2710C written
[root@rac-1 ~]# 

JROJYUN -longly dog

2021-05-20