环境介绍:
双机 操作系统:solaris 10 数据库版本:oracle 11g R1 64bit 当前主机的主机名称和IP:fly-db2 192.168.20.21 当前备机的主机名称和IP:fly-db1 192.168.20.20
背景:
主备机的物理内存均为94G,近期对备机的物理内存扩容一倍,并修改了数据库层面的相关内存参数,以及主备机的操作系统参数shmsys:shminfo_shmmax,注意是主备的shmsys:shminfo_shmmax参数均已修改,详情请查看下面链接的博文
http://fly1116.blog.51cto.com/8301004/1343788
今晚对fly-db1主机(目前的备机)的物理内存扩容一倍后,重启操作系统,因主备机的shmsys:shminfo_shmmax参数均已经修改,重启操作系统后,理论上该参数设置生效,通过双机软件将数据库从fly-db2切换到fly-db1,切换失败,在双机软件中,查看到out of memory错误,在数据库alert日志,查看到如下报错:
WARNING: The system does not seem to be configured optimally. Creating a segment of size 0x0000000220000000 failed. Please change the shm parameters so that a segment can be created for this size. While this is not a fatal issue, creating one segment may improve performance 如果手工启动,则报错如下: SQL> startup ORA-27102: out of memory SVR4 Error: 22: Invalid argument bash-3.00$ oerr ora 27102 27102, 00000, "out of memory" // *Cause: Out of memory // *Action: Consult the trace file for details
从以上报错分析,因为使用的是同一个数据库,操作系统层面只修改了shmsys:shminfo_shmmax参数,定位为当前备机fly-db1该参数设置未生效
1、对比主备机的/etc/system文件,是一致的
bash-3.00#hostname fly-db1 bash-3.00# diff /etc/system /home/oracle/system_flydb2_131220.bak
2、百度之solaris out of memory,查看到以下blog,文章中涉及/etc/project文件,我们可以这么理解/etc/project文件,如果/etc/system和/etc/project中设置了相同的参数,则操作系统会忽略/etc/system的设置,而采用/etc/project的设置,/etc/project文件可以针对特定用户设置只对自己生效的参数
https://blogs.oracle.com/mandalika/entry/oracle_on_solaris_10_fixing Oracle on Solaris 10 : Fixing the 'ORA-27102: out of memory' Error
3、查看到当前备机fly-db1主机的/etc/project中设置了max-shm-memory的最大大小为60G,即最大的共享内存段为60G,如下所示:
bash-3.00# hostname fly-db1 bash-3.00# cat /etc/project system:0:::: user.root:1:::: noproject:2:::: default:3::::project.max-shm-memory=(privileged,64424509440,deny) group.staff:10:::: bash-3.00# bc scale=5 64424509440/1024/1024/1024 60.00000
4、查看当前主机(fly-db2)的/etc/project内容,未设置max-shm-memory的最大值,所以在该操作系统,生效的是/etc/system的设置
bash-3.00#hostname fly-db2 bash-3.00# cat /etc/project system:0:::: user.root:1:::: noproject:2:::: default:3:::: group.staff:10::::
5、将当前备机fly-db1的/etc/project修改成和当前主机的/etc/project一样
bash-3.00#hostname fly-db1 bash-3.00# cat /etc/project system:0:::: user.root:1:::: noproject:2:::: default:3:::: group.staff:10::::
6、重启操作系统
bash-3.00# sync bash-3.00# sync bash-3.00# sync bash-3.00# shutdown -i6 -g0 -y
7、双机切换,成功切换到备机fly-db1
以下内容来自: https://blogs.oracle.com/mandalika/entry/oracle_on_solaris_10_fixing Symptom: As part of a database tuning effort you increase the SGA/PGA sizes; and Oracle greets with an ORA-27102: out of memory error message. The system had enough free memory to serve the needs of Oracle. SQL> startup ORA-27102: out of memory SVR4 Error: 22: Invalid argument Diagnosis $ oerr ORA 27102 27102, 00000, "out of memory" // \*Cause: Out of memory // \*Action: Consult the trace file for details Not so helpful. Let's look the alert log for some clues. % tail -2 alert.log WARNING: EINVAL creating segment of size 0x000000028a006000 fix shm parameters in /etc/system or equivalent Oracle is trying to create a 10G shared memory segment (depends on SGA/PGA sizes), but operating system (Solaris in this example) responded with an invalid argument (EINVAL) error message. There is a little hint about setting shm parameters in /etc/system. Prior to Solaris 10, shmsys:shminfo_shmmax parameter has to be set in /etc/system with maximum memory segment value that can be created. 8M is the default value on Solaris 9 and prior versions; where as 1/4th of the physical memory is the default on Solaris 10 and later. On a Solaris 10 (or later) system, it can be verified as shown below: % prtconf | grep Mem Memory size: 32760 Megabytes % id -p uid=59008(oracle) gid=10001(dba) projid=3(default) % prctl -n project.max-shm-memory -i project 3 project: 3: default NAME PRIVILEGE VALUE FLAG ACTION RECIPIENT project.max-shm-memory privileged 7.84GB - deny - system 16.0EB max deny - Now it is clear that the system is using the default value of 8G in this scenario, where as the application (Oracle) is trying to create a memory segment (10G) larger than 8G. Hence the failure.So, the solution is to configure the system with a value large enough for the shared segment being created, so Oracle succeeds in starting up the database instance. On Solaris 9 and prior releases, it can be done by adding the following line to /etc/system, followed by a reboot for the system to pick up the new value. set shminfo_shmmax = 0x000000028a006000 However shminfo_shmmax parameter was obsoleted with the release of Solaris 10; and Sun doesn't recommend setting this parameter in /etc/system even though it works as expected. On Solaris 10 and later, this value can be changed dynamically on a per project basis with the help of resource control facilities . This is how we do it on Solaris 10 and later: % prctl -n project.max-shm-memory -r -v 10G -i project 3 % prctl -n project.max-shm-memory -i project 3 project: 3: default NAME PRIVILEGE VALUE FLAG ACTION RECIPIENT project.max-shm-memory privileged 10.0GB - deny - system 16.0EB max deny - Note that changes made with the prctl command on a running system are temporary, and will be lost when the system is rebooted. To make the changes permanent, create a project with projadd command and associate it with the user account as shown below: % projadd -p 3 -c 'eBS benchmark' -U oracle -G dba -K 'project.max-shm-memory=(privileged,10G,deny)' OASB % usermod -K project=OASB oracle Finally make sure the project is created with projects -l or cat /etc/project commands. % projects -l ... ... OASB projid : 3 comment: "eBS benchmark" users : oracle groups : dba attribs: project.max-shm-memory=(privileged,10737418240,deny) % cat /etc/project ... ... OASB:3:eBS benchmark:oracle:dba:project.max-shm-memory=(privileged,10737418240,deny) With these changes, Oracle would start the database up normally. SQL> startup ORACLE instance started. Total System Global Area 1.0905E+10 bytes Fixed Size 1316080 bytes Variable Size 4429966096 bytes Database Buffers 6442450944 bytes Redo Buffers 31457280 bytes Database mounted. Database opened.
转载于:https://blog.51cto.com/fly007/1343864