In this Document
Applies to:
Oracle Database - Enterprise Edition - Version 11.2.0.1 to 12.1.0.2 [Release 11.2 to 12.1]
Information in this document applies to any platform.
Symptoms
1) ASM 11.2.0.3/11.2.0.4 configuration on Solaris SPARC T4-4 Server with 128 CPUs & RAM = 128 GB.
2) If the 128 CPUs are enabled, then ASM instance is unable to start due to the next errors:
ORA-00838: Specified value of MEMORY_TARGET is too small, needs to be at least 1328M
ORA-01078: failure in processing system parameters
[main] [ 2012-01-21 18:26:28.986 COT ] [UsmcaLogger.logException:174] ORA-04031: unable to allocate 32 bytes of shared memory ("shared pool","SELECT DECODE(null,'','Total...","SQLA","tmp")
3) But if only 64 CPUs are enabled, then ASM instance starts without problems.
4) On release 11.2, ASM uses Automatic Memory Management (AMM) by default, therefore MEMORY_MAX_TARGET & MEMORY_TARGEThave the default values = 272 MB (in the ASM instance):
MEMORY_MAX_TARGET = 272M
MEMORY_TARGET = 272M
5) And so, the SGA in the ASM instance has the same value (900 MB):
SQL> show sga
Total System Global Area 283930624 bytes
Fixed Size 2227664 bytes
Variable Size 256537136 bytes
ASM Cache 25165824 bytes
Cause
In 11.2.0.3/11.2.0.4, the "PROCESSES" parameter will be default to "available CPU cores * 80 + 40" (in the ASM spfile). As the default value for "MEMORY_TARGET" is based on "PROCESSES",
it can be insufficient if there is a large number of CPU cores or large
number of diskgroups which could cause issues (i.e. Grid Infrastructure
stack fails to stop with ORA-04031etc) per Bug:13605735 & Bug:12885278, it is recommended to increase the value of MEMORY_MAX_TARGET & MEMORY_TARGET before upgrading/installing to 11.2.0.3/11.2.0.4 (does not apply to 10g ASM).
Solution
1) Connect to the ASM instance(s) and increase the MEMORY_MAX_TARGET &MEMORY_TARGET as follow (3GB or 4GB will be enough):
SQL> alter system set MEMORY_MAX_TARGET=4096m scope=spfile SID='*';
SQL> alter system set MEMORY_TARGET=3072m scope=spfile SID='*';
Note: The number 1536m has proven to be sufficient for most environment, the change will not be effective until next restart.
2) Restart the ASM instances to apply the changes.
3) Then enable all the CPUs (e.g. total = 128) in your system:
bash-3.00# prtdiag -v|grep SPARC-T4 | grep on-line | wc -l
128
Important Note 1:
In 11.2.0.3/11.2.0.4, we increase the default PROCESSES based on the
number of CPU cores, and the default MEMORY_TARGET is based on
PROCESSES. If in 11.2.0.2, customers explicitly set MEMORY_TARGET to
some value that may not be big enough for 11.2.0.3/11.2.0.4, when they
upgrade to 11.2.0.3/11.2.0.4, ASM will fail to start with error
"memory_target is too small". We should add additional check for
MEMORY_TARGET during the upgrade prerequisite check.
You can
unset MEMORY_TARGET so that ASM can use the default value, but if
MEMORY_TARGET is explicitly set, please make sure it's large enough,
following the next rules:
1) If PROCESSES parameter is explicitly set:
The MEMORY_TARGET should be set to no less than:
256M + PROCESSES * 132K (64bit)
or
256M + PROCESSES * 120K (32bit)
2) If PROCESSES parameter is not set:
The MEMORY_TARGET should be set to no less than:
256M + (available_cpu_cores * 80 + 40) * 132K (64bit)
or
256M + (available_cpu_cores * 80 + 40) * 120K (32bit)
Important Note 2:
If this problem (ORA-4031 error) is occurring during a brand new ASM
(11.2.0.3 or 11.2.0.4) installation/configuration (e.g. when running
root.sh to start the ASM instance) and many CPUs/Cores are enabled
(above 4), then temporally disable most of them and leave just few
enabled (about 2 or 4), as the following example (64 CPUs are present
and enabled):
1) Ask your SA to disable 62 CPUs and leave enable only 2 of 64.
2) Verify only 2 CPUs were/are enabled using the next commands:
# kstat cpu_info | grep core_id | awk '{ print $2}' | uniq | wc -l
# prtdiag -v|grep SPARC-T4 | grep on-line | wc -l
3) Run root.sh again to start the ASM services/instances and complete the installation.
4)
Once the ASM instances are created and running, then increase the SGA
(AMM) in the ASM instances as follows (exact same values below):
SQL> alter system set MEMORY_MAX_TARGET=4096m scope=spfile;
SQL> alter system set MEMORY_TARGET=1536m scope=spfile;
5) Restart the ASM instances and check the AMM values:
SQL> show parameter MEMORY_MAX_TARGET
SQL> show parameter MEMORY_TARGET
6) Ask your SA to enable back the other 62 CPUs.
7) Restart CRS or HAS and make sure the ASM instances start.