DESCRIPTION
Starting with RedHat6, OL6, SLES11, and UEK2
kernels, Transparent HugePages are implemented and enabled
(default) in an attempt to improve the memory
management. Transparent HugePages are similar to
the HugePages that have been available in previous Linux
releases. The main difference is that the
Transparent HugePages are set up dynamically at run time by the
khugepaged thread in kernel while the regular HugePages had to be
preallocated at the boot up time.
Because Transparent HugePages are known
to cause unexpected node reboots and performance problems with RAC,
Oracle strongly advises to disable the use of Transparent
HugePages. In addition, Transparent Hugepages may cause problems
even in a single-instance database environment with unexpected
performance problems or delays. As such, Oracle recommends
disabling Transparent HugePages on all Database servers running
Oracle.
Note: This ONLY applies to the new feature Transparent HugePages, Oracle
highly recommends the use of standard HugePages that were
recommended for previous releases of Linux. See Document
361323.1 for additional information on
HugePages.
Note: on UEK2 and above, check
the existence of the /sys/kernel/mm/transparent_hugepage/
directory. If this directory does not exist, then
the transparent hugepage is removed from the kernel, so there is no
need to disable the transparent hugepage
OCCURRENCE
Starting with SLES11, RHEL6,
OL6, and UEK2 Kernels, Transparent HugePages are introduced and
enabled by default, and this can cause node reboots and performance
problems.
On UEK2 and above, the transparent hugepages is already
removed if /sys/kernel/mm/transparent_hugepage/ directory directory
does not exist. In that case, there is no further
need to disable the transparent hugepages.
SYMPTOMS
Node reboots.
The ocssd.log may show some of the threads are
blocked (but this does not show all the time):
2013-05-01
14:30:45.255:
[ CSSD][224204544]clssscMonitorThreads clssnmvKillBlockThread not
scheduled for 7500 msecs
2013-05-01 14:30:46.945:
[ CSSD][224204544]clssscMonitorThreads clssnmvWorkerThread not
scheduled for 8030 msecs
WORKAROUND
The information provided here for
Redhat and SLES needs to be confirmed by Redhat and SLES support
because Redhat and SLES support may provide a different ways to
disable the transparent hugepages.
Contact Redhat and SLES support on how to disable the transparent
hugepages on their respective platform.
The workaround provided here is
for Redhat 6 and lower, Oracle Linux 6 and lower, and SLES 11 and
lower
For Oracle Linux 7, Oracle published
MOS
Oracle Linux 7 - How to
disable Transparent HugePages for RHCK kernel?
Also, add the following line to /etc/default/grub file:
transparent_hugepage=never
For Redhat
7, Redhat published the following note: how to disable transparent
hugepages in Redhat 7.
For SLES 12, conatct SLES support for
the instruction on how to disable the transparent
hugepages
The
following instruction are for Redhat 6 or lower, Oracle Linux 6 or
lower, and SLES 11 or lower:
To check if the Transparent HugePages are enabled
in your server execute the following:
Default/Enabled setting
is [always]:
# cat
/sys/kernel/mm/transparent_hugepage/enabled
[always] never
Note
1: For RHEL kernel, the file path is
different from above:
# cat
/sys/kernel/mm/redhat_transparent_hugepage/enabled
[always] never
Please modify this file accordingly.
Note 2: For UEK2 kernel, as of
2.6.39-400.116.0 Transparent Hugepages has been removed from the
kernel. If it is not compiled into the kernel
then /sys/kernel/mm/transparent_hugepage will not exist.
Disabled setting is [never]:
# cat
/sys/kernel/mm/transparent_hugepage/enabled
always [never]
If "enabled"
is NOT set to
"[never]", the Transparent HugePages are being used.
You can also issue:
# grep AnonHugePages
/proc/meminfo
If the output contains a line like
"AnonHugepages: xxxx kB", with a value > 0kB, the kernel is
using Transparent HugePages.
Because the kernel currently
uses Transparent HugePages only for the anonymous
memory blocks like stack and heap, the value of AnonHugepages in
/proc/meminfo is the current amount of Transparent
HugePages that the kernel is using.
To disable Transparent HugePages boot time
either one of the following 2 methods may be used:
Add the following to the kernel boot line in
/etc/grub.conf (a symlink to /boot/grub/grub.conf) and reboot the
server (this is the preferred method):
transparent_hugepage=never
Once modified the line will read similar to
the following example:
title Oracle Linux
Server (2.6.32-300.25.1.el6uek.x86_64)
root (hd0,0)
kernel
/vmlinuz-2.6.32-300.25.1.el6uek.x86_64 ro root=LABEL=/
transparent_hugepage=never
initrd /initramfs-2.6.32-300.25.1.el6uek.x86_64.img
OR
Add the following lines in /etc/rc.local and
reboot the server (this still can be done on Redhat 7 although
rc.local is being deprecated):
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi
Please change the file path
for RHEL kernel to /sys/kernel/mm/redhat_transparent_hugepage/
accordingly.
===========================================================================
The following comment was received about
SLES11 SP3. Please note
that this is not tested, so this is provided only as a talking
point to your sys admin who knows SLES11 SP3 and who should also
contact SLES for correct information.
in SLES11 SP3 with
YAST/GRUB boot loader /etc/grub.conf overrided usind YAST.
parameter transparent_hugepage=never must be set in
YAST-bootloader- Edit settings - in line: optional kernel parameter
[before showopts].
PATCHES
none
HISTORY
06/05/2013 the alert is written
06/05/2013 the alert is published
06/06/2013 removed "cat
/sys/kernel/mm/transparent_hugepage/enabled" from the section where
we check the transparent hugepages are disabled.
06/06/2013 changed "Starting SLES11..." to 'Starting with
SLES11"
06/07/2013 changed title to include "ALERT", modified to cover
SIDB, corrected typo in 2nd solution, expanded the solution for
disable via kernel boot line and validation via
AnonHugePages.
11/26/2013 Include file path
/sys/kernel/mm/redhat_transparent_hugepage/enabled for RHEL kernel
and changes for UEK2 kernel
08/07/2017 Included references to Redhat 7, Oracle Linux 7, and
SLES 12. Also clarfied that the instruction in
this note is only for Redhat 6, Oracle Linux 6, and SLES 11 and
lower versions.
REFERENCES
NOTE:2066217.1- Oracle Linux
7 - How to disable Transparent HugePages for RHCK kernel?