Configuring HugePages for Oracle onLinux

原文链接：http://www.oracle-base.com/articles/linux/configuring-huge-pages-for-oracle-on-linux-64.php

 

Configuring HugePages for Oracle onLinux (x86-64)

• Introduction
• Configuring HugePages
• Force Oracle to use HugePages
• Disabling Transparent HugePages (RHEL6/OL6)
• Configuring 1G Hugepagesize

Introduction

For large SGA sizes, HugePages can givesubstantial benefits in virtual memory management. Without HugePages, thememory of the SGA is divided into 4K pages, which have to be managed by theLinux kernel. Using HugePages, the page size is increased to 2MB (configurableto 1G if supported by the hardware), thereby reducing the total number of pagesto be managed by the kernel and therefore reducing the amount of memoryrequired to hold the page table in memory. In addition to these changes, the memory associated with HugePages can not be swappedout, which forces the SGA to stay memory resident. The savings in memory and the effort ofpage management make HugePages pretty much mandatory for Oracle 11g systemsrunning on x86-64 architectures.

Hugepage的使用减少了对总pages管理的memory资源需求，需要注意的是SGA强制保留内存的分配的hugepages不能进行内存换出。

Note. Automatic Memory Management (AMM) is not compatible with LinuxHugePages, so apart from ASM instances and small unimportant databases, youwill probably have no need for AMM on a real database running on Linux.Instead, Automatic Shared Memory Management and Automatic PGA Management should be used as they arecompatible with HugePages.

Note：AMM和linux的hugepages不兼容。

Configuring HugePages

Run the following command to determine thecurrent HugePage usage. The default HugePage size is 2MB on Oracle Linux 5.xand as you can see from the output below, by default no HugePages are defined.

$ grep Huge /proc/meminfo

AnonHugePages:         0 kB

HugePages_Total:       0

HugePages_Free:        0

HugePages_Rsvd:        0

HugePages_Surp:        0

Hugepagesize:       2048 kB

$

Depending on the size of your SGA, you maywish to increase the value of Hugepagesize to 1G.

Hugepage基于你的SGA，你huagepagesize一般可以设置的比SGA大1G;

Create a file called"hugepages_setting.sh" with the following contents.

 

#这个脚本可以建议你可以配置多大的hugepages：

#!/bin/bash

#

# hugepages_settings.sh

#

# Linux bash script to compute values for the

# recommended HugePages/HugeTLB configuration

#

# Note: This script does calculation for all shared memory

# segments available when the script is run, no matter it

# is an Oracle RDBMS shared memory segment or not.

# Check for the kernel version

KERN=`uname -r | awk -F. '{ printf("%d.%d\n",$1,$2); }'`

# Find out the HugePage size

HPG_SZ=`grep Hugepagesize /proc/meminfo | awk {'print $2'}`

# Start from 1 pages to be on the safe side and guarantee 1 free HugePage

NUM_PG=1

# Cumulative number of pages required to handle the running shared memorysegments

for SEG_BYTES in `ipcs -m | awk {'print $5'} | grep"[0-9][0-9]*"`

do

   MIN_PG=`echo"$SEG_BYTES/($HPG_SZ*1024)" | bc -q`

   if [ $MIN_PG -gt 0 ]; then

      NUM_PG=`echo"$NUM_PG+$MIN_PG+1" | bc -q`

   fi

done

# Finish with results

case $KERN in

   '2.4') HUGETLB_POOL=`echo"$NUM_PG*$HPG_SZ/1024" | bc -q`;

          echo "Recommendedsetting: vm.hugetlb_pool = $HUGETLB_POOL" ;;

   '2.6' | '3.8') echo"Recommended setting: vm.nr_hugepages = $NUM_PG" ;;

    *) echo "Unrecognizedkernel version $KERN. Exiting." ;;

esac

# End

 

 

Note. Thanks to Bjoern Rost for pointing out the issuewhen using the script against UEK3 and the suggested fix.

Make the file executable.

$ chmod u+x hugepages_setting.sh

Make sure all the Oracle services arerunning as normal on the server, then run the script and make a note of therecommended "vm.nr_hugepages" value.

$ ./hugepages_setting.sh

Recommended setting: vm.nr_hugepages = 305

$

Edit the "/etc/sysctl.conf" fileas the "root" user, adding the following entry, adjusted based onyour output from the script. You should set the value greater than or equal tothe value displayed by the script. You only need 1 or 2 spare pages.

在"/etc/sysctl.conf"文件里配置huagepages：

 

vm.nr_hugepages=306

Run the following command as the"root" user.

说明hugepages的配置可以用sysctl –p进行实时生效

# sysctl -p

 

Alternatively, edit the"/etc/grub.conf" file, adding "hugepages=306" to the end ofthe kernel line for the default kernel and reboot.

 

You can now see the HugePages have beencreated, but are currently not being used.

你可以看到huagepages已经被创建，但是当前并没有被用到：

$ grep Huge /proc/meminfo

AnonHugePages:         0 kB

HugePages_Total:     306

HugePages_Free:      306

HugePages_Rsvd:        0

HugePages_Surp:        0

Hugepagesize:       2048 kB

$

Add the following entries into the"/etc/security/limits.conf" script, where the setting is at least thesize of the HugePages allocation in KB (HugePages * Hugepagesize). In this casethe value is 306*2048=626688.

Limits.conf详细配置请参考博客：http://blog.csdn.net/panfelix/article/details/21008887

vi  /etc/security/limits.con

 

 

* soft memlock 626688

* hard memlock 626688

Note. If you prefer, you can set theseparameters to a value just below the size of physical memory of the server.This way you can forget about it, unless you add more physical memory.

Check the MEMORY_TARGET parameters are notset for the database and SGA_TARGET and PGA_AGGREGATE_TARGET parameters arebeing used instead.

SQL> show parameter target

 

NAME                                TYPE        VALUE

------------------------------------ -----------------------------------------

archive_lag_target                  integer     0

db_flashback_retention_target       integer     1440

fast_start_io_target                integer     0

fast_start_mttr_target              integer     0

memory_max_target                   big integer 0

memory_target                       big integer 0

parallel_servers_target             integer     16

pga_aggregate_target                big integer 200M

sga_target                          big integer 600M

SQL>

Restart the server and restart thedatabase services as required.

Check the HugePages information again.

$ grep Huge /proc/meminfo

AnonHugePages:         0 kB

HugePages_Total:     306

HugePages_Free:       98

HugePages_Rsvd:       93

HugePages_Surp:        0

Hugepagesize:       2048 kB

$

You can see the HugePages are now beingused.

Remember, if you increase your memoryallocation or add new instances, you need to retest the required number ofHugePages, or risk Oracle running without them.

Force Oracle to use HugePages强制oracle使用hugepages

Sizing the number of HugePages correctlyis important because if the whole SGA doesn't fit into the available HugePages,the instance will start up without using any. This can be a big problem, butnot necessarily be obvious to spot. Later releases of the database display a"Large Pages Information" section in the alert log during startup.

****************** Large Pages Information *****************

 

Total Shared Global Region in Large Pages = 602 MB (100%)

 

Large Pages used by this instance: 301 (602 MB)

Large Pages unused system wide = 5 (10 MB) (alloc incr 4096 KB)

Large Pages configured system wide = 306 (612 MB)

Large Page size = 2048 KB

***********************************************************

If you are running Oracle 11.2.0.2 orlater, you can set the USE_LARGE_PAGES initialization parameter to"only" so the database fails to start if it is not backed byhugepages. You can read more about this here.

ALTER SYSTEM SET use_large_pages=only SCOPE=SPFILE;

SHUTDOWN IMMEDIATE;

STARTUP;

On startup the "Large PageInformation" in the alert log reflects the use of this parameter.

****************** Large Pages Information *****************

Parameter use_large_pages = ONLY

 

Total Shared Global Region in Large Pages = 602 MB (100%)

 

Large Pages used by this instance: 301 (602 MB)

Large Pages unused system wide = 5 (10 MB) (alloc incr 4096 KB)

Large Pages configured system wide = 306 (612 MB)

Large Page size = 2048 KB

***********************************************************

Attempting to start the database whenthere aren't enough HugePages to hold the SGA will now return the followingerror.

SQL> STARTUP

ORA-27137: unable to allocate large pages to create a shared memorysegment

Linux-x86_64 Error: 12: Cannot allocate memory

SQL>

The "Large Pages Information"section of the alert log output describes the startup failure and theappropriate action to take.

****************** Large Pages Information *****************

Parameter use_large_pages = ONLY

 

Large Pages unused system wide = 0 (0 KB) (alloc incr 4096 KB)

Large Pages configured system wide = 0 (0 KB)

Large Page size = 2048 KB

 

ERROR:

  Failed to allocate shared globalregion with large pages, unix errno = 12.

  Aborting Instance startup.

  ORA-27137: unable to allocateLarge Pages to create a shared memory segment

 

ACTION:

  Total Shared Global Region size is608 MB. Increase the number of

  unused large pages to atleast 304(608 MB) to allocate 100% Shared Global

  Region with Large Pages.

***********************************************************

Disabling Transparent HugePages(RHEL6/OL6)

Starting from RHEL6/OL6, TransparentHugePages are implemented and enabled by default. They are meant to improvememory management by allowing HugePages to be allocated dynamically by the"khugepaged" kernel thread, rather than at boot time likeconventional HugePages. That sounds like a good idea, but unfortunatelyTransparent HugePages don't play well with Oracle databases and are associatedwith node reboots in RAC installations and performance problems on both singleinstance and RAC installations. As a result Oracle recommends disablingTransparent HugePages on all servers running Oracle databases, as described inthis MOS note.

• ALERT: Disable Transparent HugePages on SLES11, RHEL6, OEL6 and UEK2 Kernels [ID 1557478.1]

You can check the current setting usingthe following command, which is displaying the default value of"enabled=[always]".

# cat /sys/kernel/mm/transparent_hugepage/enabled

[always] madvise never

#

The preferred method to disableTransparent HugePages is to add "transparent_hugepage=never" to thekernel boot line in the "/etc/grub.conf" file.

title Oracle Linux Server (2.6.39-400.24.1.el6uek.x86_64)

        root (hd0,0)

        kernel/vmlinuz-2.6.39-400.24.1.el6uek.x86_64 ro root=/dev/mapper/vg_ol6112-lv_rootrd_NO_LUKS  KEYBOARDTYPE=pc KEYTABLE=uk

LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16  rd_NO_DM rd_LVM_LV=vg_ol6112/lv_swaprd_LVM_LV=vg_ol6112/lv_root rhgb quiet numa=off

transparent_hugepage=never

        initrd/initramfs-2.6.39-400.24.1.el6uek.x86_64.img

The server must be rebooted for this totake effect.

Alternatively, add the following linesinto the "/etc/rc.local" file and reboot the server.

if test -f /sys/kernel/mm/transparent_hugepage/enabled; then

   echo never >/sys/kernel/mm/transparent_hugepage/enabled

fi

if test -f /sys/kernel/mm/transparent_hugepage/defrag; then

   echo never >/sys/kernel/mm/transparent_hugepage/defrag

fi

Whichever method you choose, remember tocheck the change has work after reboot.

# cat /sys/kernel/mm/transparent_hugepage/enabled

always madvise [never]

#

With Transparent HugePages disabled, youshould proceed to configure conventional HugePages, as described above.

Configuring 1G Hugepagesize

Note. As mentioned by Eugene in thecomments, Oracle currently don't recommend using 1G Hugepagesize. You can readmore about this in MOS Doc ID 1607545.1. With that in mind, the rest of thissection should probably be considered more of an academic exercise.

Check if your current hardware can supporta Hugepagesize of 1G. If the following command produces any output, it can.

# cat /proc/cpuinfo | grep pdpe1gb

Thanks to Kevin Closson for pointing out the hardwaresupport requirement.

Edit the"/etc/grub.conf" file, adding the following entries on to the kernel line ofthe default grub entry. Adjust the "hugepages" entry to the desirednumber of 1G pages. Notice this includes the disabling of TransparentHugePages, which is not mandatory, but a good idea.

transparent_hugepage=never hugepagesz=1Ghugepages=1 default_hugepagesz=1G

Check the current HugePages setup.

# grep Huge /proc/meminfo

HugePages_Total:       0

HugePages_Free:        0

HugePages_Rsvd:        0

HugePages_Surp:        0

Hugepagesize:       2048 kB

#

Reboot and check the HugePages setupagain.

#  grep Huge /proc/meminfo

HugePages_Total:       1

HugePages_Free:        1

HugePages_Rsvd:        0

HugePages_Surp:        0

Hugepagesize:    1048576 kB

#

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

G Very Large Memory andHugePages

This chapter guidesLinux system administrators to configure very large memory configurations andHugePages on Linux systems.

This chapter containsthe following sections:

·        Very Large Memory onLinux x86

·        Overview of HugePages

G.1 Very Large Memory on Linux x86

Very Large Memory (VLM)configurations allow a 32-bit Oracle Database to access more than 4GB RAM thatis traditionally available to Linux applications. The Oracle VLM option for32-bit creates a large database buffer cache using an in-memory file system (/dev/shm). Otherparts of the SGA are allocated from regular memory. VLM configurations improvedatabase performance by caching more database buffers in memory, whichsignificantly reduces the disk I/O compared to configurations without VLM. Thischapter shows how to increase the SGA memory using VLM on a 32-bit computer.

Note:

The contentsdocumented in this section apply only to 32-bit Linux operating system. With a64-bit architecture, VLM support is available natively. All 64-bit Linuxoperating systems use the physical memory directly, as the maximum availablevirtual address space is 16 EB (exabyte = 2^60 bytes.)

This section includesthe following topics:

·        Implementing VLM on32-bit Linux

·        Prerequisites forImplementing VLM

·        Methods To Increase SGALimits

·        Configuring Very LargeMemory for Oracle Database

·        Restrictions Involvedin Implementing Very Large Memory

G.1.1 Implementing VLM on 32-bit Linux

With 32-bitarchitectures, VLM is accessed through a VLM window of a specific size. The VLMwindow is a data structure in the process address space that provides access tothe whole virtual address space from a window of a specific size. On 32-bitLinux, you must set the parameterUSE_INDIRECT_DATA_BUFFERS=TRUE, and mounta shmfs or tmpfs or ramfs type of in-memory filesystem over /dev/shm to increase the usable address space.

G.1.2 Prerequisites for Implementing VLM

The following are someof the prerequisites for implementing VLM on a 32-bit operating system:

·        The computer on which Oracle Database is installed must havemore than 4GB of memory.

·        The computer must be configured to use a kernel with PAE supportupon startup.

·        The USE_INDIRECT_DATA_BUFFERS=TRUE must be present in the initialization parameter file for thedatabase instance that uses VLM support.

·        Initialization parameters DB_BLOCK_BUFFERS and DB_BLOCK_SIZE must be set to values you have chosen for the Oracle Database.

G.1.3 Methods To Increase SGA Limits

In a typical 32-bitLinux kernel, one can create an SGA of up to 2.4GB size. Using a Linux Hugememkernel enables the creation of an SGA of upto 3.2GB size. To go beyond 3.2GB ona 32-bit kernel, you must use the VLM feature.

The following are the methodsto increase SGA limits on a 32-bit computer:

·        Hugemem Kernel

·        Hugemem Kernel withVery Large Memory

G.1.3.1 Hugemem Kernel

Red Hat EnterpriseLinux 4 and Oracle Linux 4 include a new kernel known as the Hugemem kernel.The Hugemem kernel feature is also called a 4GB-4GB Split Kernel as it supportsa 4GB per process user space (versus 3GB for the other kernels), and a 4GBdirect kernel space. Using this kernel enables RHEL 4/Oracle Linux 4 to run onsystems with up to 64GB of main memory. The Hugemem kernel is required to useall the memory in system configurations containing more than 16GB of memory.The Hugemem kernel can run configurations with less memory.

A classic 32-bit 4GBvirtual address space is split 3GB for user processes and 1GB for the kernel.The new scheme (4GB/4GB) permits 4GB of virtual address space for the kerneland almost 4GB for each user process. Due to this scheme with hugemem kernel,3.2GB of SGA can be created without using the indirect data buffer method.

Note:

Red HatEnterprise Linux 5/ Oracle Linux 5 and Red Hat Enterprise Linux 6/ Oracle Linux6 on 32-bit does not have the hugemem kernel. It supports only the 3GB userprocess/ 1GB kernel split. It has a PAE kernel that supports systems with morethan 4GB of RAM and reliably upto 16GB. Since this has a 3GB/1GB kernel split,the system may run out of lowmem if the system's load consumes lots of lowmem.There is no equivalent kernel for hugemem in Enterprise Linux 5 and one isrecommended to either use Enterprise Linux 4 with hugemem or go for 64-bit.

The Hugemem kernel onlarge computers ensures better stability as compared to the performanceoverhead of address space switching.

Run the followingcommand to determine if you are using the Hugemem kernel:

$ uname -r

2.6.9-5.0.3.ELhugemem

G.1.3.2 Hugemem Kernel with Very Large Memory

If you use only Hugememkernels on 32-bit systems, then the SGA size can be increased but notsignificantly. Refer to section "HugememKernel", for more information.

Note:

Red HatEnterprise Linux 5/ Oracle Linux 5 and Red Hat Enterprise Linux 6/ Oracle Linux6 does not support the hugemem kernel. It supports a PAE kernel that can beused to implement Very Large Memory (VLM) as long as the physical memory doesnot exceed 16GB.

This section shows howthe SGA can be significantly increased by using Hugemem kernel with VLM on32-bit systems.

The SGA can beincreased to about 62GB (depending on block size) on a 32-bit system with 64GBRAM. A processor feature called Page Address Extension (PAE) permits you tophysically address 64GB of RAM. Since PAE does not enable a process or programto either address more than 4GB directly, or have a virtual address spacelarger than 4GB, a process cannot attach to shared memory directly. To addressthis issue, a shared memory filesystem (memory-based filesystem) must becreated which can be as large as the maximum allowable virtual memory supportedby the kernel. With a shared memory filesystem processes can dynamically attachto regions of the filesystem allowing applications like Oracle to havevirtually a much larger shared memory on 32-bit operating systems. This is notan issue on 64-bit operating systems.

VLM moves the databasebuffer cache part of the SGA from the System V shared memory to the sharedmemory filesystem. It is still considered one large SGA but it consists now oftwo different operating system shared memory entities. VLM uses 512MB of the non-buffercache SGA to manage VLM. This memory area is needed for mapping the indirectdata buffers (shared memory filesystem buffers) into the process address spacesince a process cannot attach to more than 4GB directly on a 32-bit system.

Note:

USE_INDIRECT_DATA_BUFFERS=TRUE must be present in the initialization parameter file for thedatabase instance that use Very Large Memory support. If this parameter is notset, then Oracle Database 11g Release 2 (11.2) or later behaves inthe same way as previous releases.

You must also manuallyset the initialization parameters DB_BLOCK_BUFFERS and SHARED_POOL_SIZE to values you have chosen for an Oracle Database. AutomaticMemory Management (AMM) cannot be used. The initialization parameter DB_BLOCK_SIZE sets the block size and in combination with DB_BLOCK_BUFFERS determines the buffer cache size for an instance

For example, if thenon-buffer cache SGA is 2.5GB, then you will only have 2GB of non-buffer cacheSGA for shared pool, large pool, and redo log buffer since 512MB is used formanaging VLM. It is not recommended to use VLM if buffer cache size is lessthan 512MB.

In RHEL 4/ Oracle Linux4 there are two different memory file systems that can be used for VLM:

·        tmpfs or shmfs: mount a shmfs with a certain size to /dev/shm, and setthe correct permissions. For tmpfs you do not need to specify a size. Tmpfs orshmfs allocated memory is pageable.

For example:

Example Mount shmfs:

# mount -t shm shmfs -o size=20g /dev/shm

Edit /etc/fstab:

shmfs /dev/shm shm size=20g 0 0

OR

Example Mount tmpfs:

# mount –t tmpfs tmpfs /dev/shm

Edit /etc/fstab:

none /dev/shm tmpfs defaults 0 0

·        ramfs: ramfs is similar to shmfs, exceptthat pages are not pageable or swappable. This approach provides the commonlydesired effect. ramfs is created by:

·         umount /dev/shm

·         mount -t ramfs ramfs /dev/shm

G.1.4 Configuring Very Large Memory for Oracle Database

Complete the followingprocedure to configure Very Large Memory on Red Hat Enterprise Linux 4/ OracleLinux 4 using ramfs:

1.   Log in as a root user:

2.  sudo -sh

3.  Password:

4.   Edit the /etc/rc.local file and add the following entries to it to configure thecomputer to mount ramfs over the /dev/shm directory, whenever you start the computer:

5.  umount /dev/shm

6.  mount -t ramfs ramfs /dev/shm

7.  chown oracle:oinstall /dev/shm

In thepreceding commands, oracle is the owner of Oracle software files and oinstall is the group for Oracle owner account. If the new configurationdisables /etc/rc.local file or you start an instance of Oracle database using a Linuxservice script present under the /etc/init.d file, then you can add those entries in the service script too.

Note, thisconfiguration will make ramfs ready even before your system autostarts crucial Oracle Databaseinstances. The commands can also be included in your startup scripts. It isimportant that you test the commands extensively by repeated restart action,after you complete configuring the computer using the following steps:

8.   Restart the server.

9.   Log in as a root user.

10.Run the following command to check if the /dev/shm directory is mounted with the ramfs type:

11. /dev/shm directory is mounted with the ramfs type:

12.  

13. # mount | grep shm

14. ramfs on /dev/shm type ramfs (rw)

15.Run the following command to check the permissions on the /dev/shm directory:

16. # ls -ld /dev/shm

17. drwxr-xr-x  3 oracle oinstall 0 Jan 13 12:12 /dev/shm

18.Edit the /etc/security/limits.conf file and add the following entries to it to increase the maxlocked memory limit:

19. soft    memlock        3145728

20. hard    memlock        3145728

21.Switch to the oracle user:

22. # sudo - oracle

23. Password:

24.Run the following command to check the max locked memory limit:

25. $ ulimit -l

26. 3145728

27.Complete the following procedure to configure instanceparameters for Very Large Memory:

a.   Replace the DB_CACHE_SIZE, DB_xK_CACHE_SIZE, sga_target, and memory_target parameters with DB_BLOCK_BUFFERS parameter.

b.   Add the USE_INDIRECT_DATA_BUFFERS=TRUE parameter.

c.   Configure SGA size according to the SGA requirements.

d.   Remove SGA_TARGET, MEMORY_TARGET, or MEMORY_MAX_TARGET parameters, if set.

28.Start the database instance.

29.Run the following commands to check the memory allocation:

30. $ ls -l /dev/shm

31. $ ipcs -m

See Also:

"ConfiguringHugePages on Linux" section formore information about HugePages.

G.1.5 Restrictions Involved in Implementing Very Large Memory

Following are thelimitations of running a computer in the Very Large Memory mode:

·        You cannot use Automatic Memory Management (AMM) while implementingVLM using ramfs, becauseAMM works on dynamic SGA tuning. With AMM swapping is possible. For example,you can unmap the unused SGA space and map it to PGA. Dynamic SGA and multipleblock size are not supported with Very Large Memory because ramfs is not swappable. To enable Very Large Memory, you must ensurethat you set the value of MEMORY_TARGET to zero.

·        VLM can be implemented only if Database Buffer Cache size isgreater than 512MB.

G.2 Overview of HugePages

HugePages is a featureintegrated into the Linux kernel 2.6. Enabling HugePages makes it possible forthe operating system to support memory pages greater than the default (usually4KB). Using very large page sizes can improve system performance by reducingthe amount of system resources required to access page table entries. HugePagesis useful for both 32-bit and 64-bit configurations. HugePage sizes vary from2MB to 256MB, depending on the kernel version and the hardware architecture.For Oracle Databases, using HugePages reduces the operating system maintenanceof page states, and increases Translation Lookaside Buffer (TLB) hit ratio.

This section includesthe following topics:

·        Tuning SGA WithHugePages

·        Configuring HugePageson Linux

·        Restrictions forHugePages Configurations

G.2.1 Tuning SGA With HugePages

Without HugePages, theoperating system keeps each 4KB of memory as a page, and when it is allocatedto the SGA, then the lifecycle of that page (dirty, free, mapped to a process,and so on) is kept up to date by the operating system kernel.

With HugePages, theoperating system page table (virtual memory to physical memory mapping) issmaller, since each page table entry is pointing to pages from 2MB to 256MB.Also, the kernel has fewer pages whose lifecyle must be monitored.

Note:

2MB size of HugePages is available with Linux x86-64, Linux x86, andIBM: Linux on System z.

The following are theadvantages of using HugePages:

·        Increasedperformance through increased TLB hits.

·        Pages are lockedin memory and are never swapped out which guarantees that shared memory likeSGA remains in RAM.

·        Contiguous pagesare preallocated and cannot be used for anything else but for System V sharedmemory (for example, SGA)

·        Less bookkeepingwork for the kernel for that part of virtual memory due to larger page sizes

G.2.2 Configuring HugePages on Linux

Complete the followingsteps to configure HugePages on the computer:

1.   Edit the memlock setting in the /etc/security/limits.conf file. The memlock setting is specified in KB and set slightly lesser than theinstalled RAM. For example, if you have 64GB RAM installed, add the followingentries to increase the max locked memory limit:

2.  *   soft   memlock    60397977

3.  *   hard   memlock    60397977

You can alsoset the memlock value higher than your SGA requirements.

4.   Login as the oracle user again and run the ulimit-l command toverify the new memlock setting:

5.  $ ulimit -l

6.  60397977

7.   Run the following command to display the value of Hugepagesize variable:

8.  $ grep Hugepagesize /proc/meminfo

9.   Complete the following procedure to create a script thatcomputes recommended values for hugepages configuration for the current shared memory segments:

Note:

Following is an examplethat may require modifications.

a.   Create a text file named hugepages_settings.sh.

b.   Add the following content in the file:

c.  #!/bin/bash

d.  #

e.  # hugepages_settings.sh

f.  #

g.  # Linux bash script to compute values for the

h.  # recommended HugePages/HugeTLB configuration

i.  #

j.  # Note: This script does calculation for all shared memory

k.  # segments available when the script is run, no matter it

l.  # is an Oracle RDBMS shared memory segment or not.

m.  # Check for the kernel version

n.  KERN=`uname -r | awk -F. '{ printf("%d.%d\n",$1,$2); }'`

o.  # Find out the HugePage size

p.  HPG_SZ=`grep Hugepagesize /proc/meminfo | awk {'print $2'}`

q.  # Start from 1 pages to be on the safe side and guarantee 1 free HugePage

r.  NUM_PG=1

s.  # Cumulative number of pages required to handle the running shared memory segments

t.  for SEG_BYTES in `ipcs -m | awk {'print $5'} | grep "[0-9][0-9]*"`

u.  do

v.     MIN_PG=`echo "$SEG_BYTES/($HPG_SZ*1024)" | bc -q`

w.     if [ $MIN_PG -gt 0 ]; then

x.        NUM_PG=`echo "$NUM_PG+$MIN_PG+1" | bc -q`

y.     fi

z.  done

aa. # Finish with results

bb. case $KERN in

cc.    '2.4') HUGETLB_POOL=`echo "$NUM_PG*$HPG_SZ/1024" | bc -q`;

dd.           echo "Recommended setting: vm.hugetlb_pool = $HUGETLB_POOL" ;;

ee.    '2.6') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;

ff.     *) echo "Unrecognized kernel version $KERN. Exiting." ;;

gg. esac

hh. # End

ii.   Run the following command to change the permission of the file:

jj. $ chmod +x hugepages_settings.sh

10.Run the hugepages_settings.sh script to compute the values for hugepages configuration:

11. $ ./hugepages_settings.sh

12.Set the following kernel parameter:

13. # sysctl -w vm.nr_hugepages=value_displayed_in_step_5

14.To make the value of the parameter available for every time yourestart the computer, edit the /etc/sysctl.conf file and add the following entry:

15. vm.nr_hugepages=value_displayed_in_step_5

16.Restart the server.

Note:

Tocheck the available hugepages, run the following command:

$ grep Huge /proc/meminfo

G.2.3 Restrictions for HugePages Configurations

Following are thelimitations of using HugePages:

·        Automatic Memory Management (AMM) and HugePages are not compatible.When you use AMM, the entire SGA memory is allocated by creating files under /dev/shm. When OracleDatabase allocates SGA with AMM, HugePages are not reserved. To use HugePageson Oracle Database 12c, You must disable AMM.

·        If you are using VLM in a 32-bit environment, then you cannot useHugePages for the Database Buffer cache. You can use HugePages for other partsof the SGA, such as shared_pool, large_pool, and so on.Memory allocation for VLM (buffer cache) is done using shared memory filesystems (ramfs/tmpfs/shmfs). Memory file systems do not reserve or useHugePages.

·        HugePages are not subject to allocation or release after systemstartup, unless a system administrator changes the HugePages configuration,either by modifying the number of pages available, or by modifying the poolsize. If the space required is not reserved in memory during system startup,then HugePages allocation fails.