linux配置大内存技术
1、理论介绍
系统进程通过虚拟地址访问内存,CPU必须把它转换成物理内存地址才能真正访问内存。为了提高这个转换效率,CPU会缓存最近的虚拟内存地址和物理内存地址的映射关系,并保存在一个
由CPU维护的映射表中。为了尽量提高内存的访问速度,需要在映射表中保存尽量多的映射关系。在linux系统中,所有内存都是以页的形式划分的,默认情况下每页是4096B,这就意味着如果
物理内存很大,映射表的条目数会很多,系统cpu在检索的过程中,需要检索的条目数就会很。从另一个角度说,如果能够让cpu减少检索内存条目的数量,可采取的办法只有增加页的尺寸,
hugepage应时而生。hugepage,一直被cache,不会被交换出去,也就是说使用hugepage的内存不能被其他的进程使用。随着内存技术的发展,现在上T的内存配置也出现,所以内存已经不是问
题,在实际应用中,如果你的物理内存能够达到8g以上,那么都可以使用大内存技术。在oracle数据库中,sga与大内存配合使用,将会对性能产生非常大的提升。
2、oracle中大内存技术的配置建议
oracle针对大内存技术,提供了一个脚本如下:
#!/bin/bash
#
# hugepages_settings.sh
#
# Linux bash script to compute values for the
# recommended HugePages/HugeTLB configuration
#
# Note: This script does calculation for all shared memory
# segments available when the script is run, no matter it
# is an Oracle RDBMS shared memory segment or not.
#
# This script is provided by Doc ID 401749.1 from My Oracle Support
#
# Welcome text
echo "
This script is provided by Doc ID 401749.1 from My Oracle Support
() where it is intended to compute values for
the recommended HugePages/HugeTLB configuration for the current shared
memory segments. Before proceeding with the execution please make sure
that:
* Oracle Database instance(s) are up and running
* Oracle Database 11g Automatic Memory Management (AMM) is not setup
(See Doc ID 749851.1)
* The shared memory segments can be listed by command:
# ipcs -m
Press Enter to proceed..."
read
# Check for the kernel version
KERN=`uname -r | awk -F. '{ printf("%d.%d\n",$1,$2); }'`
# Find out the HugePage size
HPG_SZ=`grep Hugepagesize /proc/meminfo | awk '{print $2}'`
# Initialize the counter
NUM_PG=0
# Cumulative number of pages required to handle the running shared memory segments
for SEG_BYTES in `ipcs -m | awk '{print $5}' | grep "[0-9][0-9]*"`
do
MIN_PG=`echo "$SEG_BYTES/($HPG_SZ*1024)" | bc -q`
if [ $MIN_PG -gt 0 ]; then
NUM_PG=`echo "$NUM_PG+$MIN_PG+1" | bc -q`
fi
done
RES_BYTES=`echo "$NUM_PG * $HPG_SZ * 1024" | bc -q`
# An SGA less than 100MB does not make sense
# Bail out if that is the case
if [ $RES_BYTES -lt 100000000 ]; then
echo "***********"
echo "** ERROR **"
echo "***********"
echo "Sorry! There are not enough total of shared memory segments allocated for
HugePages configuration. HugePages can only be used for shared memory segments
that you can list by command:
# ipcs -m
of a size that can match an Oracle Database SGA. Please make sure that:
* Oracle Database instance is up and running
* Oracle Database 11g Automatic Memory Management (AMM) is not configured"
exit 1
fi
# Finish with results
case $KERN in
'2.4') HUGETLB_POOL=`echo "$NUM_PG*$HPG_SZ/1024" | bc -q`;
echo "Recommended setting: vm.hugetlb_pool = $HUGETLB_POOL" ;;
'2.6') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;
*) echo "Unrecognized kernel version $KERN. Exiting." ;;
esac
# End
[root@hdyydb1 u02]# chmod u+x hugepages_settings.sh
[root@hdyydb1 u02]# ./hugepages_settings.sh
This script is provided by Doc ID 401749.1 from My Oracle Support
() where it is intended to compute values for
the recommended HugePages/HugeTLB configuration for the current shared
memory segments. Before proceeding with the execution please make sure
that:
* Oracle Database instance(s) are up and running
* Oracle Database 11g Automatic Memory Management (AMM) is not setup
(See Doc ID 749851.1)
* The shared memory segments can be listed by command:
# ipcs -m
Press Enter to proceed...
Recommended setting: vm.nr_hugepages = 2588
可以看到,运行脚本显示,将vm.nr_hugepages设置为2588
3、hugepage在oracle中的配置应用
--检查系统是否启用hugepage,或者是否可以启用
[root@hdyydb1 u02]# cat /proc/meminfo | grep -i huge
AnonHugePages: 75776 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
可以看到,HugePages_Total,HugePages_Free均为零,证明没有使用,但可以配置。如果上面什么都没有显示,证明不能配置。
--检查数据库的参数设置是否支持
SQL> show parameter sga_target
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
sga_target big integer 0
大内存技术,不支持内存自动管理,必须关闭AMM(自动内存管理)特性才能使用hugepage
调整方法如下:
ALTER SYSTEM SET sga_max_size = xxxxM SCOPE=SPFILE;
ALTER SYSTEM SET sga_target = xxxxM SCOPE=SPFILE;
ALTER SYSTEM SET PGA_AGGREGATE_TARGET = xxxxM SCOPE=SPFILE;
ALTER SYSTEM SET memory_target = 0 SCOPE=SPFILE;
ALTER SYSTEM SET memory_max_target = 0 SCOPE=SPFILE;
--根据第二步的验证结果进行配置
[root@hdyydb1 u02]# vi /etc/sysctl.conf
把vm.nr_hugepages = 2588 添加进去,并保存
[root@hdyydb1 u02]# sysctl -p
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0
kernel.msgmnb = 65536
kernel.msgmax = 65536
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 7247757312
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586
vm.min_free_kbytes = 51200
vm.nr_hugepages = 2588
检查设置:
[root@hdyydb1 u02]# cat /proc/meminfo | grep -i huge
AnonHugePages: 75776 kB
HugePages_Total: 977
HugePages_Free: 977
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
--锁定内存配置
如果不配置锁定内存,在后alert_sid.log中会有类似如下的建议:
RECOMMENDATION:
Total System Global Area size is 5 GB. For optimal performance,
prior to the next instance restart:
1. Large pages are automatically locked into physical memory.
Increase the per process memlock (soft) limit to at least 5 GB to lock
100% System Global Area's large pages into physical memory
并且,hugepage并没有真正使用起来!
主要是配置
oracle soft memlock
oracle hard memlock
注意,这里设置的值均以kb为单位的!
实际物理内存 > 锁定内存 >=HugePages_Total*Hugepagesize
[root@hdyydb1 u02]# vim /etc/security/limits.conf
添加:
oracle soft memlock 6291456
oracle hard memlock 6291456
我的物理内存是8g,sga 5g左右
4、重启操作系统、数据库后生效
[oracle@hdyydb1 ~]$ cat /proc/meminfo | grep -i huge
AnonHugePages: 43008 kB
HugePages_Total: 2588
HugePages_Free: 2168
HugePages_Rsvd: 2165
HugePages_Surp: 0
Hugepagesize: 2048 kB
free比total小,证明已经生效