Postgresql 启动报错 FATAL: could not map anonymous shared memory

FATAL: could not map anonymous shared memory: Cannot allocate memory
HINT: This error usually means that PostgreSQL’s request for a shared memory segment exceeded available memory, swap space, or huge pages. To reduce the request size (currently 8757182464 bytes), reduce PostgreSQL’s shared memory usage, perhaps by reducing shared_buffers or max_connections.

修改大页后,修改了shared_buffers参数为8G,启动数据库报错
解决方法:
修改内核参数
vm.overcommit_memory=2
或者修改
echo -1000 > /proc/self/oom_score_adj
或者减少shared buffers参数设置

If PostgreSQL itself is the cause of the system running out of memory, you can avoid the problem by changing your configuration. In some cases, it may help to lower memory-related configuration parameters, particularly shared_buffers and work_mem. In other cases, the problem may be caused by allowing too many connections to the database server itself. In many cases, it may be better to reduce max_connections and instead make use of external connection-pooling software.

On Linux 2.6 and later, it is possible to modify the kernel’s behavior so that it will not “overcommit” memory. Although this setting will not prevent the OOM killer from being invoked altogether, it will lower the chances significantly and will therefore lead to more robust system behavior. This is done by selecting strict overcommit mode via sysctl:

sysctl -w vm.overcommit_memory=2 –临时修改,重启无效
vi /etc/sysctl.conf 修改配置文件
加入vm.overcommit_memory=2
sysctl -p 载入配置文件,永久生效

or placing an equivalent entry in /etc/sysctl.conf. You might also wish to modify the related setting vm.overcommit_ratio. For details see the kernel documentation file Documentation/vm/overcommit-accounting.

Another approach, which can be used with or without altering vm.overcommit_memory, is to set the process-specific oom_score_adj value for the postmaster process to -1000, thereby guaranteeing it will not be targeted by the OOM killer. The simplest way to do this is to execute

echo -1000 > /proc/self/oom_score_adj
in the postmaster’s startup script just before invoking the postmaster. Note that this action must be done as root, or it will have no effect; so a root-owned startup script is the easiest place to do it. If you do this, you may also wish to build PostgreSQL with -DLINUX_OOM_SCORE_ADJ=0 added to CPPFLAGS. That will cause postmaster child processes to run with the normal oom_score_adj value of zero, so that the OOM killer can still target them at need.

Older Linux kernels do not offer /proc/self/oom_score_adj, but may have a previous version of the same functionality called /proc/self/oom_adj. This works the same except the disable value is -17 not -1000. The corresponding build flag for PostgreSQL is -DLINUX_OOM_ADJ=0.

https://www.kernel.org/doc/Documentation/vm/overcommit-accounting

vm.overcommit_memory的值为0时,不允许用户overcommit
vm.overcommit_memory的值为1时,总是可以overcommit
vm.overcommit_memory的值为2时,不允许超过swap+物理内存的*50%(overcommit_ratio决定这个百分比)

The Linux kernel supports the following overcommit handling modes

0 - Heuristic overcommit handling. Obvious overcommits of
address space are refused. Used for a typical system. It
ensures a seriously wild allocation fails while allowing
overcommit to reduce swap usage. root is allowed to
allocate slightly more memory in this mode. This is the
default.

1 - Always overcommit. Appropriate for some scientific
applications. Classic example is code using sparse arrays
and just relying on the virtual memory consisting almost
entirely of zero pages.

2 - Don’t overcommit. The total address space commit
for the system is not permitted to exceed swap + a
configurable amount (default is 50%) of physical RAM.
Depending on the amount you use, in most situations
this means a process will not be killed while accessing
pages but will receive errors on memory allocation as
appropriate.

    Useful for applications that want to guarantee their
    memory allocations will be available in the future
    without having to initialize every page.

The overcommit policy is set via the sysctl `vm.overcommit_memory’.

The overcommit amount can be set via vm.overcommit_ratio' (percentage)
or
vm.overcommit_kbytes’ (absolute value).

The current overcommit limit and amount committed are viewable in
/proc/meminfo as CommitLimit and Committed_AS respectively.

Gotchas

The C language stack growth does an implicit mremap. If you want absolute
guarantees and run close to the edge you MUST mmap your stack for the
largest size you think you will need. For typical stack usage this does
not matter much but it’s a corner case if you really really care

In mode 2 the MAP_NORESERVE flag is ignored.

How It Works

The overcommit is based on the following rules

For a file backed map
SHARED or READ-only - 0 cost (the file is the map not swap)
PRIVATE WRITABLE - size of mapping per instance

For an anonymous or /dev/zero map
SHARED - size of mapping
PRIVATE READ-only - 0 cost (but of little use)
PRIVATE WRITABLE - size of mapping per instance

Additional accounting
Pages made writable copies by mmap
shmfs memory drawn from the same pool

Status

o We account mmap memory mappings
o We account mprotect changes in commit
o We account mremap changes in size
o We account brk
o We account munmap
o We report the commit status in /proc
o Account and check on fork
o Review stack handling/building on exec
o SHMfs accounting
o Implement actual limit enforcement

To Do

o Account ptrace pages (this is hard)

参考:
https://www.kernel.org/doc/Documentation/vm/overcommit-accounting
http://www.postgresql.org/docs/9.4/static/kernel-resources.html –17.4.3 linux memory overcommit 章节

当你在 CentOS 上遇到 "ERROR: Could not build wheels for psycopg2, which is required to install pyproject.toml-based projects" 这样的错误时,这通常意味着你在尝试安装某个 Python 包(在这个例子中是 `psycopg2`),但是编译过程中遇到了问题,可能是因为缺少必要的依赖或库,或者版本不兼容。 具体可能的原因和解决方案如下: 1. **缺失依赖**:检查你的系统是否已经安装 PostgreSQL 及其 development headers。如果没有,你需要先安装它们: ``` sudo yum install postgresql-devel ``` 2. **编译器问题**:确保你的编译器(如 GCC 或者 C++ 编译器)是最新且可用的。如果需要更新,请运行: ``` sudo yum update gcc-c++ ``` 3. **Python版本冲突**:确认你的 Python 环境是否正确。有时,不同版本的 Python 可能需要不同的 PostgreSQL 驱动。尝试创建一个虚拟环境并激活: ```bash python3 -m venv your_venv source your_venv/bin/activate ``` 4. **pip 安装选项**:在安装时,你可以指定使用 system-wide 的 PostgreSQL 安装,而不是系统的默认: ```bash pip install psycopg2-binary ``` 5. **网络问题**:如果所有本地尝试都失败,可能是网络问题导致包无法下载。试着清理缓存然后重新尝试: ```bash pip cache purge pip install psycopg2 ``` 如果以上步骤都不能解决问题,你可能需要查看更详细的错误日志或搜索相关的社区论坛,例如 Stack Overflow,来获取特定问题的解决方案。
评论 3
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值