elasticsearch内存管理

今天elasticsearch突然崩掉了,捉急,看看log是这样的:

$ cat elasticsearch-5.2/logs/zhujian_club.log
[2017-04-05T13:23:43,919][INFO ][o.e.n.Node               ] [] initializing ...
[2017-04-05T13:23:44,337][INFO ][o.e.e.NodeEnvironment    ] [d064jZw] using [1] data paths, mounts [[/ (/dev/xvda1)]], net usable_space [23.5gb], net total_space [39.2gb], spins? [no], types [ext4]
[2017-04-05T13:23:44,338][INFO ][o.e.e.NodeEnvironment    ] [d064jZw] heap size [1.9gb], compressed ordinary object pointers [true]
[2017-04-05T13:23:44,492][INFO ][o.e.n.Node               ] node name [d064jZw] derived from node ID [d064jZwnTYis8MVNX3ArIw]; set [node.name] to override
[2017-04-05T13:23:44,508][INFO ][o.e.n.Node               ] version[5.2.0], pid[27864], build[24e05b9/2017-01-24T19:52:35.800Z], OS[Linux/3.13.0-65-generic/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_121/25.121-b13]
[2017-04-05T13:23:52,136][INFO ][o.e.p.PluginsService     ] [d064jZw] loaded module [aggs-matrix-stats]
[2017-04-05T13:23:52,136][INFO ][o.e.p.PluginsService     ] [d064jZw] loaded module [ingest-common]
[2017-04-05T13:23:52,137][INFO ][o.e.p.PluginsService     ] [d064jZw] loaded module [lang-expression]
[2017-04-05T13:23:52,137][INFO ][o.e.p.PluginsService     ] [d064jZw] loaded module [lang-groovy]
[2017-04-05T13:23:52,137][INFO ][o.e.p.PluginsService     ] [d064jZw] loaded module [lang-mustache]
[2017-04-05T13:23:52,137][INFO ][o.e.p.PluginsService     ] [d064jZw] loaded module [lang-painless]
[2017-04-05T13:23:52,137][INFO ][o.e.p.PluginsService     ] [d064jZw] loaded module [percolator]
[2017-04-05T13:23:52,137][INFO ][o.e.p.PluginsService     ] [d064jZw] loaded module [reindex]
[2017-04-05T13:23:52,137][INFO ][o.e.p.PluginsService     ] [d064jZw] loaded module [transport-netty3]
[2017-04-05T13:23:52,137][INFO ][o.e.p.PluginsService     ] [d064jZw] loaded module [transport-netty4]
[2017-04-05T13:23:52,142][INFO ][o.e.p.PluginsService     ] [d064jZw] loaded plugin [analysis-ik]
[2017-04-05T13:24:04,175][INFO ][o.e.n.Node               ] initialized
[2017-04-05T13:24:04,175][INFO ][o.e.n.Node               ] [d064jZw] starting ...
[2017-04-05T13:24:04,935][INFO ][o.e.t.TransportService   ] [d064jZw] publish_address {139.129.19.240:9300}, bound_addresses {0.0.0.0:9300}
[2017-04-05T13:24:04,956][INFO ][o.e.b.BootstrapChecks    ] [d064jZw] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2017-04-05T13:24:08,760][WARN ][o.e.m.j.JvmGcMonitorService] [d064jZw] [gc][young][2][8] duration [2.5s], collections [1]/[3.5s], total [2.5s]/[8.5s], memory [60.7mb]->[60.1mb]/[1.9gb], all_pools {[young] [32.9mb]->[16.5mb]/[66.5mb]}{[survivor] [8.3mb]->[5.5mb]/[8.3mb]}{[old] [19.8mb]->[38.1mb]/[1.9gb]}
[2017-04-05T13:24:08,765][WARN ][o.e.m.j.JvmGcMonitorService] [d064jZw] [gc][2] overhead, spent [2.5s] collecting in the last [3.5s]
[2017-04-05T13:24:08,858][INFO ][o.e.c.s.ClusterService   ] [d064jZw] new_master {d064jZw}{d064jZwnTYis8MVNX3ArIw}{oUNqyCgfSxSPLyAbFwpXMA}{139.129.19.240}{139.129.19.240:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
[2017-04-05T13:24:08,967][INFO ][o.e.h.HttpServer         ] [d064jZw] publish_address {139.129.19.240:9200}, bound_addresses {0.0.0.0:9200}
[2017-04-05T13:24:09,282][INFO ][o.e.n.Node               ] [d064jZw] started
[2017-04-05T13:24:09,560][INFO ][o.w.a.d.Monitor          ] try load config from /home/master_user/elasticsearch-5.2/config/analysis-ik/IKAnalyzer.cfg.xml
[2017-04-05T13:24:09,568][INFO ][o.w.a.d.Monitor          ] try load config from /home/master_user/elasticsearch-5.2/plugins/ik/config/IKAnalyzer.cfg.xml
[2017-04-05T13:24:14,227][WARN ][o.e.m.j.JvmGcMonitorService] [d064jZw] [gc][young][5][9] duration [3.1s], collections [1]/[3.4s], total [3.1s]/[11.6s], memory [79mb]->[76mb]/[1.9gb], all_pools {[young] [35.2mb]->[764kb]/[66.5mb]}{[survivor] [5.5mb]->[8.3mb]/[8.3mb]}{[old] [38.1mb]->[67.1mb]/[1.9gb]}
[2017-04-05T13:24:14,227][WARN ][o.e.m.j.JvmGcMonitorService] [d064jZw] [gc][5] overhead, spent [3.1s] collecting in the last [3.4s]
[2017-04-05T13:24:14,361][INFO ][o.w.a.d.Monitor          ] [Dict Loading] custom/mydict.dic
[2017-04-05T13:24:14,392][INFO ][o.w.a.d.Monitor          ] [Dict Loading] custom/single_word_low_freq.dic
[2017-04-05T13:24:14,452][INFO ][o.w.a.d.Monitor          ] [Dict Loading] custom/ext_stopword.dic
[2017-04-05T13:24:15,759][INFO ][o.e.g.GatewayService     ] [d064jZw] recovered [2] indices into cluster_state
[2017-04-05T13:24:17,376][INFO ][o.e.c.r.a.AllocationService] [d064jZw] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[index][0]] ...]).
[2017-04-05T13:24:33,846][WARN ][o.e.m.j.JvmGcMonitorService] [d064jZw] [gc][young][22][10] duration [2.7s], collections [1]/[3.6s], total [2.7s]/[14.4s], memory [139.4mb]->[95.2mb]/[1.9gb], all_pools {[young] [63.9mb]->[2.2mb]/[66.5mb]}{[survivor] [8.3mb]->[8.3mb]/[8.3mb]}{[old] [67.1mb]->[84.7mb]/[1.9gb]}
[2017-04-05T13:24:33,846][WARN ][o.e.m.j.JvmGcMonitorService] [d064jZw] [gc][22] overhead, spent [2.7s] collecting in the last [3.6s]

并没有什么error,但是最后是这样的:

[2017-04-05T13:24:33,846][WARN ][o.e.m.j.JvmGcMonitorService] [d064jZw] [gc][young][22][10] duration [2.7s], collections [1]/[3.6s], total [2.7s]/[14.4s], memory [139.4mb]->[95.2mb]/[1.9gb], all_pools {[young] [63.9mb]->[2.2mb]/[66.5mb]}{[survivor] [8.3mb]->[8.3mb]/[8.3mb]}{[old] [67.1mb]->[84.7mb]/[1.9gb]}
[2017-04-05T13:24:33,846][WARN ][o.e.m.j.JvmGcMonitorService] [d064jZw] [gc][22] overhead, spent [2.7s] collecting in the last [3.6s]

JVM出问题了。
检索了一下,原来elasticsearch的JVM默认size为2g(使用版本5.2),阿里云总共才1g。。
去config/jvm.options修改内存:

Xmx=300m
Xms=300m
  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
对于 Elasticsearch 的内存分析,我们可以从以下几个方面进行考虑: 1. JVM 堆内存分析:Elasticsearch 是基于 Java 开发的,它使用 Java 虚拟机(JVM)管理内存。通过监控和分析 JVM 的堆内存使用情况,可以了解 Elasticsearch 在内存中存储的数据量、缓存的使用情况等。可以使用工具如 VisualVM、jstat、jmap 等来获取堆内存信息并进行分析。 2. 分片和副本的内存使用:Elasticsearch 将数据分成多个分片,并在不同的节点上存储副本。每个分片和副本都会占用一定的内存空间。通过监控每个节点上的分片和副本数量,可以了解 Elasticsearch 集群中的内存使用情况。 3. 缓存的使用情况:Elasticsearch 使用了多种缓存机制来提高查询性能,如字段数据缓存、过滤器缓存、请求缓存等。通过监控和分析缓存的命中率、缓存大小等指标,可以评估缓存的使用效果和是否需要调整缓存配置。 4. 索引和搜索的内存开销:Elasticsearch 在内存中维护了索引结构和搜索相关的数据结构,如倒排索引、过滤器、聚合缓存等。通过监控和分析索引和搜索的内存开销,可以了解 Elasticsearch 在处理索引和搜索时的内存使用情况。 5. 内存溢出和泄漏问题:如果 Elasticsearch 集群中出现内存溢出或内存泄漏问题,需要进行详细的分析来确定具体原因。可以通过查看日志、分析堆转储快照(heap dump)等方式来进行故障排查。 需要注意的是,进行内存分析时应该结合实际业务场景和负载情况来进行评估和优化。每个 Elasticsearch 集群的配置和使用方式各有不同,因此具体的内存优化策略需根据实际情况进行调整。

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值