ELK
如锋
金融IT
展开
-
ELK - Using “elasticsearch.username: elastic“ is deprecated
Kibana elasticsearch username password 配置替换为serviceAccountToken原创 2022-09-18 13:16:57 · 960 阅读 · 0 评论 -
ELK - Active Directory LDAP User Authentication
Elasticsearch Active Directory踩坑记录。原创 2022-06-12 14:30:21 · 1219 阅读 · 0 评论 -
ELK - Set up basic security for the Elastic Stack plus secured HTTPS traffic
Test version: 7.151. Generate HTTP cert with the same CA for transport SSLcd /usr/share/elasticsearch./bin/elasticsearch-certutil httpGenerate a CSR? [y/N]nUse an existing CA? [y/N]yCA Path: /usr/share/elasticsearch/elastic-stack-ca.p12For how long原创 2022-03-14 21:11:26 · 1897 阅读 · 0 评论 -
ELK - Define Mutilple Pipelines For Logstash
Logstash有一个对初学者来说很大的坑,就是安装之后直接运行,它将默认只使用一个pipeline,把conf.d下面的配置文件一次全加载了。如果不同配置文件的输入输出有冲突,比如缺少必要的条件判断,就会发生意想不到的结果。使用多管道,势在必行。Default Pipelinepipeline.yml默认的pipeline配置是这样子的。# This file is where you define your pipelines. You can define multiple.# For原创 2022-02-19 12:52:04 · 903 阅读 · 0 评论 -
ELK - Set up basic security for the Elastic Stack
1. Genrate CA: elastic-stack-ca.p12cd /usr/share/elasticsearch./bin/elasticsearch-certutil caInput password2. Generate Cert: elastic-certificates.p12./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12Input passwordComplete the following step原创 2021-11-21 19:56:46 · 877 阅读 · 0 评论 -
ELK - Elasticsearch Validation Failed: this cluster currently has [1000]/[1000] maximum shards open
Elasticsearch突然就写不进去了,看日志,shards有1000的限制。Dev Tool{“error” : {“root_cause” : [{“type” : “runtime_exception”,“reason” : “Failed to persist transform configuration”}],“type” : “runtime_exception”,“reason” : “Failed to persist transform configuratio原创 2021-11-02 18:35:08 · 2949 阅读 · 0 评论 -
ELK - Filebeat重启之后不读Log了?Registry file is too large
Filebeat重启之后突然不读Log了,看起来又是正常运行。问一下有没有什么Bug,说Registry file is too large。后来我看了一下,只有几十M,不大呀?回复说,几十M已经太大了。果然,对大的理解不太一样。看一下文档,大概明白大的意思了,因为这其实是状态文件,Filebeat每时每刻都要读写。Filebeat keeps the state of each file and persists the state to disk in the registry file.原创 2021-03-20 10:36:34 · 968 阅读 · 2 评论 -
ELK - Elasticsearch JVM OutOfMemory, Kibana: [circuit_breaking_exception] [parent] Data too large
Kibana打开就报错,其实也算是Kibana的bug,看到已经有人起了Issue,就算Elasticsearch出现Out of memory,但也不至于整个网页都死了嘛。测试版本:7.9.0{“statusCode”:500,“error”:“Internal Server Error”,“message”:"[parent] Data too large, data for [<http_request>] would be [1044896078/996.4mb], which i原创 2021-01-30 10:34:31 · 714 阅读 · 0 评论 -
ELK - Logstash-keystore保存Elasticsearch或其他密码
方法好简单,文档好齐全,但还是有坑。测试版本:7.9.0set +o history# Have to setup keystore password to encrypt keystore, before running logstash-keystoreexport LOGSTASH_KEYSTORE_PASS=changeme# Have to save the password for service to kick start (installed by RPM), otherwis原创 2020-12-27 10:58:58 · 415 阅读 · 0 评论 -
ELK - Logstash配置调试技巧:Filter Grok日志模式Pattern匹配之类的确实很麻烦
之前我也很不耐烦,不过掌握了以下一些小技巧之后,感觉好多了。Logstash Config Test and Exit/usr/share/logstash/bin/logstash --config.test_and_exit --path.settings /etc/logstash -f test.confLogstash Config Automatic Reload/usr/share/logstash/bin/logstash --path.settings /etc/logstash原创 2020-11-07 18:34:07 · 1393 阅读 · 0 评论 -
ELK - Elasticsearch权威指南里的聚合分析报错: Text fields are not optimised for operations
Elasticsearch权威指南里的聚合分析报错: Text fields are not optimised for operations that require per-document field data like aggregations and sorting。Elasticsearch权威指南是指官网的《Elasticsearch:权威指南》。聚合分析GET employee/_search{ "aggs": { "all_ages": { "terms原创 2020-10-05 15:44:52 · 7927 阅读 · 2 评论 -
ELK - Elasticsearch权威指南里的简单搜索例子
Elasticsearch权威指南是指官网的《Elasticsearch:权威指南》。例子跟官网不一样,是因为类型(Type)这种用法现在已经被抛弃了,所以缺省类型都是_doc。建立索引PUT /employee/_doc/1{ "first_name" : "John", "last_name" : "Smith", "age" : 25, "about" : "I love to go rock climbing", "inter原创 2020-09-20 10:27:58 · 285 阅读 · 0 评论 -
ELK - Watcher发送告警邮件和调用接口
相比使用ElastAlert发送告警邮件,ELK提供的Wathcer要简单得多,也可以在发生警报的时候调用Web Service接口。Configure SMTPhttps://www.elastic.co/guide/en/elasticsearch/reference/current/actions-email.html以上文档提供了多种Email系统的配置方法(elasticsearch.yml),包括Gmail, Outlook, Microsoft Exchange, Amazon SES。原创 2020-05-23 11:27:26 · 1981 阅读 · 2 评论 -
ELK - 使用ElastAlert发送邮件
简单记录下之前的研究。ElastAlert开源、免费,测试结果可以成功发送邮件。https://buildmedia.readthedocs.org/media/pdf/elastalert/latest/elastalert.pdfInstall Python之前在ElastAlert最大的坑里说过,要用Python3!Install Pip如果没有pip,则需要安装。sudo e...原创 2020-01-11 10:26:21 · 1202 阅读 · 0 评论 -
ELK - Elasticsearch启动不了: Could not create the Java Virtual Machine
提供一个思路。Elasticsearch启动不了Nov 04 15:45:10 appserver01 elasticsearch[25283]: Invalid -Xlog option '-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,file...原创 2019-12-29 18:23:20 · 6551 阅读 · 3 评论 -
ELK - X-Pack设置用户密码
之前记录过怎样使用Nginx代理为Kibana添加登录验证功能,但其实Elastic本身也提供了基本的访问控制功能。虽然相关功能在X-Pack里,但还是可以免费使用的。参考官网subscriptions,可以看到在Basic License下的Security项目中包含了Role-based access control。Enable X-Pack SecurityElasticsearc...原创 2019-11-16 09:25:52 · 9341 阅读 · 0 评论 -
快速搭建ELK7.3
A quick start guild of ELK 7.3.2, including package installation and simple test.好吧,其实是发现之前的快速搭建ELK7.2有些坑,决定还是再来一次。安装版相比直接解压版还是有些好处的,最起码安装完就是Service,省却打命令行的许多麻烦。Download and installhttps://artifa...原创 2019-11-02 15:22:18 · 701 阅读 · 0 评论 -
Nginx - Basic Authentication为Kibana增加登录验证功能
文章目录Install NginxStartup NginxInstall httpd-toolsReload Nginx ConfigureChange Kibana ConfigureVerificationReferenceInstall NginxDownload package from https://nginx.org/download/nginx-1.16.1.tar.gzt...原创 2019-10-12 22:48:01 · 640 阅读 · 0 评论 -
快速搭建ELK(7.2.0)
文章目录Elastic StackELK架构ELK安装ElasticsearchCan not run elasticsearch as root远程访问ElasticsearchKibana远程访问KibanaKibana Logging配置Kibana StartupFilebeatLogstash验证真正用ELK,才发现ELK的文档那么多,眼花缭乱。也难怪,每一个单独起来也都是挺好的小工...原创 2019-07-13 11:12:03 · 1630 阅读 · 0 评论 -
ELK - ElastAlert最大的坑
我可以讲粗口吗!官网说Python2.7!实际要用Python3.6!https://elastalert.readthedocs.io/en/latest/running_elastalert.htmlhttps://github.com/Yelp/elastalert原创 2019-09-15 09:38:16 · 894 阅读 · 2 评论 -
ELK - Hearthbeat实现服务监控
Hearthbeat,心跳,顾名思义,Hearthbeat可以用来定时探测服务是否正常运行。Hearthbeat支持ICMP、TCP 和 HTTP,也支持TLS、身份验证和代理。Hearthbeat能够与Logstash、Elasticsearch和Kibana无缝协作。安装Hearthbeat,添加需要监控的服务,配置好Elasticsearch和Kibana,即可将结果输出到Elastic...原创 2019-08-01 13:13:57 · 1523 阅读 · 2 评论