ElasticSearch6.0
ElasticSearch6.0安装
#依赖jdk8 rpm -ivh elasticsearch-6.0.0.rpm vim /etc/elasticsearch/elasticsearch.yml #配置如下 node.name: es1 node.master: true node.data: true path.data: /home/es/data path.logs: /home/es/logs network.host: 192.168.55.215 http.port: 9200 bootstrap.memory_lock: false #centos6需要 bootstrap.system_call_filter: false #centos6需要 ##以下两项是head插件访问es需要配置 http.cors.enabled: true http.cors.allow-origin: "*" # 设置索引的分片数,默认为5 #index.number_of_shards: 5 # 设置索引的副本数,默认为1: #index.number_of_replicas: 1 mkdir -p /home/es/{data,logs} chown -R elasticsearch.elasticsearch /home/es vim /etc/security/limits.conf #添加或修改以下内容 * hard nproc 4096 * soft nproc 4096 * hard nofile 131072 * soft nofile 65536 /etc/elasticsearch/jvm.options #更改启动内存,测试环境512M就可以了 -Xms1g -Xmx1g /etc/sysconfig/elasticsearch #启动脚本的一些环境变量例如 JAVA_HOME=/opt/java /etc/init.d/elasticsearch start #启动es
ElasticSearch配置文件详解
https://blog.csdn.net/zxf_668899/article/details/54582849
ElasticSearch6.0-head插件安装(界面查看es索引)
#网址:http://www.cnblogs.com/Onlywjy/p/Elasticsearch.html #包D:\share\src\elk\elk6.0\elasticsearch-head插件 1.安装node tar -C /opt/ -zxvf node-v4.4.7-linux-x64.tar.gz vim /etc/profile.d/node.sh #配置node环境变量 export NODE_HOME=/opt/node-v4.4.7-linux-x64 export PATH=$PATH:$NODE_HOME/bin export NODE_PATH=$NODE_HOME/lib/node_modules source /etc/profile.d/node.sh 2.安装grunt unzip elasticsearch-head-master.zip cd elasticsearch-head-master npm install -g grunt-cli //执行后会生成node_modules文件夹 grunt -version #检查是否安装成功 修改head插件源码 修改服务器监听地址:Gruntfile.js #93行,默认端口号库9100 修改连接地址:_site/app.js #4354行,修改es连接地址,注意这个地址只能是外网地址,相对于访问head的机子 3.运行head 在elasticsearch-head-master目录下 npm install #(安装下载下来的包,如果出错再执行一遍) grunt server & #后台启动、netstat -lnp | grep 9100 4.访问http://xxx:9100
5.head使用(添加索引,往索引里写数据,修改索引数据,删除索引)
https://www.cnblogs.com/yanan7890/p/6640289.html
ElasticSearch6.0索引清理
curl -XGET 'http://192.168.55.219:9200/_cat/indices/?v' #查询索引 curl -XDELETE 'http://127.0.0.1:9200/logstash-2016-07-*' #api删除索引 脚本加api删除(推荐) vim /opt/sh/es-index-clear.sh #/bin/bash #指定日期(7天前) DATA=`date -d "1 week ago" +%Y.%m.%d` #当前日期 time=`date` #删除7天前的日志 curl -XDELETE http://127.0.0.1:9200/*-${DATA} if [ $? -eq 0 ];then echo $time"-->del $DATA log success.." >> /tmp/es-index-clear.log else echo $time"-->del $DATA log fail.." >> /tmp/es-index-clear.log fi 添加到任务计划 crontab -e 10 1 * * * sh /tmp/es-index-clear.sh > /dev/null 2>&1
elasticSearch常用命令
原网址:https://zhaoyanblog.com/archives/732.html curl 'localhost:9200/_cluster/health?pretty' 健康检查 curl 'localhost:9200/_cluster/state?pretty' 集群详细信息
curl -XPUT http://localhost:9200/user #添加索引
curl -H "Content-Type:application/json" -XPUT http://192.168.56.10:9200/user/ceshi/1/ -d '{"id":2,"name":"ceshi2","age":222}' #索引里写文档,user是索引,ceshi是索引类型,1是索引id
curl -XGET http://localhost:9200/user #查看索引内容
curl -XDELETE http://localhost:9200/user #删除索引,支持通配符
elasticsearch集群
集群配置文件 #node-master cluster.name: my_es node.name: es1 node.master: true node.data: false path.data: /home/es/data path.logs: /home/es/logs network.host: 0.0.0.0 http.port: 9200 http.cors.enabled: true #head连接需要(head连接到主) http.cors.allow-origin: "*" #head连接需要(head连接到主) discovery.zen.ping.unicast.hosts: ["192.168.56.10","192.168.56.11:9200","192.168.56.12:9200"] #node-data1 cluster.name: my_es node.name: es2 node.master: false node.data: true path.data: /home/es/data path.logs: /home/es/logs network.host: 0.0.0.0 http.port: 9200 discovery.zen.ping.unicast.hosts: ["192.168.56.10","192.168.56.11:9200","192.168.56.12:9200"] #node-data2 cluster.name: my_es node.name: es3 node.master: false node.data: true path.data: /home/es/data path.logs: /home/es/logs network.host: 0.0.0.0 http.port: 9200 discovery.zen.ping.unicast.hosts: ["192.168.56.10","192.168.56.11:9200","192.168.56.12:9200"] ##说明 页面登入head 带星的节点为master
安装kibana和使用
kibana安装
rpm -ivh kibana-6.0.0-x86_64.rpm vim /etc/kibana/kibana.yml server.port: 5601 server.host: 0.0.0.0 elasticsearch.url: "http://192.168.56.10:9200" #连接到主 logging.dest: /var/log/kibana/kibana.log /etc/init.d/kibana start #默认端口5601 kibana6.0官方文档 https://www.elastic.co/guide/en/kibana/6.0/index.html
kibana添加索引
http://192.168.56.10:5601
1.
2.
kibana对索引内容搜索
1.
2.
logstash安装和使用例子
logstash安装
ln -sv /opt/apps/java/bin/java /usr/bin/
rpm -ivh logstash-6.0.0.rpm
目录结构:
/usr/share/logstash/bin
/var/lib/logstash
/var/log/logstash
注意这几个目录的用户和组必须是logstash
如果用root用户启动logstash后 需要把上面几个目录的用户和组修改下(chown -R logstash.logstash xxxx)
#生成启动脚本
1、要跟startup.options文件的绝对路径,如果是rpm安装的在/etc/logstash/startup.options,如果是二进制包解压安装的则在解压目录下的config目录下面。
2、必须要跟启动类型,比如CentOS6是sysv,CentOS7是systemd。(备注:sysv在centos7里也能用)
3、执行脚本生成启动文件
/usr/share/logstash/bin/system-install /etc/logstash/startup.options sysv
4、修改/etc/init.d/logstash
75行 >> /var/log/logstash/logstash-plain.log 2>> /var/log/logstash/logstash-plain.log
#更改日志目录权限
chown -R logstash.logstash /var/log/logstash
#配置Logstash to elasticsearch文件
vim /etc/logstash/conf.d/log_to_es.conf
=========================================
input{
file{
path => "/var/log/messages"
start_position => "beginning"
type => "message"
}
}
output{
if [type] == "message" {
elasticsearch{
hosts => "192.168.56.10:9200"
#hosts => ["10.10.1.90:9200","10.10.1.60:9200"] #多node data 就里配置的是node data,不要配置node master
index => "message-%{+YYYY-MM}" #index => "message-%{+YYYY-MM-dd}" } } } ========================================
#更改/var/log/messages权限
chmod 644 /var/log/messages
#检查配置文件有没有错误 /usr/share/logstash/bin/logstash --path.settings /etc/logstash/ -f /etc/logstash/conf.d/log_to_es.conf --config.test_and_exit #启动 /etc/init.d/logstash start #检查启动是否成功 netstat -lnp | grep 9600 或查看日志 tail -f /var/log/logstash/logstash-plain.log