MAC 下安装ELK(四)

具体信息可看官方网站，在此仅记录关键信息以及所踩坑

准备工作

• 安装brew
• 执行brew tap elastic/tap，elastic全家桶仓库

Elasticsearch

• 安装

brew install elastic/tap/elasticsearch-full

• 启动

brew services start elastic/tap/elasticsearch-full

• 关闭

brew services stop elastic/tap/elasticsearch-full

• 配置

#注意配置后边留有一个空格
#node.name: node-1
#cluster.initial_master_nodes: ["node-1"]
xpack.ml.enabled: false
network.host: 0.0.0.0 //外网可访问
http.port: 9200
#memory
bootstrap.memory_lock: false
bootstrap.system_call_filter: false

#增加elasticsearch-head 跨域处理******
http.cors.enabled: true
http.cors.allow-origin: "*"

• 查看

默认监听9200端口，浏览器访问 http://localhost:9200

• 核心文件路径

Type	Description	Default Location	Setting
home	Elasticsearch home directory or $ES_HOME	/usr/local/var/homebrew/linked/elasticsearch-full
bin	Binary scripts including elasticsearch to start a node and elasticsearch-plugin to install plugins	/usr/local/var/homebrew/linked/elasticsearch-full/bin
conf	Configuration files including elasticsearch.yml	/usr/local/etc/elasticsearch	ES_PATH_CONF
data	The location of the data files of each index / shard allocated on the node. Can hold multiple locations.	/usr/local/var/lib/elasticsearch	path.data
logs	Log files location.	/usr/local/var/log/elasticsearch	path.logs
plugins	Plugin files location. Each plugin will be contained in a subdirectory.	/usr/local/var/homebrew/linked/elasticsearch/plugins

Kibana

• 安装

brew install elastic/tap/kibana-full

• 启动

brew services start elastic/tap/kibana-full

• 关闭

brew services stop elastic/tap/kibana-full

• 设置 /usr/local/etc/kibana/kibana.yml配置路径根据实际情况查找
  具体配置信息可查看官方配置说明

# 设置中文
i18n.locale: "zh-CN"
# 设置elasticsearch URL
elasticsearch.url: ["http://localhost:9200"]

• 查看

默认监听5601，浏览器访问http://localhost:5601
访问

• 核心文件路径

Type	Description	Default Location	Setting
home	Kibana home directory or $KIBANA_HOME	/usr/local/var/homebrew/linked/kibana-full
bin	Binary scripts including kibana to start a node and kibana-plugin to install plugins	/usr/local/var/homebrew/linked/kibana-full/bin
conf	Configuration files including kibana.yml	/usr/local/etc/kibana
data	The location of the data files of each index / shard allocated on the node. Can hold multiple locations.	/usr/local/var/lib/kibana	path.data
logs	Log files location.	/usr/local/var/log/kibana	path.logs
plugins	Plugin files location. Each plugin will be contained in a subdirectory.	/usr/local/var/homebrew/linked/kibana-full/plugins

Filebeat

Filebeat客户机是一种轻量级的、资源友好的工具，它从服务器上的文件收集日志，并将这些日志转发给Logstash实例进行处理。Filebeat是为可靠性和低延迟而设计的。Filebeat在主机上占用的资源很少，而Beats input插件最小化了Logstash实例上的资源需求。

• 安装

brew install elastic/tap/filebeat-full

• 启动

brew services start elastic/tap/filebeat-full

• 关闭

brew services stop elastic/tap/filebeat-full

• 修改配置信息，路径/usr/local/etc/filebeat

filebeat.inputs:
- type: log
  paths:
    - /xxxxx/logstash-tutorial.log
output.logstash:
  hosts: ["localhost:5044"]

• 注意

1. 如果调试记得删除一下registry文件，每次执行都会记录最终位置，删除方便重头开始。
2. 文件位置注意查看启动信息，我的位置是在/usr/local/var/lib/filebeat

Logstash

• 安装

brew install elastic/tap/logstash-full

• 启动

# brew 自带启动
brew services start elastic/tap/logstash-full
# 验证配置
logstash -f log-file.conf --config.test_and_exit
# 配置更改时 自动重启
logstash -f log-file.conf  --config.reload.automatic

• 关闭

brew services stop elastic/tap/logstash-full

• 普通测试

logstash -e 'input { stdin { } } output { stdout {} }'
# 这是一个启动命令，当出现 Pipeline main started， 输入如下信息
hello world!

# 返回如下信息

{
    "@timestamp" => 2020-04-28T06:33:24.081Z,
       "message" => "hello world!",
          "host" => "localhost",
      "@version" => "1"
}

• 测试写入elasticsearch

logstash -e 'input { stdin { } } output { elasticsearch { hosts => ["localhost:9200"] } stdout { codec => rubydebug }}'

• 修改配置信息，将日志写入logstash中, 下载日志文件

input {
    beats {
        port => "5044"
    }
}
 filter {
    grok {
        match => { "message" => "%{COMBINEDAPACHELOG}"}
    }
    geoip {
        source => "clientip"
    }
}
output {
    elasticsearch {
        hosts => [ "localhost:9200" ]
	index => "test-filebeat-%{+YYYY.MM.dd}"
    }
}

总体注意事项

如果出现JAVA或者其他依赖版本问题仔细查看提示，根据提示处理

在Kibana中最终呈现如下