ELKR分别指elasticsearch-7.2.0、logstash-7.2.0、kibana-7.2.0、rsyslog,用的当前官网最新版本7.2.0,这次测试用于读取操作系统登录和操作日志。
系统版本
CentOS Linux release 7.5.1804 (Core)
1、安装包下载
官方地址:https://www.elastic.co/cn/downloads/
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.2.0-linux-x86_64.tar.gz
wget https://artifacts.elastic.co/downloads/logstash/logstash-7.2.0.tar.gz
wget https://artifacts.elastic.co/downloads/kibana/kibana-7.2.0-linux-x86_64.tar.gz
2、创建elk用户
elasticsearch和kibana 必须在非root用户启动
groupadd elk
useradd elk -g elk
3、 解压
tar -xzf elasticsearch-7.2.0-linux-x86_64.tar.gz -C /home/elk/
tar -xzf logstash-7.2.0.tar.gz -C /home/elk/
tar -xzf kibana-7.2.0-linux-x86_64.tar.gz -C /home/elk/
4、赋予elk权限
chown -R elk:elk elasticsearch-7.2.0
chown -R elk:elk kibana-7.2.0-linux-x86_64
5、elasticsearch部署
5.1、创建数据和日志文件夹,并修改权限
mkdir -p /home/elk/es-data/logs
chown -R elk:elk /home/elk/
5.2、修改配置文件
vim elasticsearch-7.2.0/config/elasticsearch.yml
path.data: /home/elk/es-data
path.logs: /home/elk/es-data/logs
#其他项默认,可以按需求修改
5.4、启动脚本
vim /home/elk/es-start.sh
#!/bin/sh
cd /home/elk/elasticsearch-7.2.0
nohup bin/elasticsearch &
tail -f nohup.out
5.5、 启动
su - elk
sh es-start.sh
5.6、测试
curl 127.0.0.1:9200
{
"name" : "study01",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "7XETJBiFRJKALJPvzhGklQ",
"version" : {
"number" : "7.2.0",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "508c38a",
"build_date" : "2019-06-20T15:54:18.811730Z",
"build_snapshot" : false,
"lucene_version" : "8.0.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
6、logstash部署
***注:root用户操作 ***
6.1、增加配置文件
vim /home/elk/logstash-7.2.0/conf/logstash-rsyslog.conf
input {
syslog {
port => 514
type => "rsyslog"
}
}
filter {
if [type] == 'rsyslog' {
urldecode { # 编码转换#
all_fields=>true
}
mutate {
split => ["message","||"] # 拆分日志#
add_field => {"HostName" => "%{[message][0]}"}
add_field => {"Facility" => "%{[message][1]}"}
add_field => {"Mes" => "%{[message][5]}"}
remove_field => ["message","facility_label","facility","severity_label","severity","priority","timestamp","program"]
}
if [Facility] == "local5" {
mutate {
split => ["Mes",","] # 拆分日志#
add_field => {"ClientIp" => "%{[Mes][0]}"}
add_field => {"LoginUserName" => "%{[Mes][1]}"}
add_field => {"SessionId" => "%{[Mes][2]}"}
remove_field => ["Mes"]
}
}
if [Facility] == "user" {
mutate {
split => ["Mes",","] # 拆分日志#
add_field => {"Euid" => "%{[Mes][0]}"}
add_field => {"WhoInfo" => "%{[Mes][1]}"}
add_field => {"ExecPath" => "%{[Mes][2]}"}
add_field => {"ExecCmd" => "%{[Mes][3]}"}
remove_field => ["Mes"]
}
}
}
}
output {
if [type] == 'rsyslog' and [Facility] == "local5" {
elasticsearch {
hosts => ["localhost:9200"]
index => ["logstash-login-%{+YYYY.MM.dd}"]
}
}else if [type] == 'rsyslog' and [Facility] == "user" {
elasticsearch {
hosts => ["localhost:9200"]
index => ["logstash-user-%{+YYYY.MM.dd}"]
}
}
}
6.2 、启动
cd /home/elk/logstash-7.2.0/
nohup bin/logstash -f /home/elk/logstash-7.2.0/conf/logstash-rsyslog.conf
7、rsyslog配置
***注:root用户操作 ***
7.1 、添加如下配置到 /etc/bashrc 文件末尾
up_client_ip=`(who am i|cut -d\( -f2|cut -d\) -f1)`
logger -p local5.info -- $up_client_ip,$(whoami),$$
export PROMPT_COMMAND='{ msg=$(history 1 | { read x y; echo $y; });logger -p user.notice "[euid=$(whoami)]",$(who am i),`pwd`",$msg"; }'
readonly PROMPT_COMMAND
7.2 、修改配置 /etc/rsyslog.conf
vim /etc/rsyslog.conf
新增文件末尾:
$template StdLOGFormat,"%fromhost%||%syslogfacility-text%||%syslogpriority-text%||%timereported:::date-mysql%||%timegenerated:::date-mysql%||%msg%||%iut%||%programname%||%syslogtag%"
*.* @@127.0.0.1:514;StdLOGFormat
7.3 、重启
service rsyslog restart
8、 kibana部署
8.1 、修改配置
vim /home/elk/kibana-7.2.0-linux-x86_64/config/kibana.yml
server.host: "0.0.0.0"
#其他项按需求自行修改
8.2、 启动脚本
vim /home/elk/kb-start.sh
#!/bin/sh
cd /home/elk/kibana-7.2.0-linux-x86_64
nohup bin/kibana &
tail -f nohup.out
8.3 、启动
su - elk
sh kb-start.sh
8.4、 浏览器访问
在浏览器里,输入yourip:5601 访问成功即代表启动成功
8.5、 Kibana界面
启动好了之后,在浏览器里访问前端页面
点击Discover
8.6、 操作系统追加数据
创建新的测试日志
logger "跟我一起学猫叫,一起喵喵喵"
echo '测试中文' >> test.txt
cat test.txt
刷新kibana,即可看到新的日志