elk安装过程

1.安装elasticsearch
下载地址:

https://www.elastic.co/downloads/elasticsearch

a) 解压安装包elasticsearch-6.2.0.tar.gz
    /home/alex/servers/elasticsearch-6.2.0
b) 编辑配置文件
   cd config
   vi elasticsearch.yml
   内容如下:
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /tmp/elasticsearch/data
#
# Path to log files:
#
path.logs: /tmp/elasticsearch/logs
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 192.168.1.90
#
# Set a custom port for HTTP:
#
http.port: 9200

c) 启动Elasticsearch
./bin/elasticsearch

如果没有错误发生，可以进入到下一步的验证环节

d) 验证

可以使用cURL命令:

curl 'http://192.168.1.90:9200/?pretty'

或者

直接用浏览器打开

http://192.168.1.90:9200

2.安装logstash
a) 解压安装包logstash-6.2.0.tar.gz
    完整路径如下:
    /usr/local/logstash-6.2.0
b) 编辑配置文件
   cd config
   新建配置文件log4j_es.conf
   vi log4j_es.conf
   内容如下:
input {
    syslog {
        port => 12345
    }
}

#此处filter有问题，需要更改

filter {
        grok {
            match => {
                "message" => "%{DATA}\[%{DATA}}\] %{DATA:log_data}"
            }
        }
    mutate {
        add_field => ["log_data","%{log_data}"]
    }
}

output {
    elasticsearch{
        hosts => ["192.168.1.90:9200"]
        index => "log4j-%{+YYYY.MM.dd}"
        document_type => "log4j_type"
    }
    stdout {codec=>rubydebug}
}

c) 启动Logstash
./bin/logstash -f config/log4j-es.conf

3.安装kibana
a) 解压安装包kibana-6.2.0.tar.gz
完整路径如下:
/usr/local/kibana-6.2.0

b) 编辑配置文件
cd config
vi kibana.yml
内容如下:
# Kibana is served by a back end server. This setting specifies the port to use.
server.port: 5601

server.host: "192.168.1.90"

# The URL of the Elasticsearch instance to use for all your queries.
elasticsearch.url: "http://192.168.1.90:9200"

# Kibana uses an index in Elasticsearch to store saved searches, visualizations and
# dashboards. Kibana creates a new index if the index doesn't already exist.
kibana.index: ".kibana"

c) 启动Kibana
./bin/kibana

d) 验证
打开网址:
http://192.168.1.90:5601/

4. Log4j记录日志到Logstash

4-1) 新建maven项目

pom中的关键dependency配置如下：

<dependency>
    <groupId>log4j</groupId>
    <artifactId>log4j</artifactId>
    <version>1.2.17</version>
</dependency>

4-2) log4j.properties(放在resources文件夹下)
# prd

log4j.rootLogger=INFO,CONSOLE,infoLog,errorLog

log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern=%d{HH:mm:ss.SSS} [%5p] %t %c:%L - %m%n

# error log
log4j.appender.errorLog=org.apache.log4j.DailyRollingFileAppender
log4j.appender.errorLog.File=/temp/error-logs/study.log
log4j.appender.errorLog.Threshold=ERROR
log4j.appender.errorLog.DatePattern='.'yyyy-MM-dd-HH
log4j.appender.errorLog.layout=org.apache.log4j.PatternLayout
log4j.appender.errorLog.layout.ConversionPattern=%d{HH:mm:ss.SSS} [%5p] %t %c:%L - %m%n

# monitor log
log4j.category.monitor=INFO,monitor,monitorFile

log4j.additivity.monitor=false
log4j.appender.monitor=org.apache.log4j.net.SyslogAppender
log4j.appender.monitor.syslogHost=192.168.1.90:12345
log4j.appender.monitor.layout=org.apache.log4j.PatternLayout
log4j.appender.monitor.layout.ConversionPattern=[%d{yyyyMMdd HH:mm:ss.SSS}] %m

4-3) Java代码：

public class TestLog {
    public static Logger logger = Logger.getLogger(TestLog.class);
    public static Logger monitor = Logger.getLogger("monitor");

    public static void main(String[] args) {
        logger.debug("This is a debug message!");
        logger.info("This is info message!");
        logger.warn("This is a warn message!");
        logger.error("This is error message!");

        monitor.debug("This is a debug message!");
        monitor.info("This is info message!");
        monitor.warn("This is a warn message!");
        monitor.error("This is error message!");
    }
}