ELK-6.6.0的环境搭建

ELK的搭建

下载elasticsearch安装包

elasticsearch-6.6.1.tar.gz

解压后，进入config目录，修改配置文件 elasticsearch.yml

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: why-es6
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node01
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /path/to/data
#
# Path to log files:
#
path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 192.168.200.82
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["node01"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
#discovery.zen.minimum_master_nodes: 
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true

进入bin目录，使用./elasticsearch 启动，出现如下图所示报错：

不能使用root用户启动。这里使用新创建的why用户启动。
添加用户组
groupadd hongyewell
添加用户
useradd why
修改用户所属组
usermod -g hongyewell why
为新用户why设置密码
passwd why

使用why账户登录服务器，并启动es，出现如下报错

错误原因：使用非 root用户启动ES，而该用户的文件权限不足而被拒绝执行。

解决方法： chown -R 用户名:用户名  文件（目录）名

在root账户下，执行命令： chown -R why:why /soft/elasticsearch-6.6.1

又出现了其他日志目录的权限问题。

在roo账户下，执行命令：chown -R why.hongyewell *，其中* 表示 任何文件。

。。。还是报同样的错。

最后修改了elasticsearch.yml中的path.data:/soft/elasticsearch-6.6.1/path/to/data 及 path.log:/soft/elasticsearch-6.6.1/path/to/logs。

又出现上图所示问题：

ERROR: [2] bootstrap checks failed
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

[ 1 ] 每个进程最大同时打开文件数太小，可通过下面2个命令查看当前数量

ulimit -Hn
ulimit -Sn

修改/etc/security/limits.conf文件，使用root权限增加如下配置，用户退出后重新登录生效。

* hard nofile 65536
* soft nofile 65536

[ 2 ] 解决第二个问题需要用有 root 权限的用户执行如下的命令

sysctl -w vm.max_map_count=262144

再次启动后，出现如下报错：

在/etc/目录下的hosts文件添加如下内容，并将修改es配置文件，node.name: weihy2，discovery.zen.ping.unicast.hosts: ["weihy2"]

192.168.200.82 weihy2

再次启动，出现如下信息initialized starting … started，表示启动成功

通过浏览器访问http://192.168.200.82:9200/，可以看到类似如下内容

后台启动es：
bin/elasticsearch -d

使用jps查看进程
[why@weihy2 bin]$ jps
10614 Jps
10591 Elasticsearch

执行命令：curl http://192.168.200.82:9200

 

---

Logstash的安装与启动：

下载logstash安装包

logstash-6.6.1.tar.gz

解压后，进入bin目录，输入命令：./logstash -e 'input { stdin { } } output { stdout {} }'

即测试Logstash的安装，运行一个最基本的logstash管道。

The -e flag enables you to specify a configuration directly from the command line. Specifying configurations at the command line lets you quickly test configurations without having to edit a file between iterations. The pipeline in the example takes input from the standard input, stdin, and moves that input to the standard output, stdout, in a structured format.

-e的标志使你可以明确使用命令行的方式启动。

Logstash adds timestamp and IP address information to the message. Exit Logstash by issuing a CTRL-D command in the shell where Logstash is running.

Congratulations! You’ve created and run a basic Logstash pipeline. Next, you learn how to create a more realistic pipeline.

./logstash -f file.conf & 后台启动

恭喜，你已经创建了最基本的logstash管道。接下来，你可以学习如果创建更多的实时管道。