1) 数据采集
地址:
https://www.elastic.co/downloads/logstash
下载:
wget https://artifacts.elastic.co/downloads/logstash/logstash-5.1.1.tar.gz
tar –zxvf logstash-5.1.1.tar.gz
数据采集端的安装步骤与服务器端的安装步骤相同只是在配置文件有所不同
在数据采集端创建 logstash_client.conf
数据采集端的conf文件配置为:
input {
file {
type =>"type_count" #日志所在的目录
path => ["/logs/stdout.log","/logs/stderr.log"]
exclude =>["*.gz", "access.log"] #排除一些不想加入的日志
}
}
output {
stdout {}
redis {
host => "192.168.71.145"
port => 6379
data_type =>"list" #数据类型也要保持一致
key =>"key_count" #key 要与服务器端的相同
}
}
配置完成后启动logstash
执行:
cd /
mkdir logs
vim stdout.log
vim stderr.log
cd /usr/local/logstash/logstash-5.1.1
./bin/logstash --verbose -fconfig/logstash_client.conf -l logs/ &
发送测试消息:
echo 'this is a testmessage!' >> /logs/stderr.log
在个控制台可见
2) Redis:
wget http://124.202.164.16/files/2125000009862F7A/download.redis.io/releases/redis-3.2.6.tar.gz
问题1.redis出现问题zmalloc.h:50:31:错误:jemalloc/jemalloc.h:没有那个文件或目录
解决方法:[root@localhost redis-3.2.6]# makeMALLOC=libc
tar -zvxf redis-3.2.6.tar.gz
make
make install
然后启动redis
redis-server redis.conf &
添加&符号表示后台启动
3) Logstash
地址:
https://www.elastic.co/downloads/logstash
下载:
wget https://artifacts.elastic.co/downloads/logstash/logstash-5.1.1.tar.gz
tar –zxvf logstash-5.1.1.tar.gz
将一下内容写入到logstash_server.conf中
input {
redis {
host =>"127.0.0.1" #本地的reds地址
port =>6379 #redis端口
type =>"redis-input" #输入类型
data_type=> "list" #使用redis的list存储数据
key =>"key_count"
}
}
output {
stdout {}
elasticsearch {
hosts => "127.0.0.1:9200" #elasticsearch地址
codec => "json"
}
}
执行:
cd /usr/local/logstash/logstash-5.1.1
./bin/logstash --verbose -f ./config/logstash_server.conf-l ./logs/ &
4) elasticSearch
地址:
https://www.elastic.co/downloads/elasticsearch
下载:
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.1.1.tar.gz
1.通过vim 修改config/elasticsearch.yml
cluster.name: es-k 设置es的名称
network.host:192.168.71.145 设置其ip地址
2.不允许root用户启动
访问授权:
chown zhxtn /usr/local/elasticsearch -R
3.启动
./bin/elasticsearch&
4.检测
nestat -anp |grep 9200
切换到root用户
1、[2016-11-19T03:22:22,188][WARN][o.e.b.BootstrapCheck ] [4Ut8v_1]max file descriptors [4096] for elasticsearch process is too low, increase toat least [65536]
更改允许打开的最大文件描述符,修改成功后重新启动生效。
vim /etc/security/limits.conf
* soft nofile 65536
* hard nofile 65536
2 、[2016-11-19T03:22:22,188][WARN][o.e.b.BootstrapCheck ] [4Ut8v_1]max virtual memory areas vm.max_map_count [65530] is too low, increase to atleast [262144]
更改sysctl.conf的配置
vim /etc/sysctl.d/99-sysctl.conf 或
vim /etc/sysctl.conf
增加:vm.max_map_count = 262144
sysctl -p //保存生效
然后,重新启动elasticsearch,即可启动成功。
3、ava HotSpot(TM) 64-Bit ServerVM warning: INFO: os::commit_memory(0x0000000085330000, 2060255232, 0) failed;error=’Cannot allocate memory’(errno=12)
内存过小,可以减少jvm的内存,建议设置成总内存大小的一半
vim /usr/local/elastic/config/jvm.options
-Xms512m
-Xmx512m
jvm.options的配置见:ElasticSearch5.0中Jvm的配置
执行:
cd /usr/local/elasticsearch/elasticsearch-5.1.1
su zhxtn
./bin/elasticsearch &
浏览:
192.168.71.145:9200
5) kibana
地址:
https://www.elastic.co/downloads/kibana
下载:
wget https://artifacts.elastic.co/downloads/kibana/kibana-5.1.1-linux-x86_64.tar.gz
执行:
cd /usr/local/kibana/kibana-5.1.1-linux-x86_64/
./bin/kibana &
浏览:
192.168.71.145:5601