本次多节点环境部署示例是基于centos7.2进行,由于资源有限,将ES与Graylog和MongoDB部署在同一台server上。以下内容仅供参考,正式生产环境根据需要进行调整。
前置条件
准备三个节点,系统版本为centos7.2。
Ip | Hostname |
---|---|
192.168.2.121 | graylogNode1 |
192.168.2.122 | graylogNode2 |
192.168.2.123 | graylogNode3 |
可以直接使用vagrant创建三个节点,Vagrantfile如下:
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure(2) do |config|
(1..3).each do |i|
config.vm.define "graylogNode#{i}" do |s|
s.vm.box = "bento/centos-7.2"
s.vm.hostname = "graylogNode#{i}"
n = 120 + i
s.vm.provision :shell, inline: "sed 's/127\.0\.0\.1.*node.*/192\.168\.2\.#{n} node#{i}/' -i /etc/hosts"
s.vm.network "private_network", ip: "192.168.2.#{n}"
s.vm.provider "virtualbox" do |v|
v.cpus = 1
v.memory = 2048
v.name = "graylog-node#{i}"
end
end
end
end
启动节点:
$ vagrant up
在每个节点上安装好MongoDB、Elasticsearch、Graylog,具体安装步骤参考上一篇 Graylog安装使用。
集群配置
集群配置包括三个部分:配置MongoDB副本集、配置Elasticsearch集群、Graylog多节点配置。
配置MongoDB副本集
(1)指定副本集名称。
方法一:通过配置文件指定
更改每个节点上的MongoDB配置文件/etc/mongod.conf
,将bindIp
所在的行前加上#
进行注释,replication
部分添加replSetName: rs0
,更改后的配置文件如下:
# for documentation of all options, see:
# http://docs.mongodb.org/manual/reference/configuration-options/
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
# Where and how to store data.
storage:
dbPath: /var/lib/mongo
journal:
enabled: true
# engine:
# mmapv1:
# wiredTiger:
# how the process runs
processManagement:
fork: true # fork and run in background
pidFilePath: /var/run/mongodb/mongod.pid # location of pidfile
# network interfaces
net:
port: 27017
# bindIp: 127.0.0.1 # Listen to local interface only, comment to listen on all interfaces.
#security:
#operationProfiling:
#replication:
replication:
replSetName: rs0
#sharding:
## Enterprise-Only Options
#auditLog:
#snmp:
重启服务:
$ sudo systemctl restart mongod.service
方法二:使用mongo --replSet
命令行指定
在每个节点上执行命令:
$ mongod --replSet "rs0"
(2)在集群中的一个节点上执行mongo
命令行:
$ mongo
初始化副本集
使用本机hostname或IP加端口,如下
$ rs.initiate( {
_id : "rs0",
members: [ { _id : 0, host : "192.168.2.121:27017" } ]
})
查看配置
rs.conf()
配置如下:
{
"_id" : "rs0",
"version" : 1,
"protocolVersion" : NumberLong(1),
"members" : [
{
"_id" : 0,
"host" : "192.168.2.121:27017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
}
],
"settings" : {
"chainingAllowed" : true,
"heartbeatIntervalMillis" : 2000,
"heartbeatTimeoutSecs" : 10,
"electionTimeoutMillis" : 10000,
"getLastErrorModes" : {
},
"getLastErrorDefaults" : {
"w" : 1,
"wtimeout" : 0
},
"replicaSetId" : ObjectId("59ef0832a5da3378b1487f4e")
}
}
向副本集中添加成员
rs0:PRIMARY> rs.add("192.168.2.122:27017")
{ "ok" : 1 }
rs0:PRIMARY> rs.add("192.168.2.123:27017")
{ "ok" : 1 }
添加完成之后的配置:
rs0:PRIMARY> rs.config()
{
"_id" : "rs0",
"version" : 4,
"protocolVersion" : NumberLong(1),
"members" : [
{
"_id" : 0,
"host" : "192.168.2.121:27017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
},
{
"_id" : 1,
"host" : "192.168.2.122:27017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
},
{
"_id" : 2,
"host" : "192.168.2.123:27017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
}
],
"settings" : {
"chainingAllowed" : true,
"heartbeatIntervalMillis" : 2000,
"heartbeatTimeoutSecs" : 10,
"electionTimeoutMillis" : 10000,
"getLastErrorModes" : {
},
"getLastErrorDefaults" : {
"w" : 1,
"wtimeout" : 0
},
"replicaSetId" : ObjectId("59ef0832a5da3378b1487f4e")
}
}
查看状态
rs0:PRIMARY> rs.status()
创建graylog数据库,添加graylog用户
rs0:PRIMARY> use graylog
switched to db graylog
rs0:PRIMARY> db.createUser( {
... user: "graylog",
... pwd: "75PN76Db66En",
... roles: [ { role: "readWrite", db: "graylog" } ]
... });
rs0:PRIMARY> db.grantRolesToUser( "graylog" , [ { role: "dbAdmin", db: "graylog" } ])
rs0:PRIMARY> show users
rs0:PRIMARY> db.auth("graylog","75PN76Db66En")
配置Elasticsearch集群
(1)修改每个节点的配置文件
$ sudo vim /etc/elasticsearch/elasticsearch.yml
需要更改的部分如下:
#es集群名称,每个节点中cluster.name要保持一致(建议名称为graylog)
cluster.name: graylog
#节点名称
node.name: es-node-01
#当前节点IP
network.host: 192.168.2.121
#端口
http.port: 9200
#集群中的主机
discovery.zen.ping.unicast.hosts: ["192.168.2.121", "192.168.2.122", "192.168.2.123"]
#可发现的主节点
discovery.zen.minimum_master_nodes: 2
(2)重启服务
$ sudo systemctl restart elasticsearch.service
(3)查看集群状态
$ curl 'http://192.168.2.121:9200/_cluster/health?pretty=true'
{
"cluster_name" : "graylog",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 3,
"number_of_data_nodes" : 3,
"active_primary_shards" : 80,
"active_shards" : 80,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}
(4)查看集群中的节点
$ curl 'http://192.168.2.121:9200/_cat/nodes?v'
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.2.122 4 96 4 0.01 0.04 0.12 mdi - es-node-02
192.168.2.121 4 96 5 0.03 0.04 0.06 mdi * es-node-01
192.168.2.123 4 97 6 0.04 0.18 0.20 mdi - es-node-03
Graylog多节点配置
打开配置文件进行编辑:
$ sudo vim /etc/graylog/server/server.conf
(1)设置master节点
本次环境是将192.168.2.121作为graylog主节点,因此在192.168.2.121上修改配置文件/etc/graylog/server/server.conf
中is_master = true
,其他两个节点的配置文件中设置为is_master = false
。
(2)修改mongodb连接配置mongodb_uri
mongodb_uri = mongodb://graylog:75PN76Db66En@192.168.2.121:27017,192.168.2.122:27017,192.168.2.123:27017/graylog?replicaSet=rs0
(3)修改elasticsearch连接配置
elasticsearch_hosts
elasticsearch_hosts = http://192.168.2.121:9200,http://192.168.2.122:9200,http://192.168.2.123:9200
(4)开启web界面
web_enable
web_enable = true
(5)修改
web_listen_uri
#不同的节点不同的IP web_listen_uri = http://192.168.2.121:9000/
(6)修改
rest_listen_uri
#不同的节点不同的IP rest_listen_uri = http://192.168.2.121:9000/api/
重启服务
$ sudo systemctl restart graylog-server.service
创建负载均衡器,对graylog配置负载均衡
本次使用nginx进行负载均衡,安装步骤如下:
添加yum源:在/etc/yum.repos.d/
目录下新建文件nginx.repo
,添加如下内容:[nginx] name=nginx repo baseurl=http://nginx.org/packages/mainline/OS/OSRELEASE/$basearch/ gpgcheck=0 enabled=1
补充:
OS
换成你的系统,如:“centos”、“rhel”;OSRELEASE
换成系统版本,本次的系统环境是centos7,所以写成如下的内容。[nginx] name=nginx repo baseurl=http://nginx.org/packages/mainline/centos/7/$basearch/ gpgcheck=0 enabled=1
安装:
$ sudo yum -y install nginx
启动服务:
$ sudo service nginx start
。
配置Nginx:
更改Nginx的配置文件server { listen 80; listen [::]:80 default_server ipv6only=on; server_name graylog.example.org; location / { proxy_set_header Host $http_host; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Graylog-Server-URL http://$server_name/api; proxy_pass http://graylog-web-cluster; } } upstream graylog-web-cluster { server 192.168.2.121:9000 max_fails=3 fail_timeout=30s; server 192.168.2.122:9000 max_fails=3 fail_timeout=30s; server 192.168.2.123:9000 max_fails=3 fail_timeout=30s; }
重启服务:
sudo nginx -s reload
此时可以通过访问http://graylog.example.org/查看graylog,可看到已经搭建好的节点信息:
Q&A
在初始化mongodb副本集的时候不要使用
loaclahost
,否则在添加其他成员的时候会出现以下错误:rs0:PRIMARY> rs.add("192.168.2.122") { "ok" : 0, "errmsg" : "Either all host names in a replica set configuration must be localhost references, or none must be; found 1 out of 2", "code" : 103 }
解决方法:
rs0:PRIMARY> var config=rs.config() rs0:PRIMARY> config.members[0].host="192.168.2.121:27017" 192.168.2.121:27017 rs0:PRIMARY> rs.reconfig(config) { "ok" : 1 } rs0:PRIMARY> rs.config() { "_id" : "rs0", "version" : 2, "protocolVersion" : NumberLong(1), "members" : [ { "_id" : 0, "host" : "192.168.2.121:27017", "arbiterOnly" : false, "buildIndexes" : true, "hidden" : false, "priority" : 1, "tags" : { }, "slaveDelay" : NumberLong(0), "votes" : 1 } ], "settings" : { "chainingAllowed" : true, "heartbeatIntervalMillis" : 2000, "heartbeatTimeoutSecs" : 10, "electionTimeoutMillis" : 10000, "getLastErrorModes" : { }, "getLastErrorDefaults" : { "w" : 1, "wtimeout" : 0 }, "replicaSetId" : ObjectId("59ef0832a5da3378b1487f4e") } } rs0:PRIMARY> rs.add("192.168.2.122:27017") { "ok" : 1 }