Linux is not Matrix——日志搜集平台ELK（II）

   今天就是ElasticSearch了。

   ElasticSearch是一个全文索引搜索引擎，想象成百度就行了，只不过这个搜索范围是所有日志内容。

   下载ElasticSearch，并解压

wget https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/2.3.3/elasticsearch-2.3.3.tar.gz

tar zxvf ./elasticsearch-2.3.3.tar.gz

   切换到非root用户，我这里就新建了个logs用户，后台启动ElasticSearch服务。

cd ./elasticsearch-2.3.3
nohup ./bin/elasticsearch &

   这个时候我们查看一下ElasticSearch的服务端口，9200，如果有进程说明已经启动了。

   我们在上一篇不是输入了系统日志作为测试日志吗，这时候就可以检查一下了。使用curl命令看一下是否收到了数据。

curl 'http://localhost:9200/_search?pretty'

   我这里显示了很多，大概是这个样子的。

[root@ceshi ~]# curl 'http://localhost:9200/_search?pretty'
{
  "took" : 372,
  "timed_out" : false,
  "_shards" : {
    "total" : 201,
    "successful" : 201,
    "failed" : 0
  },
  "hits" : {
    "total" : 472158,
    "max_score" : 1.0,
    "hits" : [ {
      "_index" : ".kibana",
      "_type" : "config",
      "_id" : "4.4.2",
      "_score" : 1.0,
      "_source" : {
        "buildNum" : 9732,
        "defaultIndex" : "logstash-*"
      }
    }, {
      "_index" : ".kibana",
      "_type" : "index-pattern",
      "_id" : "logstash-*",
      "_score" : 1.0,
      "_source" : {
        "title" : "logstash-*",
        "timeFieldName" : "@timestamp",
        "fields" : "[{\"name\":\"_source\",\"type\":\"_source\",\"count\":0,\"scripted\":false,\"indexed\":false,\"analyzed\":false,\"doc_values\":false},{\"name\":\"type\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"geoip.longitude\",\"type\":\"number\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"@version\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"host.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"path\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"geoip.ip\",\"type\":\"ip\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"host\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"_index\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":false,\"analyzed\":false,\"doc_values\":false},{\"name\":\"path.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"type.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"geoip.latitude\",\"type\":\"number\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"message\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"@timestamp\",\"type\":\"date\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"geoip.location\",\"type\":\"geo_point\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"_id\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":false,\"analyzed\":false,\"doc_values\":false},{\"name\":\"_type\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":false,\"analyzed\":false,\"doc_values\":false},{\"name\":\"_score\",\"type\":\"number\",\"count\":0,\"scripted\":false,\"indexed\":false,\"analyzed\":false,\"doc_values\":false}]"
      }
    }, {
      "_index" : "logstash-2016.03.28",
      "_type" : "syslog",
      "_id" : "AVO9QG2vtcchLJ9BRi_N",
      "_score" : 1.0,
      "_source" : {
        "message" : "Mar 28 20:43:48 ceshi salt-master: [INFO    ] Got return from Exam-Dev-248 for job 20160328204348739817",
        "@version" : "1",
        "@timestamp" : "2016-03-28T12:43:49.219Z",
        "path" : "/var/log/messages",
        "host" : "ceshi",
        "type" : "syslog"
      }
    }, {
      "_index" : "logstash-2016.03.28",
      "_type" : "syslog",
      "_id" : "AVO9QIG6tcchLJ9BRi_l",
      "_score" : 1.0,
      "_source" : {
        "message" : "Mar 28 20:43:53 ceshi salt-master: [DEBUG   ] Sending event - data = {'tgt_type': 'glob', 'jid': '20160328204353712759', 'return': True, 'retcode': 0, 'success': True, 'schedule': '__mine_interval', 'tgt': 'Basic-Test-211', 'cmd': '_return', 'pid': 11407, '_stamp': '2016-03-28T12:43:53.715346', 'arg': [], 'fun': 'mine.update', 'id': 'Basic-Test-211'}",
        "@version" : "1",
        "@timestamp" : "2016-03-28T12:43:54.351Z",
        "path" : "/var/log/messages",
        "host" : "ceshi",
        "type" : "syslog"
      }
    }, {
      "_index" : "logstash-2016.03.28",
      "_type" : "syslog",
      "_id" : "AVO9Ra9ntcchLJ9BRjAS",
      "_score" : 1.0,
      "_source" : {
        "message" : "Mar 28 20:49:32 ceshi salt-master: [DEBUG   ] Guessing ID. The id can be explicitly in set /etc/salt/minion",
        "@version" : "1",
        "@timestamp" : "2016-03-28T12:49:33.362Z",
        "path" : "/var/log/messages",
        "host" : "ceshi",
        "type" : "syslog"
      }
    }, {
      "_index" : "logstash-2016.03.28",
      "_type" : "syslog",
      "_id" : "AVO9Ra9ntcchLJ9BRjAT",
      "_score" : 1.0,
      "_source" : {
        "message" : "Mar 28 20:49:32 ceshi salt-master: [INFO    ] Found minion id from generate_minion_id(): ceshi",
        "@version" : "1",
        "@timestamp" : "2016-03-28T12:49:33.363Z",
        "path" : "/var/log/messages",
        "host" : "ceshi",
        "type" : "syslog"
      }
    }, {
      "_index" : "logstash-2016.03.28",
      "_type" : "syslog",
      "_id" : "AVO9RbJ8tcchLJ9BRjAV",
      "_score" : 1.0,
      "_source" : {
        "message" : "Mar 28 20:49:33 ceshi salt-master: [DEBUG   ] LazyLoaded local_cache.clean_old_jobs",
        "@version" : "1",
        "@timestamp" : "2016-03-28T12:49:34.370Z",
        "path" : "/var/log/messages",
        "host" : "ceshi",
        "type" : "syslog"
      }
    }, {
      "_index" : "logstash-2016.03.28",
      "_type" : "syslog",
      "_id" : "AVO9RbJ8tcchLJ9BRjAX",
      "_score" : 1.0,
      "_source" : {
        "message" : "Mar 28 20:49:33 ceshi salt-master: [DEBUG   ] Updating roots fileserver cache",
        "@version" : "1",
        "@timestamp" : "2016-03-28T12:49:34.385Z",
        "path" : "/var/log/messages",
        "host" : "ceshi",
        "type" : "syslog"
      }
    }, {
      "_index" : "logstash-2016.03.28",
      "_type" : "syslog",
      "_id" : "AVO9RMdUtcchLJ9BRjAK",
      "_score" : 1.0,
      "_source" : {
        "message" : "Mar 28 20:48:31 ceshi salt-master: [DEBUG   ] Guessing ID. The id can be explicitly in set /etc/salt/minion",
        "@version" : "1",
        "@timestamp" : "2016-03-28T12:48:31.807Z",
        "path" : "/var/log/messages",
        "host" : "ceshi",
        "type" : "syslog"
      }
    }, {
      "_index" : "logstash-2016.03.28",
      "_type" : "syslog",
      "_id" : "AVO9RMdUtcchLJ9BRjAL",
      "_score" : 1.0,
      "_source" : {
        "message" : "Mar 28 20:48:31 ceshi salt-master: [INFO    ] Found minion id from generate_minion_id(): ceshi",
        "@version" : "1",
        "@timestamp" : "2016-03-28T12:48:31.809Z",
        "path" : "/var/log/messages",
        "host" : "ceshi",
        "type" : "syslog"
      }
    } ]
  }
}

   凑个篇幅- -。

   然后我们可以装一个web插件，kopf，插件安装文件也在bin里面。

./bin/plugin install lmenezes/elasticsearch-kopf

   然后他就会自动从github上面下载kopf的web包，安装完成之后，会看到一个plugins文件夹，里面就有刚装的kopf。然而我这里并没有启动成功。。。好吧，这个就不用装了 - -。

   curl能有数据就够了。