springboot 集成micrometer 对接elasticsearch文档_source字段缺失问题

springboot 集成micrometer 对接elasticsearch 问题

问题描述:
在springboot中对接了micrometer和elasticsearch之后发现在es中产生了索引，并且索引中有文档，但是该索引对接的文档中却没有对应的具体的内容。

问题产生原因：上面的配置过程会在es中产生一个默认的索引模板， 该模板的mapping中的 “_source”字段的设置是false, 而在查找文档的时候, 字段都展示在"_source".

官方原文如下:

_source field itself is not indexed (and thus is not searchable), but it is stored so that it can be returned when executing fetch requests, like get or search.

意思大致是, 如果_source是fasle那么将无法被查找, 但是仍然在文档中存着. 只是在查找的时候不会显示出来. 就像这样

{
“_index”: “metrics-2019-08”,
“_type”: “_doc”,
“_id”: “nWuMdWwBxBoi4XILEHVK”,
“_score”: 1.0
}

索引模板查看方式:
http://localhost:9200/_template

然后可以在mappings中看到_source的enabled是false.

“metrics_template”: {
“order”: 0,
“index_patterns”: [
“micrometer-metrics-*”
],
“settings”: {},
“mappings”: {
“_source”: {
“enabled”: false
},
“properties”: {
“duration”: {
“index”: false,
“type”: “double”
},
“total”: {
“index”: false,
“type”: “double”
},
“max”: {
“index”: false,
“type”: “double”
},
“mean”: {
“index”: false,
“type”: “double”
},
“name”: {
“type”: “keyword”
},
“count”: {
“index”: false,
“type”: “double”
},
“active”: {
“index”: false,
“type”: “double”
},
“sum”: {
“index”: false,
“type”: “double”
},
“value”: {
“index”: false,
“type”: “double”
},
“unknown”: {
“index”: false,
“type”: “double”
}
}
},
“aliases”: {}
}

解决办法

删除该索引模板, 并且重新插入一个_source字段为true的模板.最简单的方式就是复制该索引模板的json内容, 然后只修改该字段的值, 然后创建索引模板. 当然这个方式只是治标不治本, 应该还有一开始就配置好的方式,只是目前没有发现。

删除模板metrics_template：
delete http://localhost:9200/_template/metrics_template

创建模板metrics_template：
put http://localhost:9200/_template/metrics_template

“metrics_template”: {
“order”: 0,
“index_patterns”: [
“micrometer-metrics-*”
],
“settings”: {},
“mappings”: {
“_source”: {
“enabled”: true
},
“properties”: {
“duration”: {
“index”: false,
“type”: “double”
},
“total”: {
“index”: false,
“type”: “double”
},
“max”: {
“index”: false,
“type”: “double”
},
“mean”: {
“index”: false,
“type”: “double”
},
“name”: {
“type”: “keyword”
},
“count”: {
“index”: false,
“type”: “double”
},
“active”: {
“index”: false,
“type”: “double”
},
“sum”: {
“index”: false,
“type”: “double”
},
“value”: {
“index”: false,
“type”: “double”
},
“unknown”: {
“index”: false,
“type”: “double”
}
}
},
“aliases”: {}
}

micrometer集成过程

依赖:

 <dependency>
      <groupId>org.springframework.data</groupId>
      <artifactId>spring-data-elasticsearch</artifactId>
      <version>4.2.7</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/io.micrometer/micrometer-registry-elastic -->
    <dependency>
      <groupId>io.micrometer</groupId>
      <artifactId>micrometer-registry-elastic</artifactId>
      <version>1.8.0</version>
    </dependency>

代码

@Component
public class ElasticMeterRegister implements ApplicationRunner {

    public void elasticTest() {
        ElasticConfig elasticConfig = new ElasticConfig() {
            @Override
            @Nullable
            public String get(String k) {
                return null;
            }
        };

        MeterRegistry registry = new ElasticMeterRegistry(elasticConfig, Clock.SYSTEM);
        Metrics.addRegistry(registry);

        Counter counter = Metrics.counter("test.random.count", "tag1", "val1", "tag2", "val2");
        Random random = new Random();

        ScheduledExecutorService executor = Executors.newSingleThreadScheduledExecutor();
        executor.scheduleAtFixedRate(() -> {
                    int countNum = random.nextInt(100);
                    System.out.println("当前count: " + countNum);
                    counter.increment(countNum);
                },
                2L, 2L, TimeUnit.SECONDS);

        LockSupport.parkUntil(System.currentTimeMillis() + 101 * 1000);
        System.out.println("Run over ...");
        executor.shutdown();
        registry.close();
    }

    @Override
    public void run(ApplicationArguments args) throws Exception {
        elasticTest();
    }

配置文件:

management.metrics.export.elastic:
  # You will probably want disable Elastic publishing in a local development profile.
  enabled: true

  # The interval at which metrics are sent to Elastic. The default is 1 minute.
  step: 1m

  # The index to store metrics in, defaults to "micrometer-metrics"
  index: micrometer-metrics

在启动程序后在es中:

通过restapi 可以发现产生了索引

http://localhost:9200/_cat/indices?format=json