1、背景
最近做了个elasticsearch搜索的功能,因为会有多个项目模块用到所以决定做成一个springboot starter以便于其他地方用到就不需要作过多的配置,加之不久前出现一个项目的es服务出问题导致几个共用一个es集群的项目都受到影响,需要不同项目之间用的es隔离开来,所以决定手写自定义的Starter并动态的创建RestHighLevelClient。
2、starter的大概结构
3、开发流程
第一步创建一个springboot项目,命名为starter-elasticsearch
创建DynamicElasticsearchProperties类,通过@ConfigurationProperties注解把yml或者properties配置文件转化为bean,从而获取yml或者properites配置文件的属性
创建ElasticsearchProperties类,对应yml或者properites配置文件的属性
创建自动配置的bean,命名为ElasticsearchAutoConfiguration,通过@EnableConfigurationProperties()注解使得@ConfigurationProperties注解生效,后续自动装配动态的创建RestHighLevelClient将在这个类进行
假设需要引入es的项目自定义yml配置如下:
maoshi8:
elasticsearch:
datasource:
node1:
host: 123.123.123.123
port: 9200
node2:
host: 111.111.111.111
port: 9200
DynamicElasticsearchProperties类:
@ConfigurationProperties(prefix = "maoshi8.elasticsearch")
public class DynamicElasticsearchProperties {
/**
* 配置多动态数据源key datasource id
*/
private Map<String, ElasticsearchProperties> datasource = new HashMap<String, ElasticsearchProperties>();
public Map<String, ElasticsearchProperties> getDatasource() {
return datasource;
}
public void setDatasource(Map<String, ElasticsearchProperties> datasource) {
this.datasource = datasource;
}
}
@Data
public class ElasticsearchProperties {
private String schema = "http";
private String host;
private Integer port;
private String username = "elastic";
private String password = "******";
private int connectTimeOut = 1000;
private int socketTimeOut = 30000;
private int connectionRequestTimeOut = 500;
private int maxConnectNum = 100;
private int maxConnectPerRoute = 100;
private boolean uniqueConnectTimeConfig = true;
private boolean uniqueConnectNumConfig = true;
}
DynamicElasticsearchProperties类里ConfigurationProperties注解到maoshi8.elasticsearch就能截取到key=node1和key=node2的value值为ElasticsearchProperties对象的map=>datasource。其中除了host和port属性都给了默认值,我们只需要拿到动态配置的host和port就能动态的创建不同的RestHighLevelClient。这里是定义了两个node1和node2,如果定义了10个就能相应的拿到一个有10个配置对象的datasource。
第二步创建一个META-INF目录
创建META-INF目录后,在里面新建一个spring.factories文件,添加以下代码:
org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
ElasticsearchAutoConfiguration
这里的ElasticsearchAutoConfiguration是我自定义自动配置的类的reference路径。spring boot项目启动以后SpringFactoriesLoader.loadFactoryNames方法调用loadSpringFactories方法从所有的jar包中读取META-INF/spring.factories文件信息,通过反射机制将spring.factories中@Configuration类实例化为对应的java实列,发现要自动配置的bean。
第三步动态的创建RestHighLevelClient
前面已经能自动找到配置类,并且拿到yml或者properties的参数属性了,接下来就只需要在自动配置类ElasticsearchAutoConfiguration里面循环datasource里的定义配置,动态的创建RestHighLevelClient就可以了。
代码如下:
自动配置类ElasticsearchAutoConfiguration:
@Configuration
@EnableConfigurationProperties(DynamicElasticsearchProperties.class)
public class ElasticsearchAutoConfiguration {
@Autowired
private DynamicElasticsearchProperties dynamicElasticsearchProperties;
private final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
@Bean
public RestHighLevelClientUtils restHighLevelClientUtils(){
RestHighLevelClientUtils restHighLevelClientUtils = new RestHighLevelClientUtils();
Map<String, ElasticsearchProperties> datasourceMap = dynamicElasticsearchProperties.getDatasource();
Map<String, RestHighLevelClient> restHighLevelClientMap = new HashMap<>();
for (String key : datasourceMap.keySet()) {
ElasticsearchProperties elasticsearchProperties = datasourceMap.get(key);
HttpHost httpHost = new HttpHost(elasticsearchProperties.getHost(),
elasticsearchProperties.getPort(),elasticsearchProperties.getSchema());
RestClientBuilder builder = RestClient.builder(httpHost);
if (elasticsearchProperties.isUniqueConnectTimeConfig()) {
setConnectTimeOutConfig(builder ,elasticsearchProperties);
}
if (elasticsearchProperties.isUniqueConnectNumConfig()) {
setMutiConnectConfig(builder ,elasticsearchProperties);
}
RestHighLevelClient client = new RestHighLevelClient(builder);
restHighLevelClientMap.put(key,client);
RestHighLevelClientUtils.registerRestHighLevelClient(key, client);
}
return restHighLevelClientUtils;
}
/** * 异步httpclient的连接延时配置 */
public void setConnectTimeOutConfig(RestClientBuilder builder, ElasticsearchProperties elasticsearchProperties) {
builder.setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() {
@Override
public RequestConfig.Builder customizeRequestConfig(RequestConfig.Builder requestConfigBuilder) {
requestConfigBuilder.setConnectTimeout(elasticsearchProperties.getConnectTimeOut());
requestConfigBuilder.setSocketTimeout(elasticsearchProperties.getSocketTimeOut());
requestConfigBuilder.setConnectionRequestTimeout(elasticsearchProperties.getConnectionRequestTimeOut());
return requestConfigBuilder;
}
});
}
/** * 异步httpclient的连接数配置 */
public void setMutiConnectConfig(RestClientBuilder builder, ElasticsearchProperties elasticsearchProperties) {
builder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
@Override
public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
httpClientBuilder.setMaxConnTotal(elasticsearchProperties.getMaxConnectNum());
httpClientBuilder.setMaxConnPerRoute(elasticsearchProperties.getMaxConnectPerRoute());
//elasticsearch7.x版本需要密码
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(elasticsearchProperties.getUsername(), elasticsearchProperties.getPassword()));
httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
return httpClientBuilder;
}
});
}
}
工具类RestHighLevelClientUtils :
public class RestHighLevelClientUtils {
private static Map<String, RestHighLevelClient> restHighLevelClientMap = new ConcurrentHashMap<>();
public RestHighLevelClientUtils() {
}
public static RestHighLevelClient getRestHighLevelClient(String restHighLevelClientId) {
RestHighLevelClient restHighLevelClient = restHighLevelClientMap.get(restHighLevelClientId);
if(restHighLevelClient == null){
throw new RuntimeException("找不到RestHighLevelClient,restHighLevelClientId:"+ restHighLevelClientId);
}
return restHighLevelClient;
}
public RestHighLevelClient getClient(String restHighLevelClientId) {
return RestHighLevelClientUtils.getRestHighLevelClient(restHighLevelClientId);
}
public static void registerRestHighLevelClient(String restHighLevelClientId, RestHighLevelClient restHighLevelClient) {
RestHighLevelClientUtils.restHighLevelClientMap.put(restHighLevelClientId, restHighLevelClient);
}
}
这里我是用的一个工具类RestHighLevelClientUtils定义一个ConcurrentHashMap来存放动态创建的RestHighLevelClient,map的key就是datasource的key:node1、node2,value就是创建的RestHighLevelClient。
第四步引入starter使用
引入pom文件:
<!-- 添加自定义的starter-elasticsearch -->
<dependency>
<groupId>com.maoshi8</groupId>
<artifactId>starter-elasticsearch</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>
需要RestHighLevelClient使用处添加代码:
private static RestHighLevelClient client;
@Resource
private RestHighLevelClientUtils restHighLevelClientUtils;
@PostConstruct
public void init(){
client = restHighLevelClientUtils.getClient("node1");
}
需要引入的模块只需要加上自定义的yml配置,并在pom文件引入该starter,再restHighLevelClientUtils工具类就能拿到对应的RestHighLevelClient了