Elasticsearch 的API 分为 REST Client API(http请求形式)以及 transportClient API两种。相比来说transportClient API效率更高,transportClient 是通过Elasticsearch内部RPC的形式进行请求的,连接可以是一个长连接,相当于是把客户端的请求当成
Elasticsearch 集群的一个节点。但是从Elasticsearch 7 后就会移除transportClient 。主要原因是transportClient 难以向下兼容版本。
本文中所有的讲解和操作都是基于jdk 1.8 和elasticsearch 6.2.4版本。
备注:本文参考了很多Elasticsearch 的官方文档以及部l网络资料做的综合整理。
一、High REST Client
High Client 基于 Low Client, 主要目的是暴露一些 API,这些 API 可以接受请求对象为参数,返回响应对象,而对请求和响应细节的处理都是由 client 自动完成的。
API 在调用时都可以是同步或者异步两种形式
同步 API 会导致阻塞,一直等待数据返回
异步 API 在命名上会加上 async 后缀,需要有一个 listener 作为参数,等这个请求返回结果或者发生错误时,这个 listener 就会被调用,listener主要是解决自动回调的问题,有点像安卓 开发里面的listener监听回调。
Elasticsearch REST APi 官方 地址:https://www.elastic.co/guide/en/elasticsearch/reference/6.2/index.html
Maven 依赖
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>6.2.4</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>6.2.4</version>
</dependency>
client初始化:
RestHighLevelClient 实例依赖 REST low-level client builder
public class ElasticSearchClient {
private String[] hostsAndPorts;
public ElasticSearchClient(String[] hostsAndPorts) {
this.hostsAndPorts = hostsAndPorts;
}
public RestHighLevelClient getClient() { RestHighLevelClient client = null; List<HttpHost> httpHosts = new ArrayList<HttpHost>(); if (hostsAndPorts.length > 0) { for (String hostsAndPort : hostsAndPorts) { String[] hp = hostsAndPort.split(":"); httpHosts.add(new HttpHost(hp[0], Integer.valueOf(hp[1]), "http")); } client = new RestHighLevelClient( RestClient.builder(httpHosts.toArray(new HttpHost[0]))); } else { client = new RestHighLevelClient( RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); } return client; }
}
文档 API(High level rest 客户端支持下面的 文档(Document) API):
- 单文档 API:
- index API
- Get API
- Delete API
- Update API
- 多文档 API:
- Bulk API
- Multi-Get API
1、Index API:
IndexRequest:
封装好的参考方法:
private IndexRequest getIndexRequest(String index, String indexType, String docId, Map<String, Object> dataMap) {
IndexRequest indexRequest = null;
if (null == index || null == indexType) {
throw new ElasticsearchException("index or indexType must not be null");
}
if (null == docId) {
indexRequest = new IndexRequest(index, indexType);
} else {
indexRequest = new IndexRequest(index, indexType, docId);
}
return indexRequest;
}
/**
* 同步执行索引
*
* @param index
* @param indexType
* @param docId
* @param dataMap
* @throws IOException
*/
public IndexResponse execIndex(String index, String indexType, String docId, Map<String, Object> dataMap) throws IOException {
return getClient().index(getIndexRequest(index, indexType, docId, dataMap).source(dataMap));
}
/**
* 异步执行
*
* @param index
* @param indexType
* @param docId
* @param dataMap
* @param indexResponseActionListener
* @throws IOException
*/
public void asyncExecIndex(String index, String indexType, String docId, Map<String, Object> dataMap, ActionListener<IndexResponse> indexResponseActionListener) throws IOException {
getClient().indexAsync(getIndexRequest(index, indexType, docId, dataMap).source(dataMap), indexResponseActionListener);
}
API解释:
IndexRequest request = new IndexRequest( "posts", // 索引 Index "doc", // Type "1"); // 文档 Document Id String jsonString = "{" + "\"user\":\"kimchy\"," + "\"postDate\":\"2013-01-30\"," + "\"message\":\"trying out Elasticsearch\"" + "}"; request.source(jsonString, XContentType.JSON); // 文档源格式为 json string
Document Source
document source 可以是下面的格式
Map类型的输入:
Map<String, Object> jsonMap = new HashMap<>(); jsonMap.put("user", "kimchy"); jsonMap.put("postDate", new Date()); jsonMap.put("message", "trying out Elasticsearch"); IndexRequest indexRequest = new IndexRequest("posts", "doc", "1") .source(jsonMap); // 会自动将 Map 转换为 JSON 格式
XContentBuilder : 这是 Document Source 提供的帮助类,专门用来产生 json 格式的数据:
XContentBuilder builder = XContentFactory.jsonBuilder(); builder.startObject(); { builder.field("user", "kimchy"); builder.timeField("postDate", new Date()); builder.field("message", "trying out Elasticsearch"); } builder.endObject(); IndexRequest indexRequest = new IndexRequest("posts", "doc", "1") .source(builder);
Object 键对:
IndexRequest indexRequest = new IndexRequest("posts", "doc", "1") .source("user", "kimchy", "postDate", new Date(), "message", "trying out Elasticsearch");
同步索引:
IndexResponse indexResponse = client.index(request);
异步索引:异步执行函数需要添加 listener, 而对于 index 而言,这个 listener 的类型就是 ActionListener
client.indexAsync(request, listener);
异步方法执行后会立刻返回,在索引操作执行完成后,ActionListener 就会被回调:
执行成功,调用 onResponse 函数
执行失败,调用 onFailure 函数
ActionListener<IndexResponse> listener = new ActionListener<IndexResponse>() { @Override public void onResponse(IndexResponse indexResponse) { } @Override public void onFailure(Exception e) { } };
IndexResponse:
不管是同步回调还是异步回调,如果调用成功,都会返回 IndexRespose 对象。
String index = indexResponse.getIndex(); String type = indexResponse.getType(); String id = indexResponse.getId(); long version = indexResponse.getVersion(); if (indexResponse.getResult() == DocWriteResponse.Result.CREATED) { // 文档第一次创建 } else if (indexResponse.getResult() == DocWriteResponse.Result.UPDATED) { // 文档之前已存在,当前是重写 } ReplicationResponse.ShardInfo shardInfo = indexResponse.getShardInfo(); if (shardInfo.getTotal() != shardInfo.getSuccessful()) { // 成功的分片数量少于总分片数量 } if (shardInfo.getFailed() > 0) { for (ReplicationResponse.ShardInfo.Failure failure : shardInfo.getFailures()) { String reason = failure.reason(); // 处理潜在的失败信息 } }
在索引时有版本冲突的话,会抛出 ElasticsearchException
IndexRequest request = new IndexRequest("posts", "doc", "1") .source("field", "value") .version(1); // 这里是文档版本号 try { IndexResponse response = client.index(request); } catch(ElasticsearchException e) { if (e.status() == RestStatus.CONFLICT) { // 冲突了 } }
如果将 opType 设置为 create, 而且如果索引的文档与已存在的文档在 index, type 和 id 上均相同,也会抛出冲突异常。
IndexRequest request = new IndexRequest("posts", "doc", "1") .source("field", "value") .opType(DocWriteRequest.OpType.CREATE); try { IndexResponse response = client.index(request); } catch(ElasticsearchException e) { if (e.status() == RestStatus.CONFLICT) { } }
2、GET API
GET 请求
每个 GET 请求都必须需传入下面 3 个参数:
- Index
- Type
- Document id
GetRequest getRequest = new GetRequest( "posts", "doc", "1");
可选参数
下面的参数都是可选的, 里面的选项并不完整,如要获取完整的属性,请参考 官方文档
不获取源数据,默认是获取的
request.fetchSourceContext(FetchSourceContext.DO_NOT_FETCH_SOURCE);
配置返回数据中包含指定字段
String[] includes = new String[]{"message", "*Date"}; String[] excludes = Strings.EMPTY_ARRAY; FetchSourceContext fetchSourceContext = new FetchSourceContext(true, includes, excludes); request.fetchSourceContext(fetchSourceContext);
配置返回数据中排除指定字段
String[] includes = Strings.EMPTY_ARRAY; String[] excludes = new String[]{"message"}; FetchSourceContext fetchSourceContext = new FetchSourceContext(true, includes, excludes); request.fetchSourceContext(fetchSourceContext);
实时 默认为 true
request.realtime(false);
版本
request.version(2);
版本类型
request.versionType(VersionType.EXTERNAL);
同步执行
GetResponse getResponse = client.get(getRequest);
异步执行
此部分与 index 相似, 只有一点不同, 返回类型为 GetResponse
Get Response
返回的 GetResponse 对象包含要请求的文档数据(包含元数据和字段)
String index = getResponse.getIndex(); String type = getResponse.getType(); String id = getResponse.getId(); if (getResponse.isExists()) { long version = getResponse.getVersion(); String sourceAsString = getResponse.getSourceAsString(); // string 形式 Map<String, Object> sourceAsMap = getResponse.getSourceAsMap(); // map byte[] sourceAsBytes = getResponse.getSourceAsBytes(); // 字节形式 } else { // 没有发现请求的文档 }
在请求中如果包含特定的文档版本,如果与已存在的文档版本不匹配, 就会出现冲突
try { GetRequest request = new GetRequest("posts", "doc", "1").version(2); GetResponse getResponse = client.get(request); } catch (ElasticsearchException exception) { if (exception.status() == RestStatus.CONFLICT) { // 版本冲突 } }
封装好的参考方法: /** * @param index * @param indexType * @param docId * @param includes 返回需要包含的字段,可以传入空 * @param excludes 返回需要不包含的字段,可以传入为空 * @param excludes version * @param excludes versionType * @return * @throws IOException */ public GetResponse getRequest(String index, String indexType, String docId, String[] includes, String[] excludes, Integer version, VersionType versionType) throws IOException { if (null == includes || includes.length == 0) { includes = Strings.EMPTY_ARRAY; } if (null == excludes || excludes.length == 0) { excludes = Strings.EMPTY_ARRAY; } GetRequest getRequest = new GetRequest(index, indexType, docId); FetchSourceContext fetchSourceContext = new FetchSourceContext(true, includes, excludes); getRequest.realtime(true); if (null != version) { getRequest.version(version); } if (null != versionType) { getRequest.versionType(versionType); } return getClient().get(getRequest.fetchSourceContext(fetchSourceContext)); } /** * @param index * @param indexType * @param docId * @param includes * @param excludes * @return * @throws IOException */ public GetResponse getRequest(String index, String indexType, String docId, String[] includes, String[] excludes) throws IOException { return getRequest(index, indexType, docId, includes, excludes, null, null); } /** * @param index * @param indexType * @param docId * @return * @throws IOException */ public GetResponse getRequest(String index, String indexType, String docId) throws IOException { GetRequest getRequest = new GetRequest(index, indexType, docId); return getClient().get(getRequest); }
3、Exists API
如果文档存在 Exists API 返回 true, 否则返回 fasle。
Exists Request
GetRequest 用法和 Get API 差不多,两个对象的可选参数是相同的。由于 exists() 方法只返回 true 或者 false, 建议将获取 _source 以及任何存储字段的值关闭,尽量使请求轻量级。
GetRequest getRequest = new GetRequest( "posts", // Index "doc", // Type "1"); // Document id getRequest.fetchSourceContext(new FetchSourceContext(false)); // 禁用 _source 字段 getRequest.storedFields("_none_"); // 禁止存储任何字段
同步请求
boolean exists = client.exists(getRequest);
异步请求
异步请求与 Index API 相似,此处不赘述,只粘贴代码。如需详细了解,请参阅官方地址
ActionListener<Boolean> listener = new ActionListener<Boolean>() { @Override public void onResponse(Boolean exists) { } @Override public void onFailure(Exception e) { } }; client.existsAsync(getRequest, listener);
封装的参考方法:
/** * @param index * @param indexType * @param docId * @return * @throws IOException */ public Boolean existDoc(String index, String indexType, String docId) throws IOException { GetRequest getRequest = new GetRequest(index, indexType, docId); getRequest.fetchSourceContext(new FetchSourceContext(false)); getRequest.storedFields("_none_"); return getClient().exists(getRequest); }
4、Delete API
Delete Request
DeleteRequest 必须传入下面参数
DeleteRequest request = new DeleteRequest( "posts", // index "doc", // doc "1"); // document id
可选参数
超时时间
request.timeout(TimeValue.timeValueMinutes(2));
request.timeout("2m");
刷新策略
request.setRefreshPolicy(WriteRequest.RefreshPolicy.WAIT_UNTIL);
request.setRefreshPolicy("wait_for");
版本
request.version(2);
版本类型
request.versionType(VersionType.EXTERNAL);
同步执行
DeleteResponse deleteResponse = client.delete(request);
异步执行
ActionListener<DeleteResponse> listener = new ActionListener<DeleteResponse>() { @Override public void onResponse(DeleteResponse deleteResponse) { } @Override public void onFailure(Exception e) { } };
client.deleteAsync(request, listener);
Delete Response
DeleteResponse 可以检索执行操作的信息
String index = deleteResponse.getIndex(); String type = deleteResponse.getType(); String id = deleteResponse.getId(); long version = deleteResponse.getVersion(); ReplicationResponse.ShardInfo shardInfo = deleteResponse.getShardInfo(); if (shardInfo.getTotal() != shardInfo.getSuccessful()) { // 成功分片数目小于总分片 } if (shardInfo.getFailed() > 0) { for (ReplicationResponse.ShardInfo.Failure failure : shardInfo.getFailures()) { String reason = failure.reason(); // 处理潜在失败 } }
也可以来检查文档是否存在
DeleteRequest request = new DeleteRequest("posts", "doc", "does_not_exist"); DeleteResponse deleteResponse = client.delete(request); if (deleteResponse.getResult() == DocWriteResponse.Result.NOT_FOUND) { // 文档不存在 } 版本冲突时也会抛出 `ElasticsearchException try { DeleteRequest request = new DeleteRequest("posts", "doc", "1").version(2); DeleteResponse deleteResponse = client.delete(request); } catch (ElasticsearchException exception) { if (exception.status() == RestStatus.CONFLICT) { // 版本冲突 } }
封装好的参考方法:
/** * @param index * @param indexType * @param docId * @param timeValue * @param refreshPolicy * @param version * @param versionType * @return * @throws IOException */ public DeleteResponse deleteDoc(String index, String indexType, String docId, TimeValue timeValue, WriteRequest.RefreshPolicy refreshPolicy, Integer version, VersionType versionType) throws IOException { DeleteRequest deleteRequest = new DeleteRequest(index, indexType, docId); if (null != timeValue) { deleteRequest.timeout(timeValue); } if (null != refreshPolicy) { deleteRequest.setRefreshPolicy(refreshPolicy); } if (null != version) { deleteRequest.v