【ELK】自定义ES-5.4.1插件步骤2——带action-&amp;gt;transportaction分层结构并包含详细的集成测试步骤

最新推荐文章于 2023-04-06 17:13:10 发布

断桥残雪D

最新推荐文章于 2023-04-06 17:13:10 发布

阅读量545

点赞数

分类专栏： ElasticSearch

ElasticSearch 专栏收录该内容

7 篇文章 0 订阅

订阅专栏

以一个自定义的maven project为例，该项目主要实现一个简单的查询功能，通过curl -XGET http://127.0.0.1:9200/logstash-713/_taste/event命令，可以查询logstash-713索引的内容，在这里，_taste/event为一个自定义的查询命令。项目结构如下：

com.taste.elasticsearch_taste

action
LoggerUtils
Preconditions
TasteEventAction
TransportTasteEventAction

common

TasteEventRequest
TasteEventRequestBuilder
tasteEventResponse

plugin
TastePlugin

rest
TasteEventRestAction

一，插件编写

项目内部实现的映射关系为： rest***Action-> ***Action

***Action->Transport***Action

实际上，ES的client只是一个门面，在每个方法后面都有一个action来承接相应的功能。但是action并不是一个真正的实现者，他的真正的实现者是transportaction。

项目实现的整体思路如下：

1. 在***Plugin类中，不仅仅要新注册一个Rest***Action,还要将***Action类与Transport***Action类绑定到一起。

public class TastePlugin extends Plugin implements ActionPlugin{

           .....            //省略掉一些次要步骤

           //将action与transportaction绑定
      @Override
      public List<ActionHandler<? extends ActionRequest, ? extends ActionResponse>> getActions() {
         // TODO Auto-generated method stub
          List<ActionHandler<? extends ActionRequest, ? extends ActionResponse>> extra=new ArrayList<>();
         extra.add(new ActionHandler<>(TasteEventAction.INSTANSE, TransportTasteEventAction.class));
          return extra;
      }

    //注册restAction
     @Override
     public List<RestHandler> getRestHandlers(Settings settings, RestController restController, ClusterSettings clusterSettings,     IndexScopedSettings indexScopedSettings, SettingsFilter settingsFilter, IndexNameExpressionResolver indexNameExpressionResolver, Supplier<DiscoveryNodes> nodesInCluster) {

         return Collections.singletonList(new TasteEventRestAction(settings, restController));
   }   
}

2. 设计相应的***Request, ***Response和***RequestBuilder类

根据自己的需求设计这三个类，因为这个例子中，就是实现一个查询功能，所以TasteEventRequest类中定义一个ES中自带的SearchRequest变量，TasteEventResponse中定义一个ES自带的SearchResponse变量作为输出。***RequestBuilder类好像就是固定的那种模式。

public  class  TasteEventRequest  extends  ActionRequest{
    private String prefix;      //这个没啥用
    private SearchRequest searchRequest;
   @Override
   public ActionRequestValidationException validate() {
      // TODO Auto-generated method stub
      //这个函数用来检测request请求是否符合标准，还没具体实现
      return null;
   }
   public SearchRequest getSearchRequest() {
      return searchRequest;
   }
   public TasteEventRequest setSearchRequest(SearchRequest searchRequest) {
      this.searchRequest = searchRequest;
      return this;
   }
   public TasteEventRequest setSearchRequest(SearchRequestBuilder builder){
      return setSearchRequest(builder.request());
   }
   public String getPrefix() {
      return prefix;
   }
   public TasteEventRequest setPrefix(String prefix) {
      this.prefix = prefix;
      return this;
   }
   //下面这俩函数有啥用暂时不知道
   @Override
   public void readFrom(StreamInput in) throws IOException {
      // TODO Auto-generated method stub
      super.readFrom(in);
      prefix=in.readString();
   }
   
   @Override
   public void writeTo(StreamOutput out) throws IOException {
      // TODO Auto-generated method stub
      super.writeTo(out);
      out.writeString(prefix);
   }
     
}

***Response类要继承StatusToXContentObject接口，这个好像是用于将response对象转换成json对象输出用的。

public class TasteEventResponse extends ActionResponse implements StatusToXContentObject{
    private SearchResponse searchResponse;
   public TasteEventResponse(SearchResponse searchResponse) {
      // TODO Auto-generated constructor stub
      this.searchResponse=Preconditions.checkNotNull(searchResponse);
   }
   public TasteEventResponse(){
      
   }
   public SearchResponse getSearchResponse() {
        return searchResponse;
    }
   
   @Override
   public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
      // TODO Auto-generated method stub
      if (searchResponse != null) {
            searchResponse.innerToXContent(builder, ToXContent.EMPTY_PARAMS);
        }
      return builder;
   }
   @Override
   public RestStatus status() {
      // TODO Auto-generated method stub
      return OK;
   }  
}

3.编写Rest***Action类

3.1 注册新命令

restController.registerHandler(RestRequest.Method.GET, "/_taste/event", this);
           restController.registerHandler(RestRequest.Method.GET, "/{index}/_taste/event", this);
           restController.registerHandler(RestRequest.Method.GET, "/{index}/{type}/_taste/event", this);

3.2 在 prepareRequest函数中实现功能，首先构造自定义的request请求，然后将请求传入对应的***Action类中实现具体功能，要用到异步编程。

@Override
      protected RestChannelConsumer prepareRequest(RestRequest request, NodeClient client) throws IOException {
         //构造自定义的request请求
         final TasteEventRequestBuilder actionBuilder=new TasteEventRequestBuilder(client);
         SearchRequest searchRequest=new SearchRequest();
         RestSearchAction.parseSearchRequest(searchRequest, request, null);
         actionBuilder.setSearchRequest(searchRequest);
            //实现项目功能，必须要用异步编程
            return channel -> client.execute(TasteEventAction.INSTANSE, actionBuilder.request(),
                    new ActionListener<TasteEventResponse>() {
                        @Override
                        public void onResponse(TasteEventResponse response) {
                            try {
                              //将response转换成json类型，不过这个未转化成功
                                XContentBuilder builder = channel.newBuilder();
                                builder.startObject();
                                response.toXContent(builder, request);
                                builder.endObject();
                                channel.sendResponse(
                                        new BytesRestResponse(
                                              RestStatus.OK,
                                                builder));
                            } catch (Exception e) {
                                logger.debug("Failed to emit response.", e);
                                onFailure(e);
                            }
                        }
                        @Override
                        public void onFailure(Exception e) {
                            emitErrorResponse(channel, logger, e);
                        }
                    });
      }

4. 编写***Action类，这个类的内容基本都是固定的。具体的实现逻辑在Transport***Action类中。

public class TasteEventAction extends Action<TasteEventRequest, TasteEventResponse, TasteEventRequestBuilder>{
   public static final String NAME="tastevent";
   public static final TasteEventAction INSTANSE = new TasteEventAction();
   protected TasteEventAction() {
      super(NAME);
      // TODO Auto-generated constructor stub
   }
   @Override
   public TasteEventRequestBuilder newRequestBuilder(ElasticsearchClient client) {
      // TODO Auto-generated method stub
      return new TasteEventRequestBuilder(client);
   }
   @Override
   public TasteEventResponse newResponse() {
      // TODO Auto-generated method stub
      return new TasteEventResponse();
   }
}

5.编写Transport***Action类，这个类主要有三个函数，具体的解释见注释。其中使用@Inject注释的方法中，Transport***Aciton类是注入进来的，可根据需求自行改变。

public class TransportTasteEventAction extends TransportAction<TasteEventRequest, TasteEventResponse> {
   
   //因为要实现查询，所以要用到ES原有的TransportSearchAction,将它注入进来以便使用
   private final TransportSearchAction searchAction;  
   @Inject
   public TransportTasteEventAction(Settings settings, String actionName, ThreadPool threadPool,
         ActionFilters actionFilters, IndexNameExpressionResolver indexNameExpressionResolver,
         TransportService transportService,TransportSearchAction searchAction,NamedXContentRegistry xContentRegistry) {
      super(settings, TasteEventAction.NAME, threadPool,
                actionFilters,
                indexNameExpressionResolver,
                transportService.getTaskManager());
      this.searchAction=searchAction;
      // TODO Auto-generated constructor stub
      
      
      //这个函数是将 ***Action与Transport***Action绑定在一起，其实是与TransportHandler绑定在一起。
      transportService.registerRequestHandler(
                TasteEventAction.NAME,
                TasteEventRequest::new,
                ThreadPool.Names.SAME,
                new TransportHandler());
      
   }
   
    //这个函数是具体的逻辑实现函数
   @Override
   protected void doExecute(TasteEventRequest request, ActionListener<TasteEventResponse> listener) {
      // TODO Auto-generated method stub
      
      //执行查询，异步编程
      searchAction.execute(request.getSearchRequest(), new ActionListener<SearchResponse>(){
         @Override
         public void onResponse(SearchResponse response) {
            // TODO Auto-generated method stub
             listener.onResponse(new TasteEventResponse(response));
         }
         @Override
         public void onFailure(Exception e) {
            // TODO Auto-generated method stub
            listener.onFailure(e);
         }
         
      });
   }
   
   private final class TransportHandler implements TransportRequestHandler<TasteEventRequest> {
        @Override
        public void messageReceived(final TasteEventRequest request, final TransportChannel channel) throws Exception {
            execute(request, new ActionListener<TasteEventResponse>() {
                @Override
                public void onResponse(TasteEventResponse response) {
                    try {
                        channel.sendResponse(response);
                    } catch (Exception e) {
                        onFailure(e);
                    }
                }
                @Override
                public void onFailure(Exception e) {
                    try {
                        channel.sendResponse(e);
                    } catch (Exception e1) {
                        logger.warn("Failed to send error response for action ["
                                + TasteEventAction.NAME + "] and request [" + request + "]", e1);
                    }
                }
            });
        }
    }
}

二集成测试类编写，需要编写两个类。TastePlugin 和 TasteEventRestTest。

这个测试类运行时开启的节点自我感觉应该是临时节点，具体的原理也不太清楚，如果通过命令行向启动的节点中输入命令创建索引的话也会成功，查询索引也能查到，但是一旦关闭节点再次开启数据就没有了。

1. 编写TastePlugin类

该类继承ESIntegTestCase类，主要实现Elasticsearch节点的开启和插件的加载。其中要注意的几点有：

1）代码中①部分的配置是必须的，因为ESIntegTestCase中http模块默认是禁用的，但是由于我们要使用Rest API，所以需要启用http模块。①处便是启用http模块的配置

2）代码中②部分也是必须的，因为我们启用http模块的时候就需要这个插件，否则会报异常，因为没有这个插件不支持http类型的设置。

3）代码中③部分是将自定义的插件添加到ES中

4）代码中④处的固定的，也是必须的。

5）类上面的那行注释@ESIntegTestCase.ClusterScope .....可以保证集群只开启一个节点（如果不加这行注释会报错，具体不详）

@ESIntegTestCase.ClusterScope(scope = ESIntegTestCase.Scope.SUITE, numDataNodes = 1, numClientNodes = 0, transportClientRatio = 0, supportsDedicatedMasters = false)
public  class  TastePluginTest   extends  ESIntegTestCase {
   protected String restBaseUrl;
    protected Client client;
    int randomPort = 9200;
    public TastePluginTest(){
      
    }
       @Override
       protected Settings nodeSettings(int nodeOrdinal) {
           return Settings.builder()
                   .put(super.nodeSettings(nodeOrdinal))
                   .put(NetworkModule.HTTP_ENABLED.getKey(), true)     .........................................①
                   .put(HttpTransportSettings.SETTING_HTTP_PORT.getKey(), randomPort)
                   .put("network.host", "127.0.0.1")
                   .build();
       }
      
       @Override
       protected Collection<Class<? extends Plugin>> nodePlugins() {
         //return Collections.singletonList(TastePlugin.class);
         Collection<Class<? extends Plugin>> al = new ArrayList<Class<? extends Plugin>>();
         al.add(Netty4Plugin.class);  .............................................................................②
          al.add(TastePlugin.class);   .............................................................................③
          return  al;
       }
      
       @Override
       protected Collection<Class<? extends Plugin>> transportClientPlugins() { .............................................④
           return nodePlugins();
       }
   
}

2. 编写TasteEventRestTest类，测试新注册的rest命令。

public class TasteEventRestTest extends TastePluginTest {
   public void test_recommended_items_from_user() throws Exception {
      final String index = "blog";
      XContentType type = randomFrom(XContentType.values());
      try (CloseableHttpClient httpClient = HttpClientBuilder.create().build()) {
           
              //下面这个语句代表从集群中获取各个节点的ip地址，不过目前集群就开启了一个节点，也就endpoint[0]有值
              //如果集群中有多个节点，采用随机化测试，可用
              //InetSocketAddress endpoint =randomFrom(cluster().httpAddresses());来从节点集群中随机获取一个节点的ip地址
                InetSocketAddress[] endpoint = cluster().httpAddresses();

              //得到url路径，生成get请求
                this.restBaseUrl = "http://" + NetworkAddress.format(endpoint[0]);
                HttpGet get=new HttpGet(restBaseUrl + "/"+index+"/_taste/event");

                HttpResponse response = httpClient.execute(get);
                System.out.println("post请求已发送");

               //最后应该用一个断言来判断测试是否成功，不过并不知道怎么判断
               //assertEquals("{\"acknowledged\":true}", response );
                 
             }
      
   }
}

启动测试类的之后，因为集群每次重启后什么数据都没有，所以不管查什么索引都会报IndexNotFound异常，所以可以在测试代码中加断点，debug启动，然后在节点启动成功后，在执行该自定义的查询命令之前，可以先在外部git上执行命令建个索引，如执行curl -XPUT http://127.0.0.1:9200/blog和curl -XPUT http://127.0.0.1:9200/blog/es/1 -d '{"first_name":"test1"}'命令（建索引并插入一条数据），然后继续debug就可看到执行自定义的查询命令后的查询结果。通过这种方式可以调试插件。测试成功后导出jar包压缩成ES插件就可。打包成插件后执行curl -XGET http://127.0.0.1:9200/logstash-713/_taste/event命令可以查询logstash-713索引的全部内容，不过好像并没有以JSON格式输出。

断桥残雪D

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
【ELK】自定义ES-5.4.1插件步骤2——带action-&amp;gt;transportaction分层结构并包含详细的集成测试步骤

以一个自定义的maven project为例，该项目主要实现一个简单的查询功能，通过curl -XGET http://127.0.0.1:9200/logstash-713/_taste/event命令，可以查询logstash-713索引的内容，在这里，_taste/event为一个自定义的查询命令。项目结构如下：com.taste.elasticsearch_tasteaction...
复制链接

扫一扫