bulk Java提交数据,ELK研究（一）:elasticsearch java api接口操作ES集群 ---TransportClient的使用介绍 bulk批量提交数据...

Java client操作ES： 1：配置集群对象信息；2：创建客户端；3：查看集群信息

1、设置集群名字

默认集群名为elasticsearch，如果集群名称和指定的不一致则在使用节点资源时会报错。

Settings settings = Settings.builder()

.put("cluster.name", "myClusterName").build();

TransportClient client = new PreBuiltTransportClient(settings);

//Add transport addresses and do something with the client...

2、嗅探功能

通过client.transport.sniff启动嗅探功能，这样只需要指定集群中的某一个节点(不一定是主节点)，然后会加载集群中的其他节点，这样只要程序不停即使此节点宕机仍然可以连接到其他节点。

Settings settings = Settings.builder()

.put("client.transport.sniff", true).build();

TransportClient client = new PreBuiltTransportClient(settings);

3、创建client

// on startup

TransportClient client = new PreBuiltTransportClient(Settings.EMPTY)

.addTransportAddress(new TransportAddress(InetAddress.getByName("host1"), 9300))

.addTransportAddress(new TransportAddress(InetAddress.getByName("host2"), 9300));

// on shutdown

client.close();

4、完整示例–elasticsearch java api bulk批量提交数据

json数据保存在文中，代码是通过读文件的方式：

{“10190”:0,”10071”:0,”10191”:0,”10070”:0,”48”:”122186”,”type”:”122186”,”10018”:0,”10117”:0,”332”:0,”333”:0,”999”:3202,”10178”:10131,”8503”:0,”8504”:0,”10135”:”SUSP”,”31”:0,”10116”:0,”35”:”UA3202”,”10185”:0,”10184”:0,”140”:100000,”387”:0,”8538”:”P010”,”10044”:0,”10187”:0,”10043”:0,”10186”:0,”10189”:0,”10068”:[],”10188”:0,”10202”:-1,”10146”:1,”10069”:[],”10204”:0,”10203”:-1}

{“10190”:0,”10071”:0,”10191”:0,”10070”:0,”48”:”122186”,”type”:”122186”,”10018”:0,”10117”:0,”332”:0,”333”:0,”999”:3202,”10178”:10131,”8503”:0,”8504”:0,”10135”:”SUSP”,”31”:0,”10116”:0,”35”:”UA3202”,”10185”:0,”10184”:0,”140”:100000,”387”:0,”8538”:”P010”,”10044”:0,”10187”:0,”10043”:0,”10186”:0,”10189”:0,”10068”:[],”10188”:0,”10202”:-1,”10146”:1,”10069”:[],”10204”:0,”10203”:-1}

{“10190”:0,”10071”:0,”10191”:0,”10070”:0,”48”:”122186”,”type”:”122186”,”10018”:0,”10117”:0,”332”:0,”333”:0,”999”:3202,”10178”:10131,”8503”:0,”8504”:0,”10135”:”SUSP”,”31”:0,”10116”:0,”35”:”UA3202”,”10185”:0,”10184”:0,”140”:100000,”387”:0,”8538”:”P010”,”10044”:0,”10187”:0,”10043”:0,”10186”:0,”10189”:0,”10068”:[],”10188”:0,”10202”:-1,”10146”:1,”10069”:[],”10204”:0,”10203”:-1}

import org.elasticsearch.action.bulk.BulkRequestBuilder;

import org.elasticsearch.client.transport.TransportClient;

import org.elasticsearch.common.settings.Settings;

import org.elasticsearch.common.transport.InetSocketTransportAddress;

import org.elasticsearch.common.transport.TransportAddress;

import org.elasticsearch.transport.client.PreBuiltTransportClient;

import static org.elasticsearch.common.xcontent.XContentFactory.*;

import java.io.*;

import java.net.InetAddress;

import java.net.InetSocketAddress;

import java.net.UnknownHostException;

import java.util.HashMap;

import java.util.Map;

public class Main {

public static void main(String[] args) {

try {

//通过setting来制定集群信息，单机就不需要制定了集群信息了，去掉即可

Settings settings = Settings.builder().

put("cluster.name", "elk_test.cluster")

.put("client.transport.sniff",true).build();

//创建客户端clients

InetAddress address = InetAddress.getByName("192.168.0.153"); // ip

InetSocketAddress socketAddress=new InetSocketAddress(address, 9300); //socket address = ip + port

TransportAddress transportAddress = new InetSocketTransportAddress(socketAddress);

TransportClient client = new PreBuiltTransportClient(settings).addTransportAddress(transportAddress);

File file = new File("E:/json");

FileReader reader=new FileReader(file);

BufferedReader bfr=new BufferedReader(reader);

String line;

BulkRequestBuilder bulkRequest=client.prepareBulk();

int count=0;

while((line=bfr.readLine())!=null){

bulkRequest.add(client.prepareIndex("dddddddd","article").setSource(line));

if (count%10==0) {

bulkRequest.execute().actionGet();

bulkRequest=client.prepareBulk();

}

count++;

//System.out.println(line);

}

bulkRequest.execute().actionGet();

bfr.close();

reader.close();

} catch (UnknownHostException e) {

e.printStackTrace();

} catch (FileNotFoundException e) {

e.printStackTrace();

} catch (IOException e) {

e.printStackTrace();

}

}

}