当我们要把很多数据上传至ElasticSearch时,不可能一条条的put进入ElasticSearch,所以我们借助logstash帮助我们导入大量的数据,为了方便实验,我们本篇内容先模拟数据,在下篇介绍如何把模拟好的数据导入ElasticSearch.
-
通过xftp上传logstash-6.2.2.tar.gz至/optinstall/elk目录下
-
将logstash-6.2.2.tar.gz解压至/opt/bigdata/elk目录下,并改名为logstash622
-
打开IEDA,创建一个maven工程,编写能自动生成文本内容的代码
package cn.kgc.kb08.es.log;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.text.MessageFormat;
import java.util.Random;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class LogService {
private ExecutorService es = Executors.newFixedThreadPool(50);
private Random rand = new Random();
private String[] eventType = {"visit","btn_search","btn_page","btn_add_cart","btn_make_order","btn_pay"};
private String[] browser = {"chrome","firefox","360","ie","猎豹"};
private String[] timeZone = {"尼亚美","开罗","伦敦","莫斯科","新德里","北京","东京","悉尼","温哥华","纽约","巴西利亚","里约热内卢"};
private String path;
public LogService(String path) {
this.path = path;
}
private String makeMsg(){
Object[] parts = {
String.valueOf(1+rand.nextInt(10000)),
eventType[rand.nextInt(eventType.length)],
MessageFormat.format("{0}-{1}",String.valueOf(2010+rand.nextInt(11)),1+rand.nextInt(12)),
browser[rand.nextInt(browser.length)],
1+rand.nextInt(100),
timeZone[rand.nextInt(timeZone.length)]
};
return MessageFormat.format("{0}:{1}:{2}:{3}:{4}:{5}",parts);
}
private class RandFileLog implements Runnable{
@Override
public void run() {
String msg = makeMsg();
synchronized (RandomAccessFile.class){
RandomAccessFile raf = null;
try {
raf = new RandomAccessFile(path,"rw");
raf.seek(raf.length());
raf.write((msg+"\r\n").getBytes());
} catch (Exception e) {
e.printStackTrace();
}finally {
if(null != raf){
try {
raf.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
}
}
public void log(int count){
if(count<=0) return;
for (int i = 0; i <count ; i++) {
es.submit(new RandFileLog());
}
}
public static void main( String[] args )
{
new LogService("log/mylog.log").log(10);
}
}
-
根目录下创建一个log包,里面有一个mylog.log文件,运行main方法,生成文档
-
成功生成规则的文本内容,也没有出现乱码,说明程序没有问题
-
在APP类里写一个main方法,第一个参数设为要存放生成文本的路径,第二个参数是生成文本的行数,将程序打包成jar包
package cn.kgc.kb08.es;
import cn.kgc.kb08.es.log.LogService;
/**
* Hello world!
*
*/
public class App
{
public static void main( String[] args )
{
new LogService(args[0]).log(Integer.parseInt(args[1]));
}
}
以下都是生成jar包的详细过程
选择第一个APP方法
然后ok,apply应用再ok
上面是为jar包生成做准备,下面正式生成jar包,点击Build下的Build Artifacts
第一次生成jar包点Build,如果是第二次生成jar包点击Rebuild
这样就能帮我们生成一个out文件,里面就是我们需要的jar包
- 最后还剩一步,需要把生成的一个META-INF文件放入jar包中,jar包可以用压缩方式打开,把META-INF拖入即可