{"__source__":"","__source___0":"","__tag__:__instance_id__":"ba11a23ebc39f1c0-5de8230d68621-3a8defe","__tag__:__job__":"sql-1650437116-774867","__tag__:__project__":"k8s-aplus-prod-1","__tag__:__schedule_time__":"1652025600","__tag__:__trigger_time__":"1652025630","__time__":"1651939200","__topic__":"","cnt":"101","keyword":"护发素"}
{"__source__":"","__source___0":"","__tag__:__instance_id__":"ba11a23ebc39f1c0-5de8230d68621-3a8defe","__tag__:__job__":"sql-1650437116-774867","__tag__:__project__":"k8s-aplus-prod-1","__tag__:__schedule_time__":"1652025600","__tag__:__trigger_time__":"1652025630","__time__":"1651939200","__topic__":"","cnt":"102","keyword":"厨房"}
{"__source__":"","__source___0":"","__tag__:__instance_id__":"ba11a23ebc39f1c0-5de8230d68621-3a8defe","__tag__:__job__":"sql-1650437116-774867","__tag__:__project__":"k8s-aplus-prod-1","__tag__:__schedule_time__":"1652025600","__tag__:__trigger_time__":"1652025630","__time__":"1651939200","__topic__":"","cnt":"102","keyword":"纸巾"}
这是一个文件
json 数据 开头和结尾不带[ ] ,因为从文件转换成JSONArray 要中括号 且每个json要有,逗号隔开
实现一
思路 先把文件按照字符读出来 使用StringBuffer拼接。(会把换行符也写进去)
后以换行符切割 把每个切割得到的字符转换成JSONObject, 后该干啥就干啥
File file1 = new File(filename);
private static List<String> test(File file) throws IOException{
long start = System.nanoTime();
List<String> keyword = new LinkedList<>();
Reader reader = new InputStreamReader(new FileInputStream(file), "Utf-8");
//读取每个字符char
int ch = 0;
StringBuffer sb = new StringBuffer();
while ((ch = reader.read()) != -1) {
sb.append((char) ch);
}
reader.close();
String jsonStr = sb.toString();
//以换行符切割字符串
String[] str = jsonStr.split("\n");
for (int i = 0; i < str.length; i++) {
//System.out.println(JSONObject.parseObject(str[i]).getString("keyword"));
//将每个字符串转为json对象 JSONObject.parseObject(String str); keyword.add(JSONObject.parseObject(str[i]).getString("keyword"));
}
long end = System.nanoTime();
System.out.println("time"+(end-start));
return keyword;
}
第二种
BufferedReader.readLine() 他能以按行的形式去读取 遇到空格或者回车返回`
刚好每一行就是一个JSON字符串。
源码如下
charLoop:
for (i = nextChar; i < nChars; i++) {
c = cb[i];
//遇到回车或者空格就结束循环遍历
if ((c == '\n') || (c == '\r')) {
eol = true;
break charLoop;
}
}
public static List<String> readFile(){
long start = System.nanoTime();
List<String> keyWordList = new ArrayList<>();
FileReader fr = null;
BufferedReader br = null;
try {
fr = new FileReader("filename");
br = new BufferedReader(fr);
String line = "";
//读取每一行的数据
while ( (line = br.readLine()) != null) {
JSONObject lineJson = JSONObject.parseObject(line);
keyWordList.add(lineJson.getString("keyword"));
}
} catch (Exception e) {
e.printStackTrace();
}finally {
try {
br.close();
fr.close();
} catch (Exception e) {
e.printStackTrace();
}
}
long end = System.nanoTime();
System.out.println("time"+(end-start));
return keyWordList;
}
数据读取
NIO允许你用一个单独的线程或几个线程管理很多个channels(网络的或者文件的),代价是程序的处理和处理IO相比更加复杂
如果你需要同时管理成千上万的连接,但是每个连接只发送少量数据,例如一个聊天服务器,用NIO实现会更好一些,相似的,如果你需要保持很多个到其他电脑的连接,例如P2P网络,用一个单独的线程来管理所有出口连接是比较合适的
如果你只有少量的连接但是每个连接都占有很高的带宽,同时发送很多数据,传统的IO会更适合