java行数,Java中文件中的行数

I use huge data files, sometimes I only need to know the number of lines in these files, usually I open them up and read them line by line until I reach the end of the file

I was wondering if there is a smarter way to do that

解决方案

This is the fastest version I have found so far, about 6 times faster than readLines. On a 150MB log file this takes 0.35 seconds, versus 2.40 seconds when using readLines(). Just for fun, linux' wc -l command takes 0.15 seconds.

public static int countLines(String filename) throws IOException {

InputStream is = new BufferedInputStream(new FileInputStream(filename));

try {

byte[] c = new byte[1024];

int count = 0;

int readChars = 0;

boolean empty = true;

while ((readChars = is.read(c)) != -1) {

empty = false;

for (int i = 0; i < readChars; ++i) {

if (c[i] == '\n') {

++count;

}

}

}

return (count == 0 && !empty) ? 1 : count;

} finally {

is.close();

}

}

EDIT, 9 1/2 years later: I have practically no java experience, but anyways I have tried to benchmark this code against the LineNumberReader solution below since it bothered me that nobody did it. It seems that especially for large files my solution is faster. Although it seems to take a few runs until the optimizer does a decent job. I've played a bit with the code, and have produced a new version that is consistently fastest:

public static int countLinesNew(String filename) throws IOException {

InputStream is = new BufferedInputStream(new FileInputStream(filename));

try {

byte[] c = new byte[1024];

int readChars = is.read(c);

if (readChars == -1) {

// bail out if nothing to read

return 0;

}

// make it easy for the optimizer to tune this loop

int count = 0;

while (readChars == 1024) {

for (int i=0; i<1024;) {

if (c[i++] == '\n') {

++count;

}

}

readChars = is.read(c);

}

// count remaining characters

while (readChars != -1) {

System.out.println(readChars);

for (int i=0; i

if (c[i] == '\n') {

++count;

}

}

readChars = is.read(c);

}

return count == 0 ? 1 : count;

} finally {

is.close();

}

}

Benchmark resuls for a 1.3GB text file, y axis in seconds. I've performed 100 runs with the same file, and measured each run with System.nanoTime(). You can see that countLines has a few outliers, and countLinesNew has none and seems to be also slightly faster. LineNumberReader is clearly slower.

fjQQB.png

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值