12345678_0001
算法替换匹配字符串(没有正则表达式)的最有效方法之一是使用Aho-Corasick算法和高性能Trie(发音为“try”),快速散列算法和高效集合实现。简单代码一个简单的解决方案利用Apache StringUtils.replaceEach如下: private String testStringUtils(
final String text, final Map definitions ) {
final String[] keys = keys( definitions );
final String[] values = values( definitions );
return StringUtils.replaceEach( text, keys, values );
}这会减慢大文本的速度。快速代码Bor的 Aho-Corasick算法的实现引入了更多的复杂性,通过使用具有相同方法签名的façade成为实现细节: private String testBorAhoCorasick(
final String text, final Map definitions ) {
// Create a buffer sufficiently large that re-allocations are minimized.
final StringBuilder sb = new StringBuilder( text.length() <
final TrieBuilder builder = Trie.builder();
builder.onlyWholeWords();
builder.removeOverlaps();
final String[] keys = keys( definitions );
for( final String key : keys ) {
builder.addKeyword( key );
}
final Trie trie = builder.build();
final Collection emits = trie.parseText( text );
int prevIndex = 0;
for( final Emit emit : emits ) {
final int matchIndex = emit.getStart();
sb.append( text.substring( prevIndex, matchIndex ) );
sb.append( definitions.get( emit.getKeyword() ) );
prevIndex = emit.getEnd() + 1;
}
// Add the remainder of the string (contains no more matches).
sb.append( text.substring( prevIndex ) );
return sb.toString();
}基准对于基准测试,缓冲区是使用randomNumeric创建的,如下所示: private final static int TEXT_SIZE = 1000;
private final static int MATCHES_DIVISOR = 10;
private final static StringBuilder SOURCE = new StringBuilder( randomNumeric( TEXT_SIZE ) );在哪里MATCHES_DIVISOR指示要注入的变量数量: private void injectVariables( final Map definitions ) {
for( int i = (SOURCE.length() / MATCHES_DIVISOR) + 1; i > 0; i-- ) {
final int r = current().nextInt( 1, SOURCE.length() );
SOURCE.insert( r, randomKey( definitions ) );
}
}基准代码本身(JMH似乎有点过分):long duration = System.nanoTime();final String result = testBorAhoCorasick( text, definitions );duration = System.nanoTime() - duration;System.out.println( elapsed( duration ) );1,000,000:1,000一个简单的微基准测试,包含1,000,000个字符和1,000个随机放置的字符串。testStringUtils: 25秒,25533毫秒testBorAhoCorasick: 0秒,68毫秒没有比赛。10,000:1,000使用10,000个字符和1,000个匹配的字符串替换:testStringUtils: 1秒,1402毫秒testBorAhoCorasick: 0秒,37毫秒分歧关闭。1,000:10使用1,000个字符和10个匹配的字符串替换:testStringUtils: 0秒,7毫秒testBorAhoCorasick: 0秒,19毫秒对于短字符串,设置Aho-Corasick的开销会使蛮力方法黯然失色StringUtils.replaceEach。基于文本长度的混合方法是可能的,以获得两种实现的最佳效果。实现考虑比较长度超过1 MB的文本的其他实现,包括:https://github.com/RokLenarcic/AhoCorasickhttps://github.com/hankcs/AhoCorasickDoubleArrayTriehttps://github.com/raymanrt/aho-corasickhttps://github.com/ssundaresan/Aho-Corasickhttps://github.com/jmhsieh/aho-corasickhttps://github.com/quest-oss/Mensa文件与算法有关的论文和信息:http://www.cs.uku.fi/research/publications/reports/A-2005-2.pdfhttps://pdfs.semanticscholar.org/3547/ac839d02f6efe3f6f76a8289738a22528442.pdfhttp://www.ece.ncsu.edu/asic/ece792A/2009/ECE792A/Readings_files/00989753.pdfhttp://blog.ivank.net/aho-corasick-algorithm-in-as3.html