系列文章目录
jmh学习笔记-源代码编译与bench mode
jmh学习笔记-State共享对象
jmh学习笔记-State共享对象前后置方法
jmh学习笔记-代码清除
jmh学习笔记-常量折叠
jmh学习笔记-Forking分叉
jmh学习笔记-环境配置
jmh学习笔记-缓存行的处理方式
jmh学习笔记-自定义项目引入jmh
前言
许多基准测试的失败之处在于死代码消除(DCE):编译器足够聪明,可以推断出某些计算是多余的,并且可以完全消除它们。 如果被淘汰的部分是我们的基准代码,那么我们就有麻烦了。 幸运的是,JMH提供了必要的基础架构来在适当的情况下与之抗争:返回计算结果将要求JMH处理该结果以限制死代码清除。
提示:以下是本篇文章正文内容,下面案例可供参考
一、基准测试
首先创建一个空的方法作为基准测试
@Benchmark
public void baseline() {
// do nothing, this is a baseline
}
二、代码清除
定义一个变量,然后再基准方法中进行计算
private double x = Math.PI;
@Benchmark
public void measureWrong() {
// This is wrong: result is not used and the entire computation is optimized away.
Math.log(x);
}
由于编译器优化,这段代码会被优化掉,真实方法运行时其实跟执行空方法差不多。
三、限制代码清除
1、通过return解决
通过返回计算的值,这样jmh框架就会采取措施限制代码清除了。
@Benchmark
public double measureRight() {
// This is correct: the result is being used.
return Math.log(x);
}
2、通过Blackhole解决
double x1 = Math.PI;
double x2 = Math.PI * 2;
@Benchmark
public void measureRight_2(Blackhole bh) {
bh.consume(Math.log(x1));
bh.consume(Math.log(x2));
}
四、统一测试
进行测试
package org.openjdk.jmh.samples;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.infra.Blackhole;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import java.util.concurrent.TimeUnit;
@State(Scope.Thread)
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
public class JMHSample_08_DeadCode {
/**
* The downfall of many benchmarks is Dead-Code Elimination (DCE): compilers
* are smart enough to deduce some computations are redundant and eliminate
* them completely. If the eliminated part was our benchmarked code, we are
* in trouble.
*
* Fortunately, JMH provides the essential infrastructure to fight this
* where appropriate: returning the result of the computation will ask JMH
* to deal with the result to limit dead-code elimination (returned results
* are implicitly consumed by Blackholes, see JMHSample_09_Blackholes).
*/
private double x = Math.PI;
double x1 = Math.PI;
double x2 = Math.PI * 2;
@Benchmark
public void baseline() {
// do nothing, this is a baseline
}
@Benchmark
public void measureWrong() {
// This is wrong: result is not used and the entire computation is optimized away.
Math.log(x);
}
@Benchmark
public double measureRight() {
// This is correct: the result is being used.
return Math.log(x);
}
/*
* While the Math.log(x2) computation is intact, Math.log(x1)
* is redundant and optimized out.
*/
@Benchmark
public double measureWrong_1() {
Math.log(x1);
return Math.log(x2);
}
/*
* This demonstrates Option A:
*
* Merge multiple results into one and return it.
* This is OK when is computation is relatively heavyweight, and merging
* the results does not offset the results much.
*/
@Benchmark
public double measureRight_1() {
return Math.log(x1) + Math.log(x2);
}
/*
* This demonstrates Option B:
*
* Use explicit Blackhole objects, and sink the values there.
* (Background: Blackhole is just another @State object, bundled with JMH).
*/
@Benchmark
public void measureRight_2(Blackhole bh) {
bh.consume(Math.log(x1));
bh.consume(Math.log(x2));
}
/*
* ============================== HOW TO RUN THIS TEST: ====================================
*
* You can see the unrealistically fast calculation in with measureWrong(),
* while realistic measurement with measureRight().
*
* You can run this test:
*
* a) Via the command line:
* $ mvn clean install
* $ java -jar target/benchmarks.jar JMHSample_08 -f 1
* (we requested single fork; there are also other options, see -h)
*
* b) Via the Java API:
* (see the JMH homepage for possible caveats when running from IDE:
* http://openjdk.java.net/projects/code-tools/jmh/)
*/
public static void main(String[] args) throws RunnerException {
Options opt = new OptionsBuilder()
.include(JMHSample_08_DeadCode.class.getSimpleName())
.forks(1)
.build();
new Runner(opt).run();
}
}
结果如下
# JMH version: 1.26
# VM version: JDK 1.8.0_121, Java HotSpot(TM) 64-Bit Server VM, 25.121-b13
# VM invoker: C:\Program Files\Java\jdk1.8.0_121\jre\bin\java.exe
# VM options: -javaagent:D:\Program Files\JetBrains\IntelliJ IDEA 2019.3.1\lib\idea_rt.jar=62593:D:\Program Files\JetBrains\IntelliJ IDEA 2019.3.1\bin -Dfile.encoding=UTF-8
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.openjdk.jmh.samples.JMHSample_08_DeadCode.baseline
# Run progress: 0.00% complete, ETA 00:11:40
# Fork: 1 of 1
# Warmup Iteration 1: 0.729 ns/op
# Warmup Iteration 2: 0.485 ns/op
# Warmup Iteration 3: 0.496 ns/op
# Warmup Iteration 4: 0.489 ns/op
# Warmup Iteration 5: 0.491 ns/op
Iteration 1: 0.502 ns/op
Iteration 2: 0.481 ns/op
Iteration 3: 0.477 ns/op
Iteration 4: 0.481 ns/op
Iteration 5: 0.480 ns/op
Result "org.openjdk.jmh.samples.JMHSample_08_DeadCode.baseline":
0.484 ±(99.9%) 0.038 ns/op [Average]
(min, avg, max) = (0.477, 0.484, 0.502), stdev = 0.010
CI (99.9%): [0.446, 0.522] (assumes normal distribution)
# JMH version: 1.26
# VM version: JDK 1.8.0_121, Java HotSpot(TM) 64-Bit Server VM, 25.121-b13
# VM invoker: C:\Program Files\Java\jdk1.8.0_121\jre\bin\java.exe
# VM options: -javaagent:D:\Program Files\JetBrains\IntelliJ IDEA 2019.3.1\lib\idea_rt.jar=62593:D:\Program Files\JetBrains\IntelliJ IDEA 2019.3.1\bin -Dfile.encoding=UTF-8
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.openjdk.jmh.samples.JMHSample_08_DeadCode.baseline_1
# Run progress: 14.29% complete, ETA 00:10:13
# Fork: 1 of 1
# Warmup Iteration 1: 35.808 ns/op
# Warmup Iteration 2: 35.206 ns/op
# Warmup Iteration 3: 35.288 ns/op
# Warmup Iteration 4: 35.533 ns/op
# Warmup Iteration 5: 35.477 ns/op
Iteration 1: 35.930 ns/op
Iteration 2: 35.836 ns/op
Iteration 3: 36.014 ns/op
Iteration 4: 40.345 ns/op
Iteration 5: 36.994 ns/op
Result "org.openjdk.jmh.samples.JMHSample_08_DeadCode.baseline_1":
37.024 ±(99.9%) 7.371 ns/op [Average]
(min, avg, max) = (35.836, 37.024, 40.345), stdev = 1.914
CI (99.9%): [29.652, 44.395] (assumes normal distribution)
# JMH version: 1.26
# VM version: JDK 1.8.0_121, Java HotSpot(TM) 64-Bit Server VM, 25.121-b13
# VM invoker: C:\Program Files\Java\jdk1.8.0_121\jre\bin\java.exe
# VM options: -javaagent:D:\Program Files\JetBrains\IntelliJ IDEA 2019.3.1\lib\idea_rt.jar=62593:D:\Program Files\JetBrains\IntelliJ IDEA 2019.3.1\bin -Dfile.encoding=UTF-8
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.openjdk.jmh.samples.JMHSample_08_DeadCode.measureRight
# Run progress: 28.57% complete, ETA 00:08:27
# Fork: 1 of 1
# Warmup Iteration 1: 40.035 ns/op
# Warmup Iteration 2: 35.284 ns/op
# Warmup Iteration 3: 36.791 ns/op
# Warmup Iteration 4: 35.808 ns/op
# Warmup Iteration 5: 35.508 ns/op
Iteration 1: 35.552 ns/op
Iteration 2: 35.419 ns/op
Iteration 3: 36.012 ns/op
Iteration 4: 35.681 ns/op
Iteration 5: 36.013 ns/op
Result "org.openjdk.jmh.samples.JMHSample_08_DeadCode.measureRight":
35.735 ±(99.9%) 1.037 ns/op [Average]
(min, avg, max) = (35.419, 35.735, 36.013), stdev = 0.269
CI (99.9%): [34.699, 36.772] (assumes normal distribution)
# JMH version: 1.26
# VM version: JDK 1.8.0_121, Java HotSpot(TM) 64-Bit Server VM, 25.121-b13
# VM invoker: C:\Program Files\Java\jdk1.8.0_121\jre\bin\java.exe
# VM options: -javaagent:D:\Program Files\JetBrains\IntelliJ IDEA 2019.3.1\lib\idea_rt.jar=62593:D:\Program Files\JetBrains\IntelliJ IDEA 2019.3.1\bin -Dfile.encoding=UTF-8
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.openjdk.jmh.samples.JMHSample_08_DeadCode.measureRight_1
# Run progress: 42.86% complete, ETA 00:06:45
# Fork: 1 of 1
# Warmup Iteration 1: 68.090 ns/op
# Warmup Iteration 2: 65.444 ns/op
# Warmup Iteration 3: 65.828 ns/op
# Warmup Iteration 4: 65.014 ns/op
# Warmup Iteration 5: 65.322 ns/op
Iteration 1: 64.722 ns/op
Iteration 2: 64.981 ns/op
Iteration 3: 67.061 ns/op
Iteration 4: 66.807 ns/op
Iteration 5: 65.290 ns/op
Result "org.openjdk.jmh.samples.JMHSample_08_DeadCode.measureRight_1":
65.772 ±(99.9%) 4.172 ns/op [Average]
(min, avg, max) = (64.722, 65.772, 67.061), stdev = 1.083
CI (99.9%): [61.600, 69.944] (assumes normal distribution)
# JMH version: 1.26
# VM version: JDK 1.8.0_121, Java HotSpot(TM) 64-Bit Server VM, 25.121-b13
# VM invoker: C:\Program Files\Java\jdk1.8.0_121\jre\bin\java.exe
# VM options: -javaagent:D:\Program Files\JetBrains\IntelliJ IDEA 2019.3.1\lib\idea_rt.jar=62593:D:\Program Files\JetBrains\IntelliJ IDEA 2019.3.1\bin -Dfile.encoding=UTF-8
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.openjdk.jmh.samples.JMHSample_08_DeadCode.measureRight_2
# Run progress: 57.14% complete, ETA 00:05:03
# Fork: 1 of 1
# Warmup Iteration 1: 69.931 ns/op
# Warmup Iteration 2: 70.959 ns/op
# Warmup Iteration 3: 69.991 ns/op
# Warmup Iteration 4: 69.420 ns/op
# Warmup Iteration 5: 69.740 ns/op
Iteration 1: 59.170 ns/op
Iteration 2: 54.409 ns/op
Iteration 3: 82.301 ns/op
Iteration 4: 72.901 ns/op
Iteration 5: 70.240 ns/op
Result "org.openjdk.jmh.samples.JMHSample_08_DeadCode.measureRight_2":
67.804 ±(99.9%) 42.881 ns/op [Average]
(min, avg, max) = (54.409, 67.804, 82.301), stdev = 11.136
CI (99.9%): [24.923, 110.686] (assumes normal distribution)
# JMH version: 1.26
# VM version: JDK 1.8.0_121, Java HotSpot(TM) 64-Bit Server VM, 25.121-b13
# VM invoker: C:\Program Files\Java\jdk1.8.0_121\jre\bin\java.exe
# VM options: -javaagent:D:\Program Files\JetBrains\IntelliJ IDEA 2019.3.1\lib\idea_rt.jar=62593:D:\Program Files\JetBrains\IntelliJ IDEA 2019.3.1\bin -Dfile.encoding=UTF-8
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.openjdk.jmh.samples.JMHSample_08_DeadCode.measureWrong
# Run progress: 71.43% complete, ETA 00:03:22
# Fork: 1 of 1
# Warmup Iteration 1: 0.487 ns/op
# Warmup Iteration 2: 0.479 ns/op
# Warmup Iteration 3: 0.480 ns/op
# Warmup Iteration 4: 0.511 ns/op
# Warmup Iteration 5: 0.490 ns/op
Iteration 1: 0.476 ns/op
Iteration 2: 0.473 ns/op
Iteration 3: 0.475 ns/op
Iteration 4: 0.512 ns/op
Iteration 5: 0.495 ns/op
Result "org.openjdk.jmh.samples.JMHSample_08_DeadCode.measureWrong":
0.486 ±(99.9%) 0.065 ns/op [Average]
(min, avg, max) = (0.473, 0.486, 0.512), stdev = 0.017
CI (99.9%): [0.421, 0.551] (assumes normal distribution)
# JMH version: 1.26
# VM version: JDK 1.8.0_121, Java HotSpot(TM) 64-Bit Server VM, 25.121-b13
# VM invoker: C:\Program Files\Java\jdk1.8.0_121\jre\bin\java.exe
# VM options: -javaagent:D:\Program Files\JetBrains\IntelliJ IDEA 2019.3.1\lib\idea_rt.jar=62593:D:\Program Files\JetBrains\IntelliJ IDEA 2019.3.1\bin -Dfile.encoding=UTF-8
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.openjdk.jmh.samples.JMHSample_08_DeadCode.measureWrong_1
# Run progress: 85.71% complete, ETA 00:01:41
# Fork: 1 of 1
# Warmup Iteration 1: 29.781 ns/op
# Warmup Iteration 2: 27.260 ns/op
# Warmup Iteration 3: 27.911 ns/op
# Warmup Iteration 4: 27.583 ns/op
# Warmup Iteration 5: 28.106 ns/op
Iteration 1: 27.576 ns/op
Iteration 2: 27.893 ns/op
Iteration 3: 27.800 ns/op
Iteration 4: 27.821 ns/op
Iteration 5: 27.421 ns/op
Result "org.openjdk.jmh.samples.JMHSample_08_DeadCode.measureWrong_1":
27.702 ±(99.9%) 0.758 ns/op [Average]
(min, avg, max) = (27.421, 27.702, 27.893), stdev = 0.197
CI (99.9%): [26.944, 28.460] (assumes normal distribution)
# Run complete. Total time: 00:11:48
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score Error Units
JMHSample_08_DeadCode.baseline avgt 5 0.484 ± 0.038 ns/op
JMHSample_08_DeadCode.baseline_1 avgt 5 37.024 ± 7.371 ns/op
JMHSample_08_DeadCode.measureRight avgt 5 35.735 ± 1.037 ns/op
JMHSample_08_DeadCode.measureRight_1 avgt 5 65.772 ± 4.172 ns/op
JMHSample_08_DeadCode.measureRight_2 avgt 5 67.804 ± 42.881 ns/op
JMHSample_08_DeadCode.measureWrong avgt 5 0.486 ± 0.065 ns/op
JMHSample_08_DeadCode.measureWrong_1 avgt 5 27.702 ± 0.758 ns/op
从上面的测试结果可以看出,measureWrong由于代码清除,最后平均执行时间为0.486ns与baseline空方法的0.484ns几乎一致,而没有被代码清除的很明显值大很多。measureRight为35.735ns,高出两个级别。这也从另一个方面说明了measureWrong方法中代码被编译器优化清除了。而measureWrong_1与measureRight_1和measureRight_2的时间也要差上一小半,反而与baseline_1相对接近,说明在measureWrong_1基准测试中也发生了代码清除。
总结
本章介绍了在测试过程中代码清除会导致测试结果不准确以及在jmh基准测试时如何解决编译器代码优化时清除无用代码。