JMH简单使用

简介

JMH(Java Microbenchmark Harness)是用于代码微基准测试的工具套件，主要是基于方法层面的基准测试，精度可以达到纳秒级。该工具是由 Oracle 内部实现 JIT 的大牛们编写的，他们应该比任何人都了解 JIT 以及 JVM 对于基准测试的影响。

添加maven依赖

<dependency>
      <groupId>org.openjdk.jmh</groupId>
      <artifactId>jmh-core</artifactId>
      <version>1.21</version>
</dependency>
<dependency>
      <groupId>org.openjdk.jmh</groupId>
      <artifactId>jmh-generator-annprocess</artifactId>
      <version>1.21</version>
</dependency>

因为我们今天要测试fastjson和jackson的性能，所以引入它们的依赖

<!--fastjson依赖-->
<dependency>
      <groupId>com.alibaba</groupId>
      <artifactId>fastjson</artifactId>
      <version>1.2.56</version>
 </dependency>
<!--jackson依赖-->
 <dependency>
      <groupId>com.fasterxml.jackson.core</groupId>
      <artifactId>jackson-databind</artifactId>
      <version>2.9.4</version>
</dependency>
<dependency>
      <groupId>com.fasterxml.jackson.core</groupId>
      <artifactId>jackson-annotations</artifactId>
      <version>2.9.4</version>
</dependency>

例子

@BenchmarkMode(Mode.AverageTime)
@State(Scope.Thread)
@Warmup(iterations = 2, time = 1, timeUnit = TimeUnit.SECONDS)
@Fork(1)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@Measurement(iterations = 5)
public class Client {

  public static void main(String[] args) throws RunnerException {
    Options options = new OptionsBuilder()
        .build();
    new Runner(options).run();
  }

  @Benchmark
  public void testFastJsonSerialize(Blackhole blackhole) {
    List<User> userList = Arrays.asList(new User("lisi", "123"), new User("Tony", "456"));
    blackhole.consume(JSON.toJSONString(userList));
  }

  @Benchmark
  public void testJacksonSerialize(Blackhole blackhole) throws JsonProcessingException {
    List<User> userList = Arrays.asList(new User("lisi", "123"), new User("Tony", "456"));
    ObjectMapper objectMapper = new ObjectMapper();
    blackhole.consume(objectMapper.writeValueAsString(userList));
  }

  @AllArgsConstructor
  @NoArgsConstructor
  @Setter
  @Getter
  @ToString
  static class User {

    private String username;
    private String password;
  }
}

测试fastjson和jackson在序列化对象的功能上的性能对比，输出结果为

# JMH version: 1.21
# VM version: JDK 11, OpenJDK 64-Bit Server VM, 11+28
# VM invoker: D:javajdkopenjdk-11injava.exe
# VM options: -javaagent:C:Program FilesJetBrainsIntelliJ IDEA 2019.1.3libidea_rt.jar=8662:C:Program FilesJetBrainsIntelliJ IDEA 2019.1.3in -Dfile.encoding=UTF-8
# Warmup: 2 iterations, 1 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: com.imooc.sourcecode.java.jmh.test1.Client.testFastJsonSerialize

# Run progress: 0.00% complete, ETA 00:01:44
# Fork: 1 of 1
# Warmup Iteration   1: 0.858 us/op
# Warmup Iteration   2: 0.555 us/op
Iteration   1: 0.560 us/op
Iteration   2: 0.560 us/op
Iteration   3: 0.558 us/op
Iteration   4: 0.556 us/op
Iteration   5: 0.563 us/op


Result "com.imooc.sourcecode.java.jmh.test1.Client.testFastJsonSerialize":
  0.559 ±(99.9%) 0.010 us/op [Average]
  (min, avg, max) = (0.556, 0.559, 0.563), stdev = 0.003
  CI (99.9%): [0.549, 0.569] (assumes normal distribution)


# JMH version: 1.21
# VM version: JDK 11, OpenJDK 64-Bit Server VM, 11+28
# VM invoker: D:javajdkopenjdk-11injava.exe
# VM options: -javaagent:C:Program FilesJetBrainsIntelliJ IDEA 2019.1.3libidea_rt.jar=8662:C:Program FilesJetBrainsIntelliJ IDEA 2019.1.3in -Dfile.encoding=UTF-8
# Warmup: 2 iterations, 1 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: com.imooc.sourcecode.java.jmh.test1.Client.testJacksonSerialize

# Run progress: 50.00% complete, ETA 00:00:53
# Fork: 1 of 1
# Warmup Iteration   1: 79.133 us/op
# Warmup Iteration   2: 15.930 us/op
Iteration   1: 6.734 us/op
Iteration   2: 6.634 us/op
Iteration   3: 6.662 us/op
Iteration   4: 6.642 us/op
Iteration   5: 6.714 us/op


Result "com.imooc.sourcecode.java.jmh.test1.Client.testJacksonSerialize":
  6.677 ±(99.9%) 0.172 us/op [Average]
  (min, avg, max) = (6.634, 6.677, 6.734), stdev = 0.045
  CI (99.9%): [6.505, 6.849] (assumes normal distribution)


# Run complete. Total time: 00:01:46

REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.

Benchmark                     Mode  Cnt  Score   Error  Units
Client.testFastJsonSerialize  avgt    5  0.559 ± 0.010  us/op
Client.testJacksonSerialize   avgt    5  6.677 ± 0.172  us/op

可以看到fastjson确实比jackson快，fastjson平均一次0.5us（微妙）, jackson平均一次6us（微妙）。

注解分析

@Benchmark

此注解添加在方法上，表示该方法要进行基准测试

@BenchmarkMode

基准测试的类型，默认Throughput

• Throughput：整体吞吐量，每秒执行了多少次调用，单位为 ops/time
• AverageTime：用的平均时间，每次操作的平均时间，单位为 time/op
• SampleTime：随机取样，最后输出取样结果的分布
• SingleShotTime：只运行一次，往往同时把 Warmup 次数设为 0，用于测试冷启动时的性能
• All：上面的所有模式都执行一次

@State

表示一个对象的作用范围，默认Scope.Thread

• Scope.Benchmark：所有测试线程共享一个实例，测试有状态实例在多线程共享下的性能
• Scope.Group：同一个线程在同一个 group 里共享实例
• Scope.Thread：每个测试线程分配一个实例

@Warmup

程序预热相关配置，因为JVM中JIT（即时编译）的存在，某个函数多次执行之后，会被编译成机器码，提高执行速度，为了更接近真实情况，需要提前预热，让JIT充分工作。

• iterations：预热的次数，默认5
• time：每次预热的时间，默认10s
• timeUnit：时间的单位，默认秒
• batchSize：批处理大小，每次操作调用几次方法，默认1次

@Measurement

测试基准测试相关配置，参数和@Warmup相同

@Fork

进程数量，默认5

@OutputTimeUnit

统计结果的时间单位，一般为秒，微妙，纳秒

使用陷阱

在使用JMH的过程中，会存在一些陷阱，如编译器进行的优化常量折叠，

String str = "A" + "B";

编译器会直接优化成

String str = "AB";

JIT的优化死码消除

String str = "";
for (int i = 0; i < 10; i++) {
  str += i;
}

JVM认为str没有被使用，整段代码都会被优化掉。所以我们在测试的过程中要避免编译器优化和JIT优化

• 测试方法不要返回void
• 返回void情况，使用Blackhole.consume()消除JIT优化