垃圾回收G1日志解析

G1垃圾回收器介绍

首先要搞清楚G1垃圾回收的过程

主要分为下面4个步骤：

初始标记（Initial Marking）           由于常规标记对象是否直接关联到GC roots，过程简单，停顿时间短

并发标记（Concurrent Marking） 由于可达性性分析，存在较长停顿

最终标记（Final Marking）           由于把并发标记时漏掉的那部分垃圾内存标记起来，比较少，停顿时间短

筛选回收（Live Data Couting and Evacuation）  对标记后的垃圾内存进行整理，按指定计划进行回收。

在下面的案例中，红色部分明确标明了这4个阶段。

 G1垃圾收集器参数说明

以我的spark参数为例：

spark.executor.extraJavaOptions -Dtag=mobius.optimize -XX:+UseG1GC -XX:+PrintFlagsFinal -XX:+PrintReferenceGC -XX:+PrintGCDetails

-XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps-XX:+PrintAdaptiveSizePolicy -XX:+PrintGC -Xloggc:/tmp/spark.gc.log

-XX:+UnlockDiagnosticVMOptions -XX:+G1SummarizeConcMark -XX:InitiatingHeapOccupancyPercent=35 -XX:ConcGCThreads=15

-XX:+PrintGCApplicationConcurrentTime -XX:-UseGCOverheadLimit

下面是其拆分后的注释：

spark.executor.extraJavaOptions     传入java参数给spark各个excutor

-Dtag=mobius.optimize 　　　　　　标记名

-XX:+UseG1GC 　　　　　　　　　指定垃圾收集器

-XX:+PrintFlagsFinal 　　　　　　　java对象引用分为四种：强引用FinalReference，软引用SoftReference，弱引用WeakReference，虚引用PhantomReference。一般都是强引用，比如new一个对象并赋值

-XX:+PrintReferenceGC 　　　　　打印垃圾对象的引用数量

-XX:+PrintGCDetails 　　　　　　　 打印详细GC日志

-XX:+PrintGCDateStamps 　　　　 打印可读的日期时间戳 比如2018-11-12

-XX:+PrintGCTimeStamps 　　　　打印可读的时间戳 比如 09:10:11

-XX:+PrintAdaptiveSizePolicy          启用自适应大小（为了避免fullGC，允许改变eden空间和survivor空间大小）

-XX:+PrintGC 　　　　　　　　　　打印GC日志

-Xloggc:/tmp/spark.gc.log 　　　　   指定GC日志路径

-XX:+UnlockDiagnosticVMOptions

-XX:+G1SummarizeConcMark          

-XX:InitiatingHeapOccupancyPercent=35

-XX:ParallelGCThreads=33 　　　　　　 Parallel和G1收集器的标记阶段的线程数l量，默认33，对应GC日志 [ GC Workers: 33 ]（并行：多个垃圾收集线程并行工作，此时用户线程处于等待状态，stop the world）

-XX:ConcGCThreads=15 　　　　　　     CMS和G1回收器的前三个标记阶段都是多线程的，指定GC并发线程数量，默认0（并发：用户线程与垃圾收集线程同时执行，不一定是并行，可能会交替执行）

-XX:+PrintGCApplicationConcurrentTime          打印应用程序执行时间

-XX:-UseGCOverheadLimit 　　　　　       　  取消垃圾回收时的内存限制（减号）

-XX:MaxGCPauseMillis=200　　　　  　　　　垃圾回收停顿最长时间，默认是200ms

日志分析案例

下面一段日志是我在spark计算任务中产生的垃圾回收日志：

2018-11-13T05:32:44.473+0800: 1.857: Application time: 0.2130324 seconds
2018-11-13T05:32:44.571+0800: 1.954: Application time: 0.0970449 seconds
2018-11-13T05:32:44.696+0800: 2.079: Application time: 0.1249397 seconds
2018-11-13T05:32:44.874+0800: 2.258: Application time: 0.1776871 seconds
 2.258: [G1Ergonomics (Concurrent Cycles) request concurrent cycle initiation, reason: requested by GC cause, GC cause: Metadata GC Threshold]
 2.258: [G1Ergonomics (Concurrent Cycles) initiate concurrent cycle, reason: concurrent cycle initiation requested]
2018-11-13T05:32:44.875+0800: 2.259: [GC pause (Metadata GC Threshold) (young) (initial-mark) 2.260: 
[G1Ergonomics (CSet Construction) start choosing CSet, _pending_cards: 0, predicted base time: 10.00 ms, 
remaining time: 190.00 ms, target pause time: 200.00 ms]
 2.260: [G1Ergonomics (CSet Construction) add young regions to CSet, eden: 60 regions, survivors: 0 regions, predicted young region time: 3605.71 ms]
 2.260: [G1Ergonomics (CSet Construction) finish choosing CSet, eden: 60 regions, survivors: 0 regions, old: 0 regions, 
predicted pause time: 3615.71 ms, target pause time: 200.00 ms]
2018-11-13T05:32:44.948+0800: 2.331: [SoftReference, 0 refs, 0.0002103 secs]2018-11-13T05:32:44.948+0800: 2.332: 
[WeakReference, 970 refs, 0.0005817 secs]2018-11-13T05:32:44.949+0800: 2.332: [FinalReference, 4265 refs, 0.0266394 secs]
2018-11-13T05:32:44.975+0800: 2.359: [PhantomReference, 0 refs, 3 refs, 0.0000585 secs]2018-11-13T05:32:44.976+0800: 2.359: 
[JNI Weak Reference, 0.0001374 secs], 0.1040572 secs]
   [Parallel Time: 29.1 ms, GC Workers: 33]
      [GC Worker Start (ms): Min: 2260.1, Avg: 2265.0, Max: 2265.9, Diff: 5.8]
      [Ext Root Scanning (ms): Min: 0.2, Avg: 18.2, Max: 22.7, Diff: 22.5, Sum: 691.2]
      [Update RS (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
         [Processed Buffers: Min: 0, Avg: 0.0, Max: 0, Diff: 0, Sum: 0]
      [Scan RS (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.3]
      [Code Root Scanning (ms): Min: 0.0, Avg: 0.3, Max: 8.6, Diff: 8.6, Sum: 10.0]
      [Object Copy (ms): Min: 0.0, Avg: 3.9, Max: 22.9, Diff: 22.9, Sum: 147.1]
      [Termination (ms): Min: 0.0, Avg: 1.3, Max: 2.3, Diff: 2.3, Sum: 50.3]
         [Termination Attempts: Min: 1, Avg: 3.0, Max: 17, Diff: 16, Sum: 115]
      [GC Worker Other (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 1.6]
      [GC Worker Total (ms): Min: 22.7, Avg: 23.7, Max: 28.6, Diff: 5.9, Sum: 900.6]
      [GC Worker End (ms): Min: 2288.6, Avg: 2288.7, Max: 2288.8, Diff: 0.2]
   [Code Root Fixup: 1.0 ms]
   [Code Root Purge: 0.1 ms]
   [Clear CT: 1.0 ms]
   [Other: 72.8 ms]
      [Choose CSet: 0.0 ms]
      [Ref Proc: 68.9 ms]
      [Ref Enq: 0.2 ms]
      [Redirty Cards: 0.4 ms]
      [Humongous Register: 0.1 ms]
      [Humongous Reclaim: 0.1 ms]
      [Free CSet: 0.5 ms]
   [Eden: 240.0M(512.0M)->0.0B(496.0M) Survivors: 0.0B->16.0M Heap: 254.0M(10.0G)->31.7M(10.0G)]
 [Times: user=0.44 sys=0.09, real=0.11 secs]
2018-11-13T05:32:44.980+0800: 2.363: [GC concurrent-root-region-scan-start]
2018-11-13T05:32:44.996+0800: 2.380: [GC concurrent-root-region-scan-end, 0.0167701 secs]
2018-11-13T05:32:44.996+0800: 2.380: [GC concurrent-mark-start]
2018-11-13T05:32:45.013+0800: 2.396: [GC concurrent-mark-end, 0.0160995 secs]
2018-11-13T05:32:45.013+0800: 2.396: Application time: 0.0328728 seconds
2018-11-13T05:32:45.013+0800: 2.397: [GC remark 2018-11-13T05:32:45.013+0800: 2.397: 
[Finalize Marking, 0.0331140 secs] 2018-11-13T05:32:45.047+0800: 2.430: 
[GC ref-proc2018-11-13T05:32:45.047+0800: 2.430: 
[SoftReference, 0 refs, 0.0001959 secs]2018-11-13T05:32:45.047+0800: 2.430: 
[WeakReference, 0 refs, 0.0001148 secs]2018-11-13T05:32:45.047+0800: 2.430: 
[FinalReference, 0 refs, 0.0001158 secs]2018-11-13T05:32:45.047+0800: 2.430: 
[PhantomReference, 0 refs, 0 refs, 0.0002233 secs]2018-11-13T05:32:45.047+0800: 2.431: 
[JNI Weak Reference, 0.0000572 secs], 0.0008127 secs] 2018-11-13T05:32:45.047+0800: 2.431: [Unloading, 0.0078450 secs], 0.0442183 secs]
 [Times: user=0.36 sys=0.01, real=0.05 secs]
2018-11-13T05:32:45.058+0800: 2.441: Application time: 0.0001231 seconds
2018-11-13T05:32:45.058+0800: 2.442: [GC cleanup 35M->35M(10G), 0.0134109 secs]
 [Times: user=0.04 sys=0.00, real=0.01 secs]
2018-11-13T05:32:45.144+0800: 2.528: Application time: 0.0721037 seconds
2018-11-13T05:32:45.392+0800: 2.775: Application time: 0.2464978 seconds
2018-11-13T05:32:45.805+0800: 3.188: Application time: 0.4129881 seconds
2018-11-13T05:32:46.806+0800: 4.190: Application time: 1.0001058 seconds

下面是每条日志含义解析：
Application time： 应用程序时间
。。。