使用WinDbg分析.dump文件找出CPU占用与内存占用的问题根源

下载安装工具

http://msdn.microsoft.com/en-us/windows/hardware/gg463009.aspx

编写问题程序

using System;
using System.Collections.Generic;
using System.Threading;

namespace ConsoleApplication1
{
    class Program
    {
        private readonly List<Byte[]> _list = new List<byte[]>();

        void CostMemory()
        {
            for (var i = 0; i < 10; i++)
            {
                _list.Add(new byte[1024 * 1024 * 10]);
            }
        }

        void CostCpu()
        {
            new Thread(() =>
            {
                while (true)
                {
                    Console.WriteLine(_list.Count);
                }
            }).Start();
        }

        static void Main()
        {
            var o = new Program();
            o.CostMemory();
            o.CostCpu();

            Console.ReadKey(true);
        }
    }
}

执行程序并生成.dump文件

可使用操作系统的任务管理器，或adplus（adplus -hang -pn <进程名> -o <输出文件路径>）等工具，当然也可以直接使用WinDbg（使用invasive方式挂载进程后再使用.dump -m <输出文件路径>）

使用WinDbg进行分析

配置WinDbg

打开WinDbg，配置各路径

•   Symbol File Path（.pdb文件目录路径）

    C:\Users\Administrator\Desktop\ConsoleApplication1t;SRV*C:\symbolscache*http://msdl.microsoft.com/download/symbols

•   Source File Path（源代码文件目录路径）

    C:\Users\Administrator\Desktop\ConsoleApplication1

•   Image File Path（可执行文件目录路径）

    C:\Users\Administrator\Desktop\ConsoleApplication1

打开.dump文件

点击菜单File/Open Crash Dump

载入 SOS.dll扩展命令模块

  执行命令：.loadby sos clr

或者 .load C:\Windows\Microsoft.NET\Framework64\v4.0.30319\sos.dll

查找CPU占用过高的根源

执行 !threadpool 看是否是问题根源

0:000> !threadpool
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for clr.dll - 
PDB symbol for clr.dll not loaded
CPU utilization: 0%
Worker Thread: Total: 0 Running: 0 Idle: 0 MaxLimit: 0 MinLimit: 0
Work Request in Queue: 0
--------------------------------------
Number of Timers: 0
--------------------------------------
Completion Port Thread:Total: 0 Free: 0 MaxFree: 0 CurrentLimit: 0 MaxLimit: 1000 MinLimit: 0

"CPU utilization 0%"说明不是 ThreadPool 的问题

执行 !runaway 看线程的执行时间

0:000> !runaway
 User Mode Time
  Thread       Time
   4:714       0 days 0:00:00.358
   3:38c       0 days 0:00:00.000
   2:b94       0 days 0:00:00.000
   1:8bc       0 days 0:00:00.000
   0:8ec       0 days 0:00:00.000

执行时间最长的是线程4

执行 ~4 s 将当前线程切换到线程4

0:004> ~4 s
ntdll!ZwRequestWaitReplyPort+0xa:
00000000`776400da c3              ret

执行 !clrstack 显示当前线程的调用堆栈

0:004> !clrstack
OS Thread Id: 0xa68 (4)
Child SP         IP               Call Site
000000001c4feb68 00000000776400da [NDirectMethodFrameStandalone: 000000001c4feb68] System.IO.__ConsoleStream.WriteFile(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte*, Int32, Int32 ByRef, IntPtr)
000000001c4feb10 000007fef12c34a1 DomainNeutralILStubClass.IL_STUB_PInvoke(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte*, Int32, Int32 ByRef, IntPtr)*** WARNING: Unable to verify checksum for mscorlib.ni.dll
*** ERROR: Module load completed but symbols could not be loaded for mscorlib.ni.dll

000000001c4fec30 000007fef127cf9f System.IO.__ConsoleStream.WriteFileNative(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte[], Int32, Int32, Int32, Int32 ByRef)
000000001c4fec90 000007fef127cefa System.IO.__ConsoleStream.Write(Byte[], Int32, Int32)
000000001c4fecf0 000007fef1253a09 System.IO.StreamWriter.Flush(Boolean, Boolean)
000000001c4fed50 000007fef1a77b3d System.IO.TextWriter+SyncTextWriter.WriteLine(Int32)
000000001c4feda0 000007ff0016030d ConsoleApplication1.Program.<CostCpu>b__0()*** WARNING: Unable to verify checksum for ConsoleApplication1.exe
 [f:\Documents\Visual Studio 2012\Projects\2012test\ConsoleApplication1\Program.cs @ 23]
000000001c4fede0 000007fef121181c System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
000000001c4fee40 000007fef121172b System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
000000001c4fee90 000007fef12a6f2d System.Threading.ThreadHelper.ThreadStart()
000000001c4ff2e8 000007fef33a10b4 [GCFrame: 000000001c4ff2e8] 
000000001c4ff6d0 000007fef33a10b4 [DebuggerU2MCatchHandlerFrame: 000000001c4ff6d0] 

原因在CostCpu方法，在代码文件的第23行

查找内存占用过高的根源

执行 !eeheap -gc 查看托管堆的总信息

0:004> !eeheap -gc
Number of GC Heaps: 1
generation 0 starts at 0x0000000002a26220
generation 1 starts at 0x0000000002a21cf0
generation 2 starts at 0x0000000002a11000
ephemeral segment allocation context: none
         segment             begin         allocated  size
0000000002a10000  0000000002a11000  0000000002af2238  0xe1238(922168)
Large object heap starts at 0x0000000012a11000
         segment             begin         allocated  size
0000000012a10000  0000000012a11000  0000000018e171e0  0x64061e0(104882656)
Total Size:              Size: 0x64e7418 (105804824) bytes.
------------------------------
GC Heap Size:    Size: 0x64e7418 (105804824) bytes.

看来大对象堆占用最多内存

执行 !dumpheap -min 200 -stat 获取占用堆内存的各对象的统计信息

0:004> !dumpheap -min 200 -stat
total 0 objects
Statistics:
              MT    Count    TotalSize Class Name
000007fef1369750        1          216 System.AppDomain
000007fef13745d0        2          432 System.Globalization.NumberFormatInfo
000007fef1373148        1          432 System.Collections.Generic.Dictionary`2+Entry[[System.Type, mscorlib],[System.Security.Policy.EvidenceTypeDescriptor, mscorlib]][]
000007fef136b7b0        2         1056 System.Globalization.CultureData
000007fef136c7e8        2         1128 System.Int32[]
000007fef136b328        3         3256 System.Char[]
000007fef136d110        1         4752 System.Collections.Generic.Dictionary`2+Entry[[System.String, mscorlib],[System.String, mscorlib]][]
0000000000664be0       22        31784      Free
000007fef136ae78        5        34072 System.Object[]
000007fef1370bc0       11    104858384 System.Byte[]
Total 50 objects

Byte数组类型占用最多内存

执行 !dumpheap -type Byte[] -min 200 看各Byte数组占用堆内存的详细信息

0:004> !dumpheap -type Byte[] -min 200
         Address               MT     Size
0000000002a25290 000007fef1370bc0      544     
0000000012a17048 000007fef1370bc0 10485784     
0000000013417060 000007fef1370bc0 10485784     
0000000013e17078 000007fef1370bc0 10485784     
00000000148170a8 000007fef1370bc0 10485784     
00000000152170d8 000007fef1370bc0 10485784     
0000000015c17108 000007fef1370bc0 10485784     
0000000016617138 000007fef1370bc0 10485784     
0000000017017168 000007fef1370bc0 10485784     
0000000017a17198 000007fef1370bc0 10485784     
00000000184171c8 000007fef1370bc0 10485784     
total 0 objects
Statistics:
              MT    Count    TotalSize Class Name
000007fef1370bc0       11    104858384 System.Byte[]
Total 11 objects

随便挑一个大的对象出来

执行 !gcroot <Byte数组对象地址> 看对象引用关系

0:004> !gcroot 0000000012a17048 
Note: Roots found on stacks may be false positives. Run "!help gcroot" for
more info.
Scan Thread 0 OSTHread a54
RSP:54e958:Root:  0000000002a21e60(System.Threading.ThreadStart)->
  0000000002a21cb0(ConsoleApplication1.Program)->
  0000000002a21cc8(System.Collections.Generic.List`1[[System.Byte[], mscorlib]])->
  0000000002a21dc0(System.Byte[][])->
  0000000012a17048(System.Byte[])

跟踪到Program类的List类型成员了

执行 !do <Program对象地址> 查看对象的详细信息

0:004> !do 0000000002a21cb0
Name:        ConsoleApplication1.Program
MethodTable: 000007ff00044140
EEClass:     000007ff00152350
Size:        24(0x18) bytes
File:        C:\Users\Administrator\Desktop\ConsoleApplication1\ConsoleApplication1.exe
Fields:
              MT    Field   Offset                 Type VT     Attr            Value Name
000007ff00032600  4000001        8 ...yte[], mscorlib]]  0 instance 0000000002a21cc8 _list

就是_list了。有源代码文件的话这一步其实就可以略过，就算没有源代码文件也可以利用!dumpdomain与!SaveModule命令导出模块，然后使用Reflector进行反编译。

资源

  Windbg 分析 Dump File 简单演示(转) 
SOS.dll（SOS 调试扩展）
《WinDbg用法详解》学习摘录
WinDBG命令概览

使用Windbg调试.Net应用程序

用Windbg调试.NET程序的资源泄漏