coredump是一个包含程序意外终止时的内存快照的文件。它可以用于事后调试,以了解崩溃发生的原因以及其中涉及的变量。通过GOTRACEBACK,Go提供了一个环境变量来控制程序崩溃时产生的输出。这个变量可以强制生成coredump,便于调试。
让golang程序生成core文件
a. ulimit -c unlimited 修改 core 文件的大小
b. 环境变量export GOTRACEBACK=crash 说明golang程序产生coredump
可以使用gdb对coredump进行查看,delve对golang的兼容更好
1. 编译delve
# git clone https://github.com/derekparker/delve.git
# cd delve/cmd/dlv/
# go build
2. 将 dlv 拷贝到线上有coredump的服务器。
分析coredump
./dlv core ./engine core.1871450_engine --check-go-version=false
3. 输入help可以查看Delve支持的命令
Type 'help' for list of commands.
(dlv) help
The following commands are available:
Running the program:
call ------------------------ Resumes process, injecting a function call (EXPERIMENTAL!!!)
continue (alias: c) --------- Run until breakpoint or program termination.
next (alias: n) ------------- Step over to next source line.
rebuild --------------------- Rebuild the target executable and restarts it. It does not work if the executable was not built by delve.
restart (alias: r) ---------- Restart process.
rev ------------------------- Reverses the execution of the target program for the command specified.
rewind (alias: rw) ---------- Run backwards until breakpoint or program termination.
step (alias: s) ------------- Single step through program.
step-instruction (alias: si) Single step a single cpu instruction.
stepout (alias: so) --------- Step out of the current function.
Manipulating breakpoints:
break (alias: b) ------- Sets a breakpoint.
breakpoints (alias: bp) Print out info for active breakpoints.
clear ------------------ Deletes breakpoint.
clearall --------------- Deletes multiple breakpoints.
condition (alias: cond) Set breakpoint condition.
on --------------------- Executes a command when a breakpoint is hit.
toggle ----------------- Toggles on or off a breakpoint.
trace (alias: t) ------- Set tracepoint.
watch ------------------ Set watchpoint.
Viewing program variables and memory:
args ----------------- Print function arguments.
display -------------- Print value of an expression every time the program stops.
examinemem (alias: x) Examine memory:
locals --------------- Print local variables.
print (alias: p) ----- Evaluate an expression.
regs ----------------- Print contents of CPU registers.
set ------------------ Changes the value of a variable.
vars ----------------- Print package variables.
whatis --------------- Prints type of an expression.
Listing and switching between threads and goroutines:
goroutine (alias: gr) -- Shows or changes current goroutine
goroutines (alias: grs) List program goroutines.
thread (alias: tr) ----- Switch to the specified thread.
threads ---------------- Print out info for every traced thread.
Viewing the call stack and selecting frames:
deferred --------- Executes command in the context of a deferred call.
down ------------- Move the current frame down.
frame ------------ Set the current frame, or execute command on a different frame.
stack (alias: bt) Print stack trace.
up --------------- Move the current frame up.
Other commands:
check (alias: checkpoint) ----------- Creates a checkpoint at the current position.
checkpoints ------------------------- Print out info for existing checkpoints.
clear-checkpoint (alias: clearcheck) Deletes checkpoint.
config ------------------------------ Changes configuration parameters.
disassemble (alias: disass) --------- Disassembler.
dump -------------------------------- Creates a core dump from the current process state
edit (alias: ed) -------------------- Open where you are in $DELVE_EDITOR or $EDITOR
exit (alias: quit | q) -------------- Exit the debugger.
funcs ------------------------------- Print list of functions.
help (alias: h) --------------------- Prints the help message.
libraries --------------------------- List loaded dynamic libraries
list (alias: ls | l) ---------------- Show source code.
source ------------------------------ Executes a file containing a list of delve commands
sources ----------------------------- Print list of source files.
types ------------------------------- Print list of types
4. goroutine 显示或修改当前goroutine
(dlv) goroutine
Thread 1871459 at /usr/local/go/src/runtime/sys_linux_amd64.s:165
Goroutine 2072788:
Runtime: /usr/local/go/src/runtime/sys_linux_amd64.s:165 runtime.raise (0x474461)
User: /data/home/XXX/XXX/XXX.go:104 XXXXXX/XXX/XXX/XXX (0x14f454d)
Go:/data/home/XXX/XXX/XXXXXXX/XXX/XXX/XXX@v0.0.0-xxx/xxx.go:281 XXXXXX/XXX/XXX/XXX (0xc9c9e7)
Start: /data/home/XXX/XXX/XXXXXXX/XXX/XXX/XXX@v0.0.0-xxx/xxx.go:122 XXXXXX/XXX/XXX/XXX (0xc9be20)
可以看到core发生的位置