NMI watchdog: BUG: soft lockup

    今天测试环境一虚拟机运行中突然报错，，， 没见过的内核报错，于是google一番。

　 系统日志：

Nov 22 14:32:30 dev-30 kernel: NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [kworker/0:0:31524]
Nov 22 14:32:30 dev-30 kernel: Modules linked in: binfmt_misc ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd sg pcspkr vmw_balloon joydev vmw_vmci parport_pc parport shpchp i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod drm_kms_helper crc_t10dif crct10dif_generic
Nov 22 14:32:31 dev-30 kernel: syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw vmxnet3 i2c_core vmw_pvscsi floppy
Nov 22 14:32:31 dev-30 kernel: CPU: 0 PID: 31524 Comm: kworker/0:0 Kdump: loaded Not tainted 3.10.0-862.11.6.el7.x86_64 #1
Nov 22 14:32:31 dev-30 kernel: Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 06/22/2012
Nov 22 14:32:31 dev-30 kernel: Workqueue: events_freezable vmballoon_work [vmw_balloon]
Nov 22 14:32:31 dev-30 kernel: task: ffff9041caaf1fa0 ti: ffff90418f3ac000 task.ti: ffff90418f3ac000
Nov 22 14:32:31 dev-30 kernel: RIP: 0010:[<ffffffffc02f7297>]  [<ffffffffc02f7297>] vmballoon_lock_page+0x57/0x150 [vmw_balloon]
Nov 22 14:32:31 dev-30 kernel: RSP: 0018:ffff90418f3afda0  EFLAGS: 00000246
Nov 22 14:32:31 dev-30 kernel: RAX: 0000000000000000 RBX: 0000000200000000 RCX: 0000000000000002
Nov 22 14:32:31 dev-30 kernel: RDX: 0000000000005670 RSI: 0000000000000000 RDI: ffffffffc02fa438
Nov 22 14:32:31 dev-30 kernel: RBP: ffff90418f3afdc0 R08: ffffffffac47600f R09: 0000000000280383
Nov 22 14:32:31 dev-30 kernel: R10: ffffe6bbca00e100 R11: fffffffffffffffa R12: ffff9041fffcf008
Nov 22 14:32:31 dev-30 kernel: R13: ffff9041caaf1fa0 R14: ffff9041caaf1fa0 R15: ffff9041caaf1fa0
Nov 22 14:32:31 dev-30 kernel: FS:  0000000000000000(0000) GS:ffff9041ffc00000(0000) knlGS:0000000000000000
Nov 22 14:32:31 dev-30 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 22 14:32:31 dev-30 kernel: CR2: 00007fed8de1a8d0 CR3: 00000004177ec000 CR4: 00000000000407f0
Nov 22 14:32:31 dev-30 kernel: Call Trace:
Nov 22 14:32:31 dev-30 kernel: [<ffffffffc02f80d5>] vmballoon_work+0x5a5/0x6ff [vmw_balloon]
Nov 22 14:32:31 dev-30 kernel: [<ffffffffabab613f>] process_one_work+0x17f/0x440
Nov 22 14:32:31 dev-30 kernel: [<ffffffffabab71d6>] worker_thread+0x126/0x3c0
Nov 22 14:32:31 dev-30 kernel: [<ffffffffabab70b0>] ? manage_workers.isra.24+0x2a0/0x2a0
Nov 22 14:32:31 dev-30 kernel: [<ffffffffababdf21>] kthread+0xd1/0xe0
Nov 22 14:32:31 dev-30 kernel: [<ffffffffababde50>] ? insert_kthread_work+0x40/0x40
Nov 22 14:32:31 dev-30 kernel: [<ffffffffac1255f7>] ret_from_fork_nospec_begin+0x21/0x21
Nov 22 14:32:31 dev-30 kernel: [<ffffffffababde50>] ? insert_kthread_work+0x40/0x40
Nov 22 14:32:31 dev-30 kernel: Code: 39 c1 0f 85 cc 00 00 00 bb 6f 6d 6c 45 48 89 cf 41 83 84 24 c4 00 00 00 01 89 d8 31 f6 ba 70 56 00 00 b9 02 00 00 00 4c 89 cb ed <48> 85 c0 49 89 c6 89 1f 0f 84 9b 00 00 00 48 83 f8 07 0f 84 81

知识点：　　

watchdog： watchdog是为了保证系统正常运行，或者从死循环，死锁等一场状态退出的一种机制。

https://blog.csdn.net/whatday/article/details/73770736

http://oenhan.com/kernel-deadlock-check

soft lockup CPU死锁的问题

https://blog.csdn.net/sunny05296/article/details/82858071