[BUG] Kernel Panic with message "kernel BUG at net/sunrpc/sched.c:655!"

Kernel Panic with message "kernel BUG at net/sunrpc/sched.c:655!"

Issue:

System panic with the following message:

Jul  9 02:33:20 server kernel: ------------[ cut here ]------------
Jul  9 02:33:20 server kernel: kernel BUG at net/sunrpc/sched.c:655!
Jul  9 02:33:20 server kernel: invalid opcode: 0000 [#1] SMP
Jul  9 02:33:20 server kernel: last sysfs file:
/sys/devices/system/node/node0/meminfo
Jul  9 02:33:20 server kernel: ------------[ cut here ]------------
Jul  9 02:33:20 server kernel: WARNING: at lib/list_debug.c:48
list_del+0x6e/0xa0() (Tainted: G---------------  T)
Jul  9 02:33:20 server kernel: Hardware name: VMware Virtual Platform
Jul  9 02:33:20 server kernel: list_del corruption. prev->next should
be ffff880b93d0a410, but was ffff880b93d0a418
        

Environment:

• Red Hat Enterprise Linux 6.3
• VMWare ESX virtual machine

Resolution:

This issue is still under investigation at this time. However, you may work around this issue by disabling the NMI watchdog:

Temporary:

To disable the NMI watchdog in run-time, write "0" to /proc/sys/kernel/nmi_watchdog:

# echo 0 > /proc/sys/kernel/nmi_watchdog

Permanent:

Add argument nmi_watchdog=0 in /boot/grub/grub.conf file, then reboot the system:

# vim /boot/grub/grub.conf

title Red Hat Enterprise Linux Server (2.6.18-308.11.1.el5)
    root (hd0,0)
    kernel /vmlinuz-2.6.18-308.11.1.el5 ro root=/dev/VolGroup00/LogVol00 \
       rhgb quiet crashkernel=128M@16M nmi_watchdog=0
    initrd /initrd-2.6.18-308.11.1.el5.img
       

***Note: The example kernel line from above has been wrapped for formatting reasons.

OR

Add a line "sysctl -w kernel.nmi_watchdog=0" in /etc/sysctl.conf file, then reboot the system:

# vim /etc/sysctl.conf
sysctl -w kernel.nmi_watchdog=0
       

Root Cause:

This is a bug in the RHEL 6.3 kernel related with RPC calls.

Back to top...