kernel: WARNING: at net/sched/sch_generic.c dev_watchdog()

Issue:

Network driver backtraces found in /var/log/messages, server loses network connectivity.
Example backtrace below:

WARNING: at net/sched/sch_generic.c:261 dev_watchdog+0x26d/0x280() (Not tainted)
Hardware name:
NETDEV WATCHDOG: eth0 (e100): transmit queue 0 timed out
Modules linked in: iptable_nat nf_nat (edited...)
Pid: 0, comm: swapper Not tainted 2.6.32-71.el6.x86_64 #1
Call Trace:
[] warn_slowpath_common+0x87/0xc0
[] warn_slowpath_fmt+0x46/0x50
[] dev_watchdog+0x26d/0x280
[] ? mod_timer+0x146/0x230
[] ? dev_watchdog+0x0/0x280
[] run_timer_softirq+0x197/0x340
[] ? tick_sched_timer+0x0/0xc0
[] ? lapic_next_event+0x1d/0x30
[] __do_softirq+0xb7/0x1e0
[] ? hrtimer_interrupt+0x140/0x250
[] call_softirq+0x1c/0x30
[] do_softirq+0x65/0xa0
[] irq_exit+0x85/0x90
[] smp_apic_timer_interrupt+0x71/0x9c
[] apic_timer_interrupt+0x13/0x20
[] ? mwait_idle+0x71/0xd0
[] ? atomic_notifier_call_chain+0x1a/0x20
[] cpu_idle+0xb6/0x110
[] start_secondary+0x1fc/0x23f

Environment:

  • Red Hat Enterprise Linux 6
  • e1000, e1000e, ixgbe, igb, bnx2, r8169 network cards (possibly others)

Resolution:

There is no resolution to this issue yet. Please contact Red Hat Support for assistance if you believe you have encountered this issue.

Ensure that all Network Interface Card (NIC) firmware is updated and try an updated kernel with a newer driver for your hardware.

For the ones that the workaround does not work, a BIOS update might be needed.

Possible Workarounds:

  • Try turning off tx and rx checksum with following command if that makes any difference:

    # ethtool -K eth0 rx off
    # ethtool -K eth0 tx off

  • Disable tso for the NIC:

    # ethtool -K eth tso off

  • Configure bonding to reduce the impact of the problem.

  • Try the following kernel parameters one at a time to see if issue is resolved:

    pcie_aspm=off
    hpet=disable
    intel_iommu=off

Root Cause:

The root cause of this issue is not currently known.

Back to top...

 

+ Recent posts