戴尔r510服务器修复,DELL R510服务器宕机案例(1)

我的博客已迁移到xdoujiang.com请去那边和我交流

早上发现有台机器宕机了 查看日志发现

May 12 05:01:09 10.1.1.1 [4269014.583472] BUG: scheduling while atomic: swapper/0/0x10000100

May 12 05:01:09 10.1.1.1  0000000000015780

May 12 05:01:09 10.1.1.1  0000000000015780

May 12 05:01:09 10.1.1.1  0000000000015780

May 12 05:01:09 10.1.1.1

May 12 05:01:09 10.1.1.1 [4269014.961827] <0>

May 12 05:01:09 10.1.1.1  ffff8802615e9900

May 12 05:01:09 10.1.1.1  ffffffff8126315f

May 12 05:01:09 10.1.1.1  ffff880829b7cc00

May 12 05:01:09 10.1.1.1  ffff880829558a00

May 12 05:01:09 10.1.1.1

May 12 05:01:09 10.1.1.1 [4269014.969988] Call Trace:

May 12 05:01:09 10.1.1.1 [4269014.972630] 

May 12 05:01:09 10.1.1.1

May 12 05:01:09 10.1.1.1 [4269014.974944]  [] ? sch_direct_xmit+0x7f/0x14c

May 12 05:01:09 10.1.1.1 [4269014.980981]  [] ? rebalance_domains+0x180/0x480

May 12 05:01:09 10.1.1.1 [4269014.987276]  [] ? run_rebalance_domains+0x3b/0xc7

May 12 05:01:09 10.1.1.1 [4269014.993746]  [] ? __do_softirq+0xdd/0x1a6

May 12 05:01:09 10.1.1.1 [4269014.999521]  [] ? lapic_next_event+0x18/0x1d

May 12 05:01:09 10.1.1.1 [4269015.005558]  [] ? call_softirq+0x1c/0x30

May 12 05:01:09 10.1.1.1 [4269015.011245]  [] ? do_softirq+0x3f/0x7c

May 12 05:01:09 10.1.1.1 [4269015.016759]  [] ? irq_exit+0x36/0x76

May 12 05:01:09 10.1.1.1 [4269015.022099]  [] ? smp_apic_timer_interrupt+0x87/0x95

May 12 05:01:09 10.1.1.1 [4269015.028830]  [] ? apic_timer_interrupt+0x13/0x20

May 12 05:01:09 10.1.1.1 [4269015.035209] 

May 12 05:01:09 10.1.1.1

May 12 05:01:09 10.1.1.1 [4269015.037526]  [] ? acpi_idle_enter_bm+0x27d/0x2af [processor]

May 12 05:01:09 10.1.1.1 [4269015.044968]  [] ? acpi_idle_enter_bm+0x276/0x2af [processor]

May 12 05:01:09 10.1.1.1 [4269015.052411]  [] ? cpuidle_idle_call+0x94/0xee

May 12 05:01:09 10.1.1.1 [4269015.058534]  [] ? cpu_idle+0xa2/0xda

May 12 05:01:09 10.1.1.1  [] find_busiest_group+0x466/0xa2e

May 12 05:01:09 10.1.1.1 [4269015.090513]  RSP

May 12 05:01:09 10.1.1.1 [4269015.094550] ---[ end trace 2ff8aecaebc3aefc ]---

May 12 05:01:09 10.1.1.1 [4269015.094557] BUG: scheduling while atomic: swapper/0/0x10000100

May 12 05:01:09 10.1.1.1 [4269015.094562] Modules linked in: ipmi_devintf ipmi_si ipmi_msghandler netconsole configfs usbhid hid ohci_hcd loop snd_pcm snd_timer snd soundcore snd_page_alloc evdev dcdbas psmouse pcspkr iTCO_wdt iTCO_vendor_support serio_raw wmi power_meter button processor ext3 jbd mbcache uhci_hcd ses sd_mod crc_t10dif enclosure megaraid_sas ehci_hcd scsi_mod thermal thermal_sys usbcore nls_base bnx2 [last unloaded: scsi_wait_scan]

May 12 05:01:09 10.1.1.1 [4269015.094592] CPU 1:

May 12 05:01:09 10.1.1.1 [4269015.094594] Modules linked in: ipmi_devintf ipmi_si ipmi_msghandler netconsole configfs usbhid hid ohci_hcd loop snd_pcm snd_timer snd soundcore snd_page_alloc evdev dcdbas psmouse pcspkr iTCO_wdt iTCO_vendor_support serio_raw wmi power_meter button processor ext3 jbd mbcache uhci_hcd ses sd_mod crc_t10dif enclosure megaraid_sas ehci_hcd scsi_mod thermal thermal_sys usbcore nls_base bnx2 [last unloaded: scsi_wait_scan]

May 12 05:01:09 10.1.1.1 [4269015.094623] Pid: 0, comm: swapper Tainted: G      D    2.6.32-5-amd64 #1 PowerEdge R510

May 12 05:01:09 10.1.1.1 [4269015.094626] RIP: 0010:[]  [] acpi_idle_enter_bm+0x27d/0x2af [processor]

May 12 05:01:09 10.1.1.1 [4269015.094636] RSP: 0018:ffff88082ccadea8  EFLAGS: 00000202

May 12 05:01:09 10.1.1.1 [4269015.094639] RAX: 0000000000010360 RBX: ffff88082abe9530 RCX: 00000000000f4240

May 12 05:01:09 10.1.1.1 [4269015.094642] RDX: ffff88001dc20000 RSI: 0000000000000000 RDI: 00000000003c8aab

May 12 05:01:09 10.1.1.1 [4269015.094646] RBP: ffffffff8101166e R08: 0000000000000f65 R09: 13dd46c52c8402dd

May 12 05:01:09 10.1.1.1 [4269015.094649] R10: 000000024c0a03b2 R11: 0000000000000001 R12: ffff88001dc35780

May 12 05:01:09 10.1.1.1 [4269015.094652] R13: ffffffff814634c0 R14: ffffffff81041ade R15: ffff880829608748

May 12 05:01:09 10.1.1.1 [4269015.094656] FS:  0000000000000000(0000) GS:ffff88001dc20000(0000) knlGS:0000000000000000

May 12 05:01:09 10.1.1.1 [4269015.094660] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b

May 12 05:01:09 10.1.1.1 [4269015.094663] CR2: 00007f9edc6e34a8 CR3: 0000000001001000 CR4: 00000000000006e0

May 12 05:01:09 10.1.1.1 [4269015.094665] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000

May 12 05:01:09 10.1.1.1 [4269015.094669] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400

May 12 05:01:09 10.1.1.1 [4269015.094671] Call Trace:

May 12 05:01:09 10.1.1.1 [4269015.094678]  [] ? acpi_idle_enter_bm+0x276/0x2af [processor]

May 12 05:01:09 10.1.1.1 [4269015.094685]  [] ? cpuidle_idle_call+0x94/0xee

May 12 05:01:09 10.1.1.1 [4269015.094691]  [] ? cpu_idle+0xa2/0xda

May 12 05:01:09 10.1.1.1 [4269015.095372] divide error: 0000 [#2] SMP

May 12 05:01:09 10.1.1.1 [4269015.095375] last sysfs file: /sys/devices/pci0000:00/0000:00:03.0/0000:02:00.0/host0/scsi_host/host0/proc_name

May 12 05:01:09 10.1.1.1 [4269015.095379] CPU 1

May 12 05:01:09 10.1.1.1 [4269015.095380] Modules linked in: ipmi_devintf ipmi_si ipmi_msghandler netconsole configfs usbhid hid ohci_hcd loop snd_pcm snd_timer snd soundcore snd_page_alloc evdev dcdbas psmouse pcspkr iTCO_wdt iTCO_vendor_support serio_raw wmi power_meter button processor ext3 jbd mbcache uhci_hcd ses sd_mod crc_t10dif enclosure megaraid_sas ehci_hcd scsi_mod thermal thermal_sys usbcore nls_base bnx2 [last unloaded: scsi_wait_scan]

May 12 05:01:09 10.1.1.1 [4269015.095411] Pid: 0, comm: swapper Tainted: G      D    2.6.32-5-amd64 #1 PowerEdge R510

May 12 05:01:09 10.1.1.1 [4269015.095414] RIP: 0010:[]  [] find_busiest_group+0x466/0xa2e

May 12 05:01:09 10.1.1.1 [4269015.095422] RSP: 0018:ffff88001dc23c30  EFLAGS: 00010206

May 12 05:01:09 10.1.1.1 [4269015.095425] RAX: 0000000000003c00 RBX: 000000000000000f RCX: 0000000000000000

May 12 05:01:10 10.1.1.1 [4269015.095428] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000200

May 12 05:01:10 10.1.1.1 [4269015.095431] RBP: 0000000000015780 R08: 0000000000000000 R09: ffff88001dc4fc40

May 12 05:01:10 10.1.1.1 [4269015.095434] R10: 7fffffffffffffff R11: ffffffff813ce9f4 R12: ffff88001dc2fb30

May 12 05:01:10 10.1.1.1 [4269015.095437] R13: 000000000000000f R14: 0000000000015780 R15: ffff88001dc4fbf0

May 12 05:01:10 10.1.1.1 [4269015.095441] FS:  0000000000000000(0000) GS:ffff88001dc20000(0000) knlGS:0000000000000000

May 12 05:01:10 10.1.1.1 [4269015.095444] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b

May 12 05:01:10 10.1.1.1 [4269015.095447] CR2: 0000000001ca30d0 CR3: 0000000819b21000 CR4: 00000000000006e0

May 12 05:01:10 10.1.1.1 [4269015.095450] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000

May 12 05:01:10 10.1.1.1 [4269015.095453] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400

May 12 05:01:10 10.1.1.1 [4269015.095457] Process swapper (pid: 0, threadinfo ffff88082ccac000, task ffff88082cc83880)

May 12 05:01:10 10.1.1.1 [4269015.095459] Stack:

May 12 05:01:10 10.1.1.1 [4269015.095461]  ffff88081c199498 0000000000015788 0000000000015780 0000000000000008

May 12 05:01:10 10.1.1.1 [4269015.095465] <0> 0000000000015780 0000000000015780 0000000000015780 0000000000015780

May 12 05:01:10 10.1.1.1 [4269015.095469] <0> ffff88081c199600 0000000000000082 00000000000000c1 ffff88001dc357e8

May 12 05:01:10 10.1.1.1 [4269015.095473] Call Trace:

May 12 05:01:10 10.1.1.1 [4269015.095475] 

May 12 05:01:10 10.1.1.1 [4269015.095482]  [] ? rebalance_domains+0x180/0x480

May 12 05:01:10 10.1.1.1 [4269015.095488]  [] ? native_sched_clock+0x2e/0x66

May 12 05:01:10 10.1.1.1 [4269015.095494]  [] ? run_rebalance_domains+0x3b/0xc7

May 12 05:01:10 10.1.1.1 [4269015.095499]  [] ? __do_softirq+0xdd/0x1a6

May 12 05:01:10 10.1.1.1 [4269015.095510]  [] ? lapic_next_event+0x18/0x1d

May 12 05:01:10 10.1.1.1 [4269015.095515]  [] ? call_softirq+0x1c/0x30

May 12 05:01:10 10.1.1.1 [4269015.095519]  [] ? do_softirq+0x3f/0x7c

May 12 05:01:10 10.1.1.1 [4269015.095524]  [] ? irq_exit+0x36/0x76

May 12 05:01:10 10.1.1.1 [4269015.095528]  [] ? smp_apic_timer_interrupt+0x87/0x95

May 12 05:01:10 10.1.1.1 [4269015.095533]  [] ? apic_timer_interrupt+0x13/0x20

May 12 05:01:10 10.1.1.1 [4269015.095535] 

May 12 05:01:10 10.1.1.1 [4269015.095542]  [] ? acpi_idle_enter_bm+0x27d/0x2af [processor]

May 12 05:01:10 10.1.1.1 [4269015.095548]  [] ? acpi_idle_enter_bm+0x276/0x2af [processor]

May 12 05:01:10 10.1.1.1 [4269015.095554]  [] ? cpuidle_idle_call+0x94/0xee

May 12 05:01:10 10.1.1.1 [4269015.095559]  [] ? cpu_idle+0xa2/0xda

May 12 05:01:10 10.1.1.1 [4269015.095562] Code: 01 00 00 00 74 13 48 8b 94 24 f0 01 00 00 c7 02 00 00 00 00 e9 91 00 00 00 41 8b 77 08 48 8b 84 24 78 01 00 00 31 d2 48 c1 e0 0a <48> f7 f6 48 8b b4 24 80 01 00 00 48 89 84 24 70 01 00 00 31 c0

May 12 05:01:10 10.1.1.1 [4269015.095589] RIP  [] find_busiest_group+0x466/0xa2e

May 12 05:01:10 10.1.1.1 [4269015.095594]  RSP

May 12 05:01:10 10.1.1.1 [4269015.095598] ---[ end trace 2ff8aecaebc3aefd ]---

May 12 05:01:10 10.1.1.1 [4269015.095604] divide error: 0000 [#3]

May 12 05:01:10 10.1.1.1 [4269015.095608] Kernel panic - not syncing: Fatal exception in interrupt

May 12 05:01:10 10.1.1.1 [4269015.095612] SMP

May 12 05:01:10 10.1.1.1 [4269015.095615] last sysfs file: /sys/devices/pci0000:00/0000:00:03.0/0000:02:00.0/host0/scsi_host/host0/proc_name

May 12 05:01:10 10.1.1.1 [4269015.095621] Pid: 0, comm: swapper Tainted: G      D    2.6.32-5-amd64 #1

May 12 05:01:10 10.1.1.1 [4269015.095626] CPU 3 Call Trace:

May 12 05:01:10 10.1.1.1 [4269015.095629]

May 12 05:01:10 10.1.1.1 [4269015.095631] 

May 12 05:01:10 10.1.1.1 [4269015.095634] Modules linked in: ipmi_devintf [] ? panic+0x86/0x143

May 12 05:01:10 10.1.1.1 [4269015.095644]  ipmi_si ipmi_msghandler [] ? show_registers+0x1e9/0x1f8

May 12 05:01:10 10.1.1.1 [4269015.095654]  netconsole configfs [] ? down_trylock+0x28/0x2e

May 12 05:01:10 10.1.1.1 [4269015.095664]  usbhid hid [] ? console_unblank+0x16/0x60

May 12 05:01:10 10.1.1.1 [4269015.095673]  ohci_hcd loop [] ? oops_end+0xa7/0xb4

May 12 05:01:10 10.1.1.1 [4269015.095682]  snd_pcm snd_timer [] ? do_divide_error+0x85/0x8f

May 12 05:01:10 10.1.1.1 [4269015.095690]  snd soundcore [] ? find_busiest_group+0x466/0xa2e

May 12 05:01:10 10.1.1.1 [4269015.095699]  snd_page_alloc evdev [] ? select_task_rq_fair+0x472/0x836

May 12 05:01:10 10.1.1.1 [4269015.095709]  dcdbas psmouse [] ? select_task_rq_fair+0x1fa/0x836

May 12 05:01:10 10.1.1.1 [4269015.095718]  pcspkr iTCO_wdt [] ? divide_error+0x1b/0x20

May 12 05:01:10 10.1.1.1 [4269015.095726]  iTCO_vendor_support serio_raw [] ? find_busiest_group+0x466/0xa2e

May 12 05:01:10 10.1.1.1 [4269015.095736]  wmi power_meter button [] ? rebalance_domains+0x180/0x480

May 12 05:01:10 10.1.1.1 [4269015.095745]  processor ext3 [] ? native_sched_clock+0x2e/0x66

May 12 05:01:10 10.1.1.1 [4269015.095754]  jbd mbcache [] ? run_rebalance_domains+0x3b/0xc7

May 12 05:01:10 10.1.1.1 [4269015.095764]  uhci_hcd ses [] ? __do_softirq+0xdd/0x1a6

May 12 05:01:10 10.1.1.1 [4269015.095772]  sd_mod crc_t10dif [] ? lapic_next_event+0x18/0x1d

May 12 05:01:10 10.1.1.1 [4269015.095781]  enclosure megaraid_sas [] ? call_softirq+0x1c/0x30

May 12 05:01:10 10.1.1.1 [4269015.095789]  ehci_hcd scsi_mod [] ? do_softirq+0x3f/0x7c

May 12 05:01:10 10.1.1.1 [4269015.095797]  thermal thermal_sys [] ? irq_exit+0x36/0x76

May 12 05:01:10 10.1.1.1 [4269015.095806]  usbcore nls_base [] ? smp_apic_timer_interrupt+0x87/0x95

May 12 05:01:10 10.1.1.1 [4269015.095815]  bnx2 [last unloaded: scsi_wait_scan] [] ? apic_timer_interrupt+0x13/0x20

May 12 05:01:10 10.1.1.1 [4269015.095824]

May 12 05:01:10 10.1.1.1 [4269015.095826] 

May 12 05:01:10 10.1.1.1 [4269015.095831] Pid: 2887, comm: Thttpd Tainted: G      D    2.6.32-5-amd64 #1 PowerEdge R510

May 12 05:01:10 10.1.1.1 [4269015.095837] RIP: 0010:[]  [] ? acpi_idle_enter_bm+0x27d/0x2af [processor]

May 12 05:01:10 10.1.1.1 [4269015.095852]  [] find_busiest_group+0x466/0xa2e

May 12 05:01:10 10.1.1.1 [4269015.095857] RSP: 0018:ffff88081bbc3b88  EFLAGS: 00010006

May 12 05:01:10 10.1.1.1 [4269015.095865]  [] ? acpi_idle_enter_bm+0x276/0x2af [processor]

May 12 05:01:10 10.1.1.1 [4269015.095870] RAX: 0000000000003c00 RBX: 000000000000000f RCX: 0000000000000000

May 12 05:01:10 10.1.1.1 [4269015.095877] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000200

May 12 05:01:10 10.1.1.1 [4269015.095884]  [] ? cpuidle_idle_call+0x94/0xee

May 12 05:01:10 10.1.1.1 [4269015.095889] RBP: 0000000000015780 R08: 0000000000000000 R09: ffff88001dc4fc40

May 12 05:01:10 10.1.1.1 [4269015.095895] R10: 0000000000000064 R11: 0000000000000246 R12: ffff88001dc6fb30

May 12 05:01:10 10.1.1.1 [4269015.095903]  [] ? cpu_idle+0xa2/0xda

May 12 05:01:10 10.1.1.1 [4269015.095908] R13: 000000000000000f R14: 0000000000015780 R15: ffff88001dc4fbf0

May 12 05:01:10 10.1.1.1 [4269015.095913] FS:  00007f8e9fc786e0(0000) GS:ffff88001dc60000(0000) knlGS:0000000000000000

May 12 05:01:10 10.1.1.1 [4269015.095916] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b

May 12 05:01:10 10.1.1.1 [4269015.095919] CR2: 00007f07e2a15210 CR3: 000000081bbfb000 CR4: 00000000000006e0

May 12 05:01:10 10.1.1.1 [4269015.095922] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000

May 12 05:01:10 10.1.1.1 [4269015.095925] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400

May 12 05:01:10 10.1.1.1 [4269015.095929] Process Thttpd (pid: 2887, threadinfo ffff88081bbc2000, task ffff8808296d1530)

May 12 05:01:10 10.1.1.1 [4269015.095931] Stack:

May 12 05:01:10 10.1.1.1 [4269015.095933]  0000000029675c00 0000000000015788 0000000000015780 0000000000000008

May 12 05:01:10 10.1.1.1 [4269015.095937] <0> 0000000000015780 0000000000015780 0000000000015780 ffff880829b7cd00

May 12 05:01:10 10.1.1.1 [4269015.095941] <0> ffff880828f00000 ffff880829b7cd00 ffffffffa000c6e0 ffffffff812501be

May 12 05:01:10 10.1.1.1 [4269015.095946] Call Trace:

May 12 05:01:10 10.1.1.1 [4269015.095955]  [] ? dev_hard_start_xmit+0x211/0x2db

May 12 05:01:10 10.1.1.1 [4269015.095961]  [] ? schedule+0x2b3/0x7b4

May 12 05:01:10 10.1.1.1 [4269015.095968]  [] ? __mod_timer+0x141/0x153

May 12 05:01:10 10.1.1.1 [4269015.095973]  [] ? __mod_timer+0x141/0x153

May 12 05:01:10 10.1.1.1 [4269015.095978]  [] ? try_to_del_timer_sync+0x63/0x6c

May 12 05:01:10 10.1.1.1 [4269015.095982]  [] ? schedule_timeout+0xa5/0xdd

May 12 05:01:10 10.1.1.1 [4269015.095987]  [] ? process_timeout+0x0/0x5

May 12 05:01:10 10.1.1.1 [4269015.095993]  [] ? fget+0x14/0x53

May 12 05:01:10 10.1.1.1 [4269015.095999]  [] ? sys_epoll_wait+0x186/0x256

May 12 05:01:10 10.1.1.1 [4269015.096004]  [] ? default_wake_function+0x0/0x9

May 12 05:01:10 10.1.1.1 [4269015.096009]  [] ? filp_close+0x5b/0x62

May 12 05:01:10 10.1.1.1 [4269015.096013]  [] ? system_call_fastpath+0x16/0x1b

May 12 05:01:10 10.1.1.1 [4269015.096016] Code: 01 00 00 00 74 13 48 8b 94 24 f0 01 00 00 c7 02 00 00 00 00 e9 91 00 00 00 41 8b 77 08 48 8b 84 24 78 01 00 00 31 d2 48 c1 e0 0a <48> f7 f6 48 8b b4 24 80 01 00 00 48 89 84 24 70 01 00 00 31 c0

May 12 05:01:10 10.1.1.1 [4269015.096042] RIP  [] find_busiest_group+0x466/0xa2e

May 12 05:01:10 10.1.1.1 [4269015.096048]  RSP

May 12 05:01:10 10.1.1.1 [4269015.096051] ---[ end trace 2ff8aecaebc3aefe ]---

查看到以下信息 查了相关资料 说是2.6.32-5内核BUG

May 12 05:01:09 10.1.1.1 [4269015.094623] Pid: 0, comm: swapper Tainted: G      D    2.6.32-5-amd64 #1 PowerEdge R510May 12 05:01:09 10.1.1.1 [4269014.583472] BUG: scheduling while atomic: swapper/0/0x10000100

May 12 05:01:10 10.1.1.1 [4269015.095604] divide error: 0000 [#3]

May 12 05:01:10 10.1.1.1 [4269015.095608] Kernel panic - not syncing: Fatal exception in interrupt

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值