netbsd-bugs@netbsd.org
[Top] [All Lists]

kern/39297: mfi calls tsleep() from mfi_intr()

Subject: kern/39297: mfi calls tsleep) from mfi_intr(
From:
Date: Tue, 5 Aug 2008 17:25:00 +0000 UTC
>Number:         39297
>Category:       kern
>Synopsis:       mfi driver calls tsleep() from mfi_intr()
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Aug 05 17:25:00 +0000 2008
>Originator:     Greg Oster
>Release:        NetBSD 4.99.71
>Organization:
>Environment:
System: NetBSD hapi 4.99.71 NetBSD 4.99.71 (GENERIC) #0: Thu Jul 31 11:15:42 
CST 2008  root@hapi:/u1/builds/build247/src/sys/arch/amd64/compile/obj/GENERIC 
amd64
Architecture: amd64
Machine: amd64
>Description:

        Running 4.99.71 (and some revisions earlier) on a machine with
using the mfi will result in the machine eventually locking up.  Breaking
into ddb yields the following:

login: fatal breakpoint trap in supervisor mode
trap type 1 code 0 rip ffffffff804dba45 cs 8 rflags 202 cr2  ffff8000720a8000 cp
l 8 rsp ffff800062c4b7f8
Stopped in pid 0.2 (system) at  netbsd:breakpoint+0x5:  leave
db{0}> tr
breakpoint() at netbsd:breakpoint+0x5
comintr() at netbsd:comintr+0x53a
Xintr_ioapic_edge6() at netbsd:Xintr_ioapic_edge6+0xef
--- interrupt ---
mutex_spin_retry() at netbsd:mutex_spin_retry+0x5a
ltsleep() at netbsd:ltsleep+0xe5
mfi_mgmt() at netbsd:mfi_mgmt+0xe1
mfi_scsipi_request() at netbsd:mfi_scsipi_request+0x331
scsipi_run_queue() at netbsd:scsipi_run_queue+0x16e
mfi_intr() at netbsd:mfi_intr+0xc0
intr_biglock_wrapper() at netbsd:intr_biglock_wrapper+0x1d
Xintr_ioapic_level2() at netbsd:Xintr_ioapic_level2+0xf7
--- interrupt ---
x86_mwait() at netbsd:x86_mwait+0xd
idle_loop() at netbsd:idle_loop+0xe9
db{0}> ps
 PID           PPID     PGRP        UID S   FLAGS LWPS          COMMAND    WAIT
 2165          2339      498          0 2  0x4000    1              cc1 biowait
 2339          2262      498          0 2  0x4000    1               cc    wait
 2262           529      498          0 2  0x4000    1               sh    wait
 529            498      498          0 2  0x4000    1               sh    wait
 498            468      498          0 2  0x4000    1               sh    wait
 468            428      468          0 2  0x4001    1              csh   pause
 428            267      428          0 2  0x4100    1             sshd  select
 454            357        2       1000 2  0x4100    2           mysqld       *
 393            343      343         12 2  0x4100    1             qmgr  kqueue
 382            343      343         12 2  0x4100    1           pickup  kqueue
 381              1      381          0 2  0x4000    1            getty  ttyraw
 302              1      302          0 2  0x4000    1            getty  ttyraw
 307              1      307          0 2  0x4000    1            getty  ttyraw
 383              1      383          0 2  0x4000    1            getty  ttyraw
 378              1      378          0 2       0    1             cron nanoslp
 360              1      360          0 2       0    1            inetd  kqueue
 357              1        2          0 2  0x4000    1               sh    wait
 343              1      343          0 2  0x4100    1           master  kqueue
 267              1      267          0 2       0    1             sshd  select
 123              1      123          0 2       0    1          syslogd
 89               1       89          0 2       0    1         dhclient  select
 1                0        1          0 2  0x4001    1             init    wait
>0               -1        0          0 2 0x20002   77           system       *
db{0}> ps/l
 PID         LID S     FLAGS       STRUCT LWP *               NAME WAIT
 2165          1 3         4   ffff80007cbc57c0                cc1 biowait
 2339          1 3        84   ffff80007cbc7bc0                 cc wait
 2262          1 3        84   ffff800072bb1020                 sh wait
 529           1 3        84   ffff800072bb1bc0                 sh wait
 498           1 3        84   ffff800072bb17e0                 sh wait
 468           1 3        84   ffff800072a6d3e0                csh pause
 428           1 3        84   ffff800072a6d7c0               sshd select
 454           2 3        84   ffff800072a6d000             mysqld sigwait
               1 3        84   ffff8000729b3420             mysqld select
 393           1 3        84   ffff8000729287e0               qmgr kqueue
 382           1 3        84   ffff8000728b1000             pickup kqueue
 381           1 3        84   ffff800072a6dba0              getty ttyraw
 302           1 3        84   ffff800072600040              getty ttyraw
 307           1 3        84   ffff8000725317e0              getty ttyraw
 383           1 3        84   ffff800072531bc0              getty ttyraw
 378           1 3        84   ffff8000729b3040               cron nanoslp
 360           1 3        84   ffff800072600420              inetd kqueue
 357           1 3        84   ffff800072600800                 sh wait
 343           1 3        84   ffff800072928bc0             master kqueue
 267           1 3        84   ffff8000728b1ba0               sshd select
 123           1 2         4   ffff800072531020            syslogd
 89            1 3        84   ffff800072531400           dhclient select
 1             1 3        84   ffff800067b71ba0               init wait
>0            85 3       204   ffff800072600be0            physiod physiod
              84 3       204   ffff800072523000        vmem_rehash vmem_rehash
              83 3       204   ffff8000725233e0           aiodoned aiodoned
              82 3       204   ffff8000725237c0            ioflush xscmd
              81 3       204   ffff800072523ba0           pgdaemon pgdaemon
              80 3       204   ffff800067b70800          cryptoret crypto_wait
              79 3       204   ffff800067b70be0               usb3 usbevt
              78 3       204   ffff800067b6d020               usb2 usbevt
              77 3       204   ffff800067b6d400               usb1 usbevt
              76 3       204   ffff800067b70420         usbtask-dr usbtsk
              75 3       204   ffff800067b70040         usbtask-hc usbtsk
              74 3       204   ffff800067b717c0               usb0 usbevt
              73 3       204   ffff800067b73040          atapibus0 sccomp
              72 3       204   ffff800067b73420               mfi0 mfi0
              71 3       204   ffff800067b73800          coretemp7 coretemp7
              70 3       204   ffff800067b73be0          coretemp6 coretemp6
              69 3       204   ffff800067b72020          coretemp5 coretemp5
              68 3       204   ffff800067b72400          coretemp4 coretemp4
              67 3       204   ffff800067b727e0          coretemp3 coretemp3
              66 3       204   ffff800067b72bc0          coretemp2 coretemp2
              65 3       204   ffff800067b71000          coretemp1 coretemp1
              64 3       204   ffff800067b713e0          coretemp0 coretemp0
              55 3       204   ffff800067b6d7e0            atabus0 atath
              54 3       204   ffff800067b6dbc0           scsibus0 sccomp
              53 3       204   ffff800067b69000            xcall/7 xcall
              52 1       204   ffff800067b693e0          softser/7
              51 1       204   ffff800067b697c0          softclk/7
              50 1       204   ffff800067b69ba0          softbio/7
              49 1       204   ffff800067b67040          softnet/7
              48 7       205   ffff800067b67420             idle/7
              47 3       204   ffff800067b67800            xcall/6 xcall
              46 1       204   ffff800067b67be0          softser/6
              45 1       204   ffff800067b65020          softclk/6
              44 1       204   ffff800067b65400          softbio/6
              43 1       204   ffff800067b657e0          softnet/6
              42 7       205   ffff800067b65bc0             idle/6
              41 3       204   ffff800067b63000            xcall/5 xcall
              40 1       204   ffff800067b633e0          softser/5
              39 1       204   ffff800067b637c0          softclk/5
              38 1       204   ffff800067b63ba0          softbio/5
              37 1       204   ffff800067b61040          softnet/5
              36 7       205   ffff800067b61420             idle/5
              35 3       204   ffff800067b61800            xcall/4 xcall
              34 1       204   ffff800067b61be0          softser/4
              33 1       204   ffff800067b60020          softclk/4
              32 1       204   ffff800067b60400          softbio/4
              31 1       204   ffff800067b607e0          softnet/4
              30 7       205   ffff800067b60bc0             idle/4
              29 3       204   ffff8000626de000            xcall/3 xcall
              28 1       204   ffff8000626de3e0          softser/3
              27 1       204   ffff8000626de7c0          softclk/3
              26 1       204   ffff8000626deba0          softbio/3
              25 1       204   ffff8000626dc040          softnet/3
              24 7       205   ffff8000626dc420             idle/3
              23 3       204   ffff8000626dc800            xcall/2 xcall
              22 1       204   ffff8000626dcbe0          softser/2
              21 1       204   ffff8000626db020          softclk/2
              20 1       204   ffff8000626db400          softbio/2
              19 1       204   ffff8000626db7e0          softnet/2
              18 7       205   ffff8000626dbbc0             idle/2
              17 3       204   ffff8000626d9000            xcall/1 xcall
              16 1       204   ffff8000626d93e0          softser/1
              15 1       204   ffff8000626d97c0          softclk/1
              14 1       204   ffff8000626d9ba0          softbio/1
              13 1       204   ffff8000626d3040          softnet/1
              12 7       205   ffff8000626d3420             idle/1
              11 3       204   ffff8000626d3800             sysmon smtaskq
              10 3       204   ffff8000626d3be0           pmfevent pmfevent
               9 3       204   ffff8000626cc020            cachegc cachegc
               8 3       204   ffff8000626cc400              vrele vrele
               7 3       204   ffff8000626cc7e0            xcall/0 xcall
               6 1       204   ffff8000626ccbc0          softser/0
               5 1       204   ffff8000626ca000          softclk/0
               4 1       204   ffff8000626ca3e0          softbio/0
               3 1       204   ffff8000626ca7c0          softnet/0
           >   2 7       205   ffff8000626caba0             idle/0 mfi_mgmt
               1 3       204   ffffffff80b57b40            swapper schedule
db{0}> tr
breakpoint() at netbsd:breakpoint+0x5
comintr() at netbsd:comintr+0x53a
Xintr_ioapic_edge6() at netbsd:Xintr_ioapic_edge6+0xef
--- interrupt ---
mutex_spin_retry() at netbsd:mutex_spin_retry+0x5a
ltsleep() at netbsd:ltsleep+0xe5
mfi_mgmt() at netbsd:mfi_mgmt+0xe1
mfi_scsipi_request() at netbsd:mfi_scsipi_request+0x331
scsipi_run_queue() at netbsd:scsipi_run_queue+0x16e
mfi_intr() at netbsd:mfi_intr+0xc0
intr_biglock_wrapper() at netbsd:intr_biglock_wrapper+0x1d
Xintr_ioapic_level2() at netbsd:Xintr_ioapic_level2+0xf7
--- interrupt ---
x86_mwait() at netbsd:x86_mwait+0xd
idle_loop() at netbsd:idle_loop+0xe9
db{0}> cpu 1
No such command
db{0}> mach cpu 1
using CPU 1
db{0}> tr
x86_mwait() at netbsd:x86_mwait+0xd
idle_loop() at netbsd:idle_loop+0xe9
Bad frame pointer: 0xffff8000626d3420
db{0}> mach cpu 2
using CPU 2
db{0}> tr
x86_mwait() at netbsd:x86_mwait+0xd
idle_loop() at netbsd:idle_loop+0xe9
Bad frame pointer: 0xffff8000626dbbc0
db{0}> mach cpu 3
using CPU 3
db{0}> tr
x86_mwait() at netbsd:x86_mwait+0xd
idle_loop() at netbsd:idle_loop+0xe9
Bad frame pointer: 0xffff8000626dc420
db{0}> mach cpu 4
using CPU 4
db{0}> tr
x86_mwait() at netbsd:x86_mwait+0xd
idle_loop() at netbsd:idle_loop+0xe9
?() at 0
db{0}> mach cpu 5
using CPU 5
db{0}> tr
x86_mwait() at netbsd:x86_mwait+0xd
idle_loop() at netbsd:idle_loop+0xe9
Bad frame pointer: 0xffff800067b61420
db{0}> mach cpu 6
using CPU 6
db{0}> tr
x86_mwait() at netbsd:x86_mwait+0xd
idle_loop() at netbsd:idle_loop+0xe9
Bad frame pointer: 0xffff800067b65bc0
db{0}> mach cpu 7 
using CPU 7
db{0}> tr
x86_mwait() at netbsd:x86_mwait+0xd
idle_loop() at netbsd:idle_loop+0xe9
Bad frame pointer: 0xffff800067b67420
db{0}> c
[*system resumes totally hung position here* break into ddb again]
fatal breakpoint trap in supervisor mode
trap type 1 code 0 rip ffffffff804dba45 cs 8 rflags 202 cr2  ffff8000720a8000 cp
l 8 rsp ffff800062c4b7f8
Stopped in pid 0.2 (system) at  netbsd:breakpoint+0x5:  leave
db{0}> 
fatal breakpoint trap in supervisor mode
trap type 1 code 0 rip ffffffff804dba45 cs 8 rflags 202 cr2  ffff8000720a8000 cp
l 8 rsp ffff800062c4b7f8
Stopped in pid 0.2 (system) at  netbsd:breakpoint+0x5:  leave
db{0}> call cpu_reset

Communications disconnect (Back at rizzo)
----------------------------------------------------
(/amd/amun/staff/oster/) C-Kermit>c
Connecting to /dev/tty00, speed 57600
 Escape character: Ctrl-] (ASCII 29, GS): enabled
Type the escape character followed by C to get back,
or followed by ? to see other options.
----------------------------------------------------

>> NetBSD/x86 BIOS Boot, Revision 3.3
>> (oster@quadrapus, Wed Nov 14 14:18:22 CST 2007)
>> Memory: 636/3406464 k
Press return to boot now, any other key for boot menu
booting hd0a:netbsd - starting in 0  
9662688+475136+778144 [689592+447629]=0xc801a8
Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
    2006, 2007, 2008
    The NetBSD Foundation, Inc.  All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
    The Regents of the University of California.  All rights reserved.

NetBSD 4.99.71 (GENERIC) #0: Thu Jul 31 11:15:42 CST 2008
        root@hapi:/u1/builds/build247/src/sys/arch/amd64/compile/obj/GENERIC
total memory = 16383 MB
avail memory = 15870 MB
SMBIOS rev. 2.4 @ 0xcffbc000 (62 entries)
Dell Inc. PowerEdge 2950
mainbus0 (root)
cpu0 at mainbus0 apid 0: Intel 686-class, 2660MHz, id 0x6fb
cpu1 at mainbus0 apid 4: Intel 686-class, 2660MHz, id 0x6fb
cpu2 at mainbus0 apid 1: Intel 686-class, 2660MHz, id 0x6fb
cpu3 at mainbus0 apid 5: Intel 686-class, 2660MHz, id 0x6fb
cpu4 at mainbus0 apid 2: Intel 686-class, 2660MHz, id 0x6fb
cpu5 at mainbus0 apid 6: Intel 686-class, 2660MHz, id 0x6fb
cpu6 at mainbus0 apid 3: Intel 686-class, 2660MHz, id 0x6fb
cpu7 at mainbus0 apid 7: Intel 686-class, 2660MHz, id 0x6fb
ioapic0 at mainbus0 apid 8
ioapic1 at mainbus0 apid 9
acpi0 at mainbus0: Intel ACPICA 20080321
attimer1 at acpi0 (TMR, PNP0100): AT Timer
attimer1: io 0x40-0x5f irq 0
COMA (PNP0501) at acpi0 not configured
COMB (PNP0501) at acpi0 not configured
hpet0 at acpi0 (HPET, PNP0103-0)
hpet0: mem 0xfed00000-0xfed003ff
pci0 at mainbus0 bus 0: configuration mode 1
pchb0 at pci0 dev 0 function 0
pchb0: vendor 0x8086 product 0x25c0 (rev. 0x12)
ppb0 at pci0 dev 2 function 0: vendor 0x8086 product 0x25e2 (rev. 0x12)
pci1 at ppb0 bus 6
ppb1 at pci1 dev 0 function 0: vendor 0x8086 product 0x3500 (rev. 0x01)
pci2 at ppb1 bus 7
ppb2 at pci2 dev 0 function 0: vendor 0x8086 product 0x3510 (rev. 0x01)
pci3 at ppb2 bus 8
ppb3 at pci3 dev 0 function 0: vendor 0x1166 product 0x0103 (rev. 0xc3)
ppb3: disabling notification events
pci4 at ppb3 bus 9
bnx0 at pci4 dev 0 function 0: Broadcom NetXtreme II BCM5708 1000Base-T
bnx0: Ethernet address 00:1c:23:ca:dd:3a
brgphy0 at bnx0 phy 1: BCM5708C 1000BASE-T media interface, rev. 6
brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FD
X, auto
ppb4 at pci2 dev 1 function 0: vendor 0x8086 product 0x3514 (rev. 0x01)
pci5 at ppb4 bus 10
ppb5 at pci1 dev 0 function 3: vendor 0x8086 product 0x350c (rev. 0x01)
ppb5: disabling notification events
pci6 at ppb5 bus 11
ppb6 at pci0 dev 3 function 0: vendor 0x8086 product 0x25e3 (rev. 0x12)
pci7 at ppb6 bus 1
ppb7 at pci7 dev 0 function 0: vendor 0x8086 product 0x0370 (rev. 0x00)
ppb7: disabling notification events
pci8 at ppb7 bus 2
mfi0 at pci8 dev 14 function 0: Dell PERC 5/i integrated
mfi0: interrupting at ioapic1 pin 14
mfi0: logical drives 2, version 5.1.1-0040, 256MB RAM
scsibus0 at mfi0: 64 targets, 8 luns per target
ppb8 at pci7 dev 0 function 2: vendor 0x8086 product 0x0372 (rev. 0x00)
ppb8: disabling notification events
pci9 at ppb8 bus 3
ppb9 at pci0 dev 4 function 0: vendor 0x8086 product 0x25f8 (rev. 0x12)
pci10 at ppb9 bus 12
ppb10 at pci0 dev 5 function 0: vendor 0x8086 product 0x25e5 (rev. 0x12)
pci11 at ppb10 bus 13
ppb11 at pci0 dev 6 function 0: vendor 0x8086 product 0x25f9 (rev. 0x12)
pci12 at ppb11 bus 14
ppb12 at pci0 dev 7 function 0: vendor 0x8086 product 0x25e7 (rev. 0x12)
pci13 at ppb12 bus 15
pchb1 at pci0 dev 16 function 0
pchb1: vendor 0x8086 product 0x25f0 (rev. 0x12)
pchb2 at pci0 dev 16 function 1
pchb2: vendor 0x8086 product 0x25f0 (rev. 0x12)
pchb3 at pci0 dev 16 function 2
pchb3: vendor 0x8086 product 0x25f0 (rev. 0x12)
pchb4 at pci0 dev 17 function 0
pchb4: vendor 0x8086 product 0x25f1 (rev. 0x12)
pchb5 at pci0 dev 19 function 0
pchb5: vendor 0x8086 product 0x25f3 (rev. 0x12)
pchb6 at pci0 dev 21 function 0
pchb6: vendor 0x8086 product 0x25f5 (rev. 0x12)
pchb7 at pci0 dev 22 function 0
pchb7: vendor 0x8086 product 0x25f6 (rev. 0x12)
ppb13 at pci0 dev 28 function 0: vendor 0x8086 product 0x2690 (rev. 0x09)
pci14 at ppb13 bus 4
ppb14 at pci14 dev 0 function 0: vendor 0x1166 product 0x0103 (rev. 0xc3)
ppb14: disabling notification events
pci15 at ppb14 bus 5
bnx1 at pci15 dev 0 function 0: Broadcom NetXtreme II BCM5708 1000Base-T
bnx1: Ethernet address 00:1c:23:ca:dd:38
brgphy1 at bnx1 phy 1: BCM5708C 1000BASE-T media interface, rev. 6
brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FD
X, auto
uhci0 at pci0 dev 29 function 0: vendor 0x8086 product 0x2688 (rev. 0x09)
uhci0: interrupting at ioapic0 pin 21
usb0 at uhci0: USB revision 1.0
uhci1 at pci0 dev 29 function 1: vendor 0x8086 product 0x2689 (rev. 0x09)
uhci1: interrupting at ioapic0 pin 20
usb1 at uhci1: USB revision 1.0
uhci2 at pci0 dev 29 function 2: vendor 0x8086 product 0x268a (rev. 0x09)
uhci2: interrupting at ioapic0 pin 21
usb2 at uhci2: USB revision 1.0
ehci0 at pci0 dev 29 function 7: vendor 0x8086 product 0x268c (rev. 0x09)
ehci0: interrupting at ioapic0 pin 21
ehci0: companion controllers, 2 ports each: uhci0 uhci1 uhci2
usb3 at ehci0: USB revision 2.0
ppb15 at pci0 dev 30 function 0: vendor 0x8086 product 0x244e (rev. 0xd9)
pci16 at ppb15 bus 16
vga0 at pci16 dev 13 function 0: vendor 0x1002 product 0x515e (rev. 0x02)
wsdisplay0 at vga0 kbdmux 1
drm at vga0 not configured
pcib0 at pci0 dev 31 function 0
pcib0: vendor 0x8086 product 0x2670 (rev. 0x09)
piixide0 at pci0 dev 31 function 1
piixide0: Intel 631xESB/632xESB IDE Controller (rev. 0x09)
piixide0: primary channel interrupting at ioapic0 pin 14
atabus0 at piixide0 channel 0
piixide0: secondary channel ignored (disabled)
isa0 at pcib0
com0 at isa0 port 0x3f8-0x3ff irq 4: ns16550a, working fifo
com0: console
com1 at isa0 port 0x2f8-0x2ff irq 3: ns16550a, working fifo
pckbc0 at isa0 port 0x60-0x64
pcppi0 at isa0 port 0x61
midi0 at pcppi0: PC speaker (CPU-intensive output)
sysbeep0 at pcppi0
attimer1: attached to pcppi0
scsibus0: waiting 2 seconds for devices to settle...
atapibus0 at atabus0: 2 targets
cd0 at atapibus0 drive 0: <HL-DT-STCD-RW/DVD-ROM GCC-T10N, , A102> cdrom removab
le
uhub0 at usb0: vendor 0x8086 UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1 at usb1: vendor 0x8086 UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub2 at usb2: vendor 0x8086 UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub3 at usb3: vendor 0x8086 EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub4 at uhub3 port 5: vendor 0x04b4 product 0x6560, class 9/0, rev 2.00/0.0b, a
ddr 2
uhub4: multiple transaction translators
sd0 at scsibus0 target 0 lun 0: <DELL, PERC 5/i, 1.03> disk fixed
sd0: fabricating a geometry
sd0: 136 GB, 139392 cyl, 64 head, 32 sec, 512 bytes/sect x 285474816 sectors
sd0: fabricating a geometry
sd1 at scsibus0 target 1 lun 0: <DELL, PERC 5/i, 1.03> disk fixed
sd1: fabricating a geometry
sd1: 136 GB, 139392 cyl, 64 head, 32 sec, 512 bytes/sect x 285474816 sectors
sd1: fabricating a geometry
Kernelized RAIDframe activated
pad0: outputs: 44100Hz, 16-bit, stereo
audio0 at pad0: half duplex
boot device: sd0
root on sd0a dumps on sd0b
/: replaying log to memory
root file system type: ffs
Thu Jul 31 15:51:30 CST 2008
swapctl: adding /dev/sd0b as swap device at priority 0
Starting file system checks:
/dev/rsd0a: file system is journaled; not checking
/dev/rsd0e: file system is journaled; not checking
/: replaying log to disk
Setting tty flags.
Setting sysctl variables:
kern.maxproc: 1044 -> 4000
kern.maxfiles: 3404 -> 1200000
kern.sched.min_catch: 1 -> 1
Starting network.
Hostname: hapi
IPv6 mode: host
Configuring network interfaces: bnx1.
Adding interface aliases:
Starting dhclient.
Building databases: dev, utmp, utmpx done
Starting syslogd.
Checking for core dump...
savecore: no core dump
Setting date via ntp.
Mounting all filesystems...
/u1: replaying log to disk
Clearing temporary files.
Checking quotas: done.
Starting virecover.
Starting local daemons:.
Updating motd.
Starting sshd.
postfix/postfix-script: starting the Postfix mail system
Starting mysqld.
Starting inetd.
Starting cron.
Thu Jul 31 15:51:51 CST 2008

NetBSD/amd64 (hapi) (console)

login: fatal breakpoint trap in supervisor mode
trap type 1 code 0 rip ffffffff804dba45 cs 8 rflags 202 cr2  7f7ffd610000 cpl 8 
rsp ffff800062c4b7f8
Stopped in pid 0.2 (system) at  netbsd:breakpoint+0x5:  leave
db{0}> tr
breakpoint() at netbsd:breakpoint+0x5
comintr() at netbsd:comintr+0x53a
Xintr_ioapic_edge6() at netbsd:Xintr_ioapic_edge6+0xef
--- interrupt ---
x86_pause() at netbsd:x86_pause+0x2
ltsleep() at netbsd:ltsleep+0xe5
mfi_mgmt() at netbsd:mfi_mgmt+0xe1
mfi_scsipi_request() at netbsd:mfi_scsipi_request+0x331
scsipi_run_queue() at netbsd:scsipi_run_queue+0x16e
mfi_intr() at netbsd:mfi_intr+0xc0
intr_biglock_wrapper() at netbsd:intr_biglock_wrapper+0x1d
Xintr_ioapic_level2() at netbsd:Xintr_ioapic_level2+0xf7
--- interrupt ---
x86_mwait() at netbsd:x86_mwait+0xd
idle_loop() at netbsd:idle_loop+0xe9
db{0}> c
[*system resumes totally hung position here* break into ddb again]
fatal breakpoint trap in supervisor mode
trap type 1 code 0 rip ffffffff804dba45 cs 8 rflags 202 cr2  7f7ffd610000 cpl 8 
rsp ffff800062c4b7f8
Stopped in pid 0.2 (system) at  netbsd:breakpoint+0x5:  leave
db{0}> c
[*system resumes totally hung position here*  break into ddb again]
fatal breakpoint trap in supervisor mode
trap type 1 code 0 rip ffffffff804dba45 cs 8 rflags 202 cr2  7f7ffd610000 cpl 8 
rsp ffff800062c4b7f8
Stopped in pid 0.2 (system) at  netbsd:breakpoint+0x5:  leave
db{0}> mach cpu 1
using CPU 1
db{0}> tr
x86_mwait() at netbsd:x86_mwait+0xd
idle_loop() at netbsd:idle_loop+0xe9
Bad frame pointer: 0xffff8000626d3420
db{0}> mach cpu 2
using CPU 2
db{0}> tr
x86_mwait() at netbsd:x86_mwait+0xd
idle_loop() at netbsd:idle_loop+0xe9
Bad frame pointer: 0xffff8000626dbbc0
db{0}> mach cpu 3
using CPU 3
db{0}> tr
x86_mwait() at netbsd:x86_mwait+0xd
idle_loop() at netbsd:idle_loop+0xe9
Bad frame pointer: 0xffff8000626dc420
db{0}> mach cpu 4
using CPU 4
db{0}> tr
x86_mwait() at netbsd:x86_mwait+0xd
idle_loop() at netbsd:idle_loop+0xe9
?() at 0
db{0}> mach cpu 5
using CPU 5
db{0}> tr
x86_mwait() at netbsd:x86_mwait+0xd
idle_loop() at netbsd:idle_loop+0xe9
Bad frame pointer: 0xffff800067b61420
db{0}> mach cpu 6
using CPU 6
db{0}> tr
x86_mwait() at netbsd:x86_mwait+0xd
idle_loop() at netbsd:idle_loop+0xe9
Bad frame pointer: 0xffff800067b65bc0
db{0}> ps/l
 PID         LID S     FLAGS       STRUCT LWP *               NAME WAIT
 486           1 3         4   ffff800072b41000                tar tstile
 489           1 3         4   ffff800072b413e0               gzip tstile
 476           1 3        84   ffff800072b417c0                csh pause
 465           1 3        84   ffff800072b41ba0               sshd select
 434           1 3         4   ffff8000725e1420             mysqld biowait
 388           1 3        84   ffff8000729d7040              getty ttyraw
 382           1 3        84   ffff8000728b53e0              getty ttyraw
 381           1 3        84   ffff8000725327e0              getty ttyraw
 302           1 3        84   ffff800072532bc0              getty ttyraw
 383           1 3        84   ffff8000729d7420               cron nanoslp
 374           1 3        84   ffff8000725e1040              inetd kqueue
 355           1 3        84   ffff8000729497e0               qmgr kqueue
 362           1 3        84   ffff8000728b5000             pickup kqueue
 357           1 3        84   ffff8000725e1800                 sh wait
 343           1 3        84   ffff800072949bc0             master kqueue
 267           1 3        84   ffff8000728b5ba0               sshd select
 123           1 2         4   ffff800072532020            syslogd
 89            1 3        84   ffff800072532400           dhclient select
 1             1 3        84   ffff800067b71ba0               init wait
>0            85 3       204   ffff8000725e1be0            physiod physiod
              84 3       204   ffff800072524000        vmem_rehash vmem_rehash
              83 3       204   ffff8000725243e0           aiodoned aiodoned
              82 3       204   ffff8000725247c0            ioflush xscmd
              81 3       204   ffff800072524ba0           pgdaemon pgdaemon
              80 3       204   ffff800067b70800          cryptoret crypto_wait
              79 3       204   ffff800067b70be0               usb3 usbevt
              78 3       204   ffff800067b6d020               usb2 usbevt
              77 3       204   ffff800067b6d400               usb1 usbevt
              76 3       204   ffff800067b70420         usbtask-dr usbtsk
              75 3       204   ffff800067b70040         usbtask-hc usbtsk
              74 3       204   ffff800067b717c0               usb0 usbevt
              73 3       204   ffff800067b73040          atapibus0 sccomp
              72 3       204   ffff800067b73420               mfi0 mfi0
              71 3       204   ffff800067b73800          coretemp7 coretemp7
              70 3       204   ffff800067b73be0          coretemp6 coretemp6
              69 3       204   ffff800067b72020          coretemp5 coretemp5
              68 3       204   ffff800067b72400          coretemp4 coretemp4
              67 3       204   ffff800067b727e0          coretemp3 coretemp3
              66 3       204   ffff800067b72bc0          coretemp2 coretemp2
              65 3       204   ffff800067b71000          coretemp1 coretemp1
              64 3       204   ffff800067b713e0          coretemp0 coretemp0
              55 3       204   ffff800067b6d7e0            atabus0 atath
              54 3       204   ffff800067b6dbc0           scsibus0 sccomp
              53 3       204   ffff800067b69000            xcall/7 xcall
              52 1       204   ffff800067b693e0          softser/7
              51 1       204   ffff800067b697c0          softclk/7
              50 1       204   ffff800067b69ba0          softbio/7
              49 1       204   ffff800067b67040          softnet/7
              48 7       205   ffff800067b67420             idle/7
              47 3       204   ffff800067b67800            xcall/6 xcall
              46 1       204   ffff800067b67be0          softser/6
              45 1       204   ffff800067b65020          softclk/6
              44 1       204   ffff800067b65400          softbio/6
              43 1       204   ffff800067b657e0          softnet/6
              42 7       205   ffff800067b65bc0             idle/6
              41 3       204   ffff800067b63000            xcall/5 xcall
              40 1       204   ffff800067b633e0          softser/5
              39 1       204   ffff800067b637c0          softclk/5
              38 1       204   ffff800067b63ba0          softbio/5
              37 1       204   ffff800067b61040          softnet/5
              36 7       205   ffff800067b61420             idle/5
              35 3       204   ffff800067b61800            xcall/4 xcall
              34 1       204   ffff800067b61be0          softser/4
              33 1       204   ffff800067b60020          softclk/4
              32 1       204   ffff800067b60400          softbio/4
              31 1       204   ffff800067b607e0          softnet/4
              30 7       205   ffff800067b60bc0             idle/4
              29 3       204   ffff8000626de000            xcall/3 xcall
              28 1       204   ffff8000626de3e0          softser/3
              27 1       204   ffff8000626de7c0          softclk/3
              26 1       204   ffff8000626deba0          softbio/3
              25 1       204   ffff8000626dc040          softnet/3
              24 7       205   ffff8000626dc420             idle/3
              23 3       204   ffff8000626dc800            xcall/2 xcall
              22 1       204   ffff8000626dcbe0          softser/2
              21 1       204   ffff8000626db020          softclk/2
              20 1       204   ffff8000626db400          softbio/2
              19 1       204   ffff8000626db7e0          softnet/2
              18 7       205   ffff8000626dbbc0             idle/2
              17 3       204   ffff8000626d9000            xcall/1 xcall
              16 1       204   ffff8000626d93e0          softser/1
              15 1       204   ffff8000626d97c0          softclk/1
              14 1       204   ffff8000626d9ba0          softbio/1
              13 1       204   ffff8000626d3040          softnet/1
              12 7       205   ffff8000626d3420             idle/1
              11 3       204   ffff8000626d3800             sysmon smtaskq
              10 3       204   ffff8000626d3be0           pmfevent pmfevent
               9 3       204   ffff8000626cc020            cachegc cachegc
               8 3       204   ffff8000626cc400              vrele vrele
               7 3       204   ffff8000626cc7e0            xcall/0 xcall
               6 1       204   ffff8000626ccbc0          softser/0
               5 1       204   ffff8000626ca000          softclk/0
               4 1       204   ffff8000626ca3e0          softbio/0
               3 1       204   ffff8000626ca7c0          softnet/0
           >   2 7       205   ffff8000626caba0             idle/0 mfi_mgmt
               1 3       204   ffffffff80b57b40            swapper schedule
db{0}> tr
x86_mwait() at netbsd:x86_mwait+0xd
idle_loop() at netbsd:idle_loop+0xe9
Bad frame pointer: 0xffff800067b65bc0
db{0}> mach cpu 0
using CPU 0
db{0}> tr
breakpoint() at netbsd:breakpoint+0x5
comintr() at netbsd:comintr+0x53a
Xintr_ioapic_edge6() at netbsd:Xintr_ioapic_edge6+0xef
--- interrupt ---
mutex_spin_retry() at netbsd:mutex_spin_retry+0x5a
ltsleep() at netbsd:ltsleep+0xe5
mfi_mgmt() at netbsd:mfi_mgmt+0xe1
mfi_scsipi_request() at netbsd:mfi_scsipi_request+0x331
scsipi_run_queue() at netbsd:scsipi_run_queue+0x16e
mfi_intr() at netbsd:mfi_intr+0xc0
intr_biglock_wrapper() at netbsd:intr_biglock_wrapper+0x1d
Xintr_ioapic_level2() at netbsd:Xintr_ioapic_level2+0xf7
--- interrupt ---
x86_mwait() at netbsd:x86_mwait+0xd
idle_loop() at netbsd:idle_loop+0xe9
db{0}> 



>How-To-Repeat:

        Boot -current on a Dell PowerEdge 2950.
        Extract a tar file.  
        Or attempt a build.sh.  
        Or just wait.
        Observe system is completely locked up.
        Enter ddb.
        Observe that ltsleep() has been called from mfi_intr().

>Fix:
        Figure out a different way of doing whatever mfi_mgmt() thinks
needs to be done by sleeping?





<Prev in Thread] Current Thread [Next in Thread>