RELENG_6 vm_fault panic on filesystem mount

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

RELENG_6 vm_fault panic on filesystem mount

Joerg Pernfuss
Hi,


last week I upgraded my desktop box from RELENG_5 to RELENG_6 without
any hassles except one: one of my partitions can't be mounted anymore,
the box panics instantly.
/dev/mlxd0s2g was my /usr/ports/distfiles and was happily mounted up
until the update.

From the output of the backtrace (below) I figure the problem for me
lies in /usr/src/sys/ufs/ufs/ufs_dirhash.c line 232; this block:

      if (ep->d_reclen == 0 || ep->d_reclen >
          DIRBLKSIZ - (pos & (DIRBLKSIZ - 1))) {
              /* Corrupted directory. */
              brelse(bp);
              goto fail;
      }

Which is consistent with the very first panic message I got saying
something about 'Fatal Trap 12, page fault while in kernel mode,
 ufs_dirhash: bad dir' - sadly, I didn't write down that one.
Hasn't reappeared since then.

fsck'ing that filesystem does nothing though...
        elessar@loki: ~> sudo fsck -f /dev/mlxd0s2g
        ** /dev/mlxd0s2g
        ** Last Mounted on /usr/ports/distfiles
        ** Phase 1 - Check Blocks and Sizes
        ** Phase 2 - Check Pathnames
        ** Phase 3 - Check Connectivity
        ** Phase 4 - Check Reference Counts
        ** Phase 5 - Check Cyl groups
        882 files, 660784 used, 352231 free (159 frags, 44009 blocks, 0.0% fragmentation)

        ***** FILE SYSTEM MARKED CLEAN *****
        elessar@loki: ~>

That was right after the panic, so it was unclean. Apart from that,
fsck finds nothing wrong.

The backtrace of the dump reads:

panic: vm_fault: fault on nofault entry, addr: d71f7000
cpuid = 0
Uptime: 1m33s
Dumping 1535 MB (2 chunks)
  chunk 0: 1MB (159 pages) ... ok
  chunk 1: 1535MB (392944 pages) [...]

#0  doadump () at pcpu.h:165
165             __asm __volatile("movl %%fs:0,%0" : "=r" (td));
(kgdb) backtrace
#0  doadump () at pcpu.h:165
During symbol reading, Incomplete CFI data; unspecified registers at 0xc05de3b3.
#1  0xc05debcd in boot (howto=0x104) at /usr/src/sys/kern/kern_shutdown.c:399
#2  0xc05de66b in panic (fmt=0x0) at /usr/src/sys/kern/kern_shutdown.c:555
#3  0xc07236e5 in vm_fault (map=0xc1060000, vaddr=0xd71f7000, fault_type=0x1, fault_flags=0x0) at /usr/src/sys/vm/vm_fault.c:884
#4  0xc079caf0 in trap_pfault (frame=0xeabb6640, usermode=0x0, eva=0xd71f7004) at /usr/src/sys/i386/i386/trap.c:731
#5  0xc079cfd7 in trap (frame= {tf_fs = 0xeabb0008, tf_es = 0xeabb0028, tf_ds = 0xeabb0028, tf_edi = 0xd6c53a30, tf_esi = 0xc4c2da00, tf_ebp = 0xeabb
66d8, tf_isp = 0xeabb666c, tf_ebx = 0x0, tf_edx = 0x0, tf_ecx = 0xd71f7000, tf_eax = 0xd71f3000, tf_trapno = 0xc, tf_err = 0x0, tf_eip = 0xc070c00c,
tf_cs = 0x20, tf_eflags = 0x10286, tf_esp = 0xffffffe8, tf_ss = 0x0}) at /usr/src/sys/i386/i386/trap.c:432
#6  0xc078806a in calltrap () at /usr/src/sys/i386/i386/exception.s:139
#7  0xc070c00c in ufsdirhash_build (ip=0xc5426948) at /usr/src/sys/ufs/ufs/ufs_dirhash.c:232
#8  0xc070f5c3 in ufs_lookup (ap=0xeabb6824) at /usr/src/sys/ufs/ufs/ufs_lookup.c:192
#9  0xc070d8f4 in ufs_extattr_lookup (start_dvp=0xc5414dd0, lockparent=0x2, dirname=0xd71f3000 "\002", vp=0xd71f3000, td=0xc5558780) at /usr/src/sys/
ufs/ufs/ufs_extattr.c:274
#10 0xc070dfd6 in ufs_extattr_autostart (mp=0xc4ca3000, td=0xc5558780) at /usr/src/sys/ufs/ufs/ufs_extattr.c:463
#11 0xc0706fa6 in ffs_mount (mp=0xc4ca3000, td=0xc5558780) at /usr/src/sys/ufs/ffs/ffs_vfsops.c:779
#12 0xc0640d57 in vfs_donmount (td=0xc5558780, fsflags=0x8008, fsoptions=0xeabb6bf4) at /usr/src/sys/kern/vfs_mount.c:739
#13 0xc06427c0 in kernel_mount (ma=0xc5235240, flags=0x0) at pcpu.h:162
#14 0xc070369d in ffs_cmount (ma=0xc5235240, data=0x0, flags=0xd71f3000, td=0xc5558780) at /usr/src/sys/ufs/ffs/ffs_vfsops.c:382
#15 0xc0642505 in mount (td=0xc5558780, uap=0xeabb6d04) at /usr/src/sys/kern/vfs_mount.c:566
#16 0xc079d3bc in syscall (frame= {tf_fs = 0x3b, tf_es = 0x3b, tf_ds = 0x3b, tf_edi = 0xbfbfddcc, tf_esi = 0xbfbfe864, tf_ebp = 0xbfbfde58, tf_isp =
0xeabb6d64, tf_ebx = 0xbfbfde80, tf_edx = 0xffffffff, tf_ecx = 0xbfbfeba2, tf_eax = 0x15, tf_trapno = 0xc, tf_err = 0x2, tf_eip = 0x480c2c5f, tf_cs =
 0x33, tf_eflags = 0x256, tf_esp = 0xbfbfddac, tf_ss = 0x3b}) at /usr/src/sys/i386/i386/trap.c:976
#17 0xc07880bf in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:200
#18 0x00000033 in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb) quit

The last line about my stack confuses me even more.

I'd be happy for any help. As said above, this can be reproduced in the
blink of a mount and as it is my desktop, I can try out whatever you
need.
The sources I built from are about two weeks old. I didn't update so far,
as I couldn't find anything in the commit logs indicating it has been
fixed already. Well, if there is something that can be fixed.

Thanks, Joerg.

PS: verbose dmesg following.

ce Directory header at 0xc00faff0
bios32: Entry = 0xfb460 (c00fb460)  Rev = 0  Len = 1
pcibios: PCI BIOS entry at 0xf0000+0xb490
pnpbios: Found PnP BIOS data at 0xc00fbe90
pnpbios: Entry = f0000:bec0  Rev = 1.0
Other BIOS signatures found:
MADT: Found IO APIC ID 2, Interrupt 0 at 0xfec00000
ioapic0: Routing external 8259A's -> intpin 0
ioapic0: intpin 0 -> ExtINT (edge, high)
ioapic0: intpin 1 -> ISA IRQ 1 (edge, high)
ioapic0: intpin 2 -> ISA IRQ 2 (edge, high)
ioapic0: intpin 3 -> ISA IRQ 3 (edge, high)
ioapic0: intpin 4 -> ISA IRQ 4 (edge, high)
ioapic0: intpin 5 -> ISA IRQ 5 (edge, high)
ioapic0: intpin 6 -> ISA IRQ 6 (edge, high)
ioapic0: intpin 7 -> ISA IRQ 7 (edge, high)
ioapic0: intpin 8 -> ISA IRQ 8 (edge, high)
ioapic0: intpin 9 -> ISA IRQ 9 (edge, high)
ioapic0: intpin 10 -> ISA IRQ 10 (edge, high)
ioapic0: intpin 11 -> ISA IRQ 11 (edge, high)
ioapic0: intpin 12 -> ISA IRQ 12 (edge, high)
ioapic0: intpin 13 -> ISA IRQ 13 (edge, high)
ioapic0: intpin 14 -> ISA IRQ 14 (edge, high)
ioapic0: intpin 15 -> ISA IRQ 15 (edge, high)
ioapic0: intpin 16 -> PCI IRQ 16 (level, low)
ioapic0: intpin 17 -> PCI IRQ 17 (level, low)
ioapic0: intpin 18 -> PCI IRQ 18 (level, low)
ioapic0: intpin 19 -> PCI IRQ 19 (level, low)
ioapic0: intpin 20 -> PCI IRQ 20 (level, low)
ioapic0: intpin 21 -> PCI IRQ 21 (level, low)
ioapic0: intpin 22 -> PCI IRQ 22 (level, low)
ioapic0: intpin 23 -> PCI IRQ 23 (level, low)
MADT: Interrupt override: source 0, irq 2
ioapic0: Routing IRQ 0 -> intpin 2
ioapic0: intpin 2 trigger: edge
ioapic0: intpin 2 polarity: high
MADT: Interrupt override: source 9, irq 9
ioapic0: intpin 9 trigger: level
ioapic0: intpin 9 polarity: low
ioapic0 <Version 1.1> irqs 0-23 on motherboard
cpu0 BSP:
     ID: 0x00000000   VER: 0x00040011 LDR: 0x01000000 DFR: 0x0fffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00000000 err: 0x00010000 pcm: 0x00010000
wlan: <802.11 Link Layer>
ath_rate: <Atsushi Onoe's rate control algorithm>
crypto: <crypto core>
mem: <memory>
Pentium Pro MTRR support enabled
null: <null device, zero device>
random: <entropy source, Software, Yarrow>
io: <I/O>
VESA: information block
56 45 53 41 00 03 33 57 00 c0 01 00 00 00 bd 52
00 c0 00 02 00 01 48 57 00 c0 4f 57 00 c0 5b 57
00 c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
VESA: 17 mode(s) found
VESA: v3.0, 32768k memory, flags:0x1, mode table:0xc00c52bd (c00052bd)
VESA: Matrox Graphics Inc.
VESA: Matrox Matrox G450 00
ath_hal: 0.9.14.9 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413)
npx0: [FAST]
npx0: <math processor> on motherboard
npx0: INT 16 interface
acpi0: <VIA694 AWRDACPI> on motherboard
acpi0: [MPSAFE]
pci_open(1):    mode 1 addr port (0x0cf8) is 0x00000000
pci_open(1a):   mode1res=0x80000000 (0x80000000)
pci_cfgcheck:   device 0 [class=060000] [hdr=00] is there (id=06911106)
pcibios: BIOS version 2.10
Found $PIR table, 8 entries at 0xc00fdd20
PCI-Only Interrupts: 5 10 11
Location  Bus Device Pin  Link  IRQs
slot 1      0    8    A   0x02  3 4 5 7 9 10 11 12 14 15
slot 1      0    8    B   0x03  3 4 5 7 9 10 11 12 14 15
slot 1      0    8    C   0x05  3 4 5 7 9 10 11 12 14 15
slot 1      0    8    D   0x01  3 4 5 7 9 10 11 12 14 15
slot 2      0    9    A   0x03  3 4 5 7 9 10 11 12 14 15
slot 2      0    9    B   0x05  3 4 5 7 9 10 11 12 14 15
slot 2      0    9    C   0x01  3 4 5 7 9 10 11 12 14 15
slot 2      0    9    D   0x02  3 4 5 7 9 10 11 12 14 15
slot 3      0   10    A   0x05  3 4 5 7 9 10 11 12 14 15
slot 3      0   10    B   0x01  3 4 5 7 9 10 11 12 14 15
slot 3      0   10    C   0x02  3 4 5 7 9 10 11 12 14 15
slot 3      0   10    D   0x03  3 4 5 7 9 10 11 12 14 15
slot 4      0   11    A   0x01  3 4 5 7 9 10 11 12 14 15
slot 4      0   11    B   0x02  3 4 5 7 9 10 11 12 14 15
slot 4      0   11    C   0x03  3 4 5 7 9 10 11 12 14 15
slot 4      0   11    D   0x05  3 4 5 7 9 10 11 12 14 15
slot 5      0   12    A   0x02  3 4 5 7 9 10 11 12 14 15
slot 5      0   12    B   0x03  3 4 5 7 9 10 11 12 14 15
slot 5      0   12    C   0x05  3 4 5 7 9 10 11 12 14 15
slot 5      0   12    D   0x01  3 4 5 7 9 10 11 12 14 15
slot 6      0   13    A   0x03  3 4 5 7 9 10 11 12 14 15
slot 6      0   13    B   0x05  3 4 5 7 9 10 11 12 14 15
slot 6      0   13    C   0x01  3 4 5 7 9 10 11 12 14 15
slot 6      0   13    D   0x02  3 4 5 7 9 10 11 12 14 15
embedded    0    1    A   0x01  3 4 5 7 9 10 11 12 14 15
embedded    0    1    B   0x02  3 4 5 7 9 10 11 12 14 15
embedded    0    1    C   0x03  3 4 5 7 9 10 11 12 14 15
embedded    0    1    D   0x05  3 4 5 7 9 10 11 12 14 15
embedded    0    7    C   0x03  3 4 5 7 9 10 11 12 14 15
embedded    0    7    D   0x05  3 4 5 7 9 10 11 12 14 15
acpi_bus_number: root bus has no _BBN, assuming 0
AcpiOsDerivePciId: bus 0 dev 7 func 0
acpi_bus_number: root bus has no _BBN, assuming 0
AcpiOsDerivePciId: bus 0 dev 7 func 0
acpi0: Power Button (fixed)
pci_link0: <ACPI PCI Link LNKA> irq 5 on acpi0
pci_link0: Links after initial probe:
Index  IRQ  Rtd  Ref  IRQs
    0    5   N     0  1 3 4 5 6 7 10 11 12 14 15
pci_link0: Links after initial validation:
Index  IRQ  Rtd  Ref  IRQs
    0    5   N     0  1 3 4 5 6 7 10 11 12 14 15
pci_link0: Links after disable:
Index  IRQ  Rtd  Ref  IRQs
    0  255   N     0  1 3 4 5 6 7 10 11 12 14 15
pci_link1: <ACPI PCI Link LNKB> irq 11 on acpi0
pci_link1: Links after initial probe:
Index  IRQ  Rtd  Ref  IRQs
    0   11   N     0  1 3 4 5 6 7 10 11 12 14 15
pci_link1: Links after initial validation:
Index  IRQ  Rtd  Ref  IRQs
    0   11   N     0  1 3 4 5 6 7 10 11 12 14 15
pci_link1: Links after disable:
Index  IRQ  Rtd  Ref  IRQs
    0  255   N     0  1 3 4 5 6 7 10 11 12 14 15
pci_link2: <ACPI PCI Link LNKC> irq 10 on acpi0
pci_link2: Links after initial probe:
Index  IRQ  Rtd  Ref  IRQs
    0   10   N     0  1 3 4 5 6 7 10 11 12 14 15
pci_link2: Links after initial validation:
Index  IRQ  Rtd  Ref  IRQs
    0   10   N     0  1 3 4 5 6 7 10 11 12 14 15
pci_link2: Links after disable:
Index  IRQ  Rtd  Ref  IRQs
    0  255   N     0  1 3 4 5 6 7 10 11 12 14 15
pci_link3: <ACPI PCI Link LNKD> irq 11 on acpi0
pci_link3: Links after initial probe:
Index  IRQ  Rtd  Ref  IRQs
    0   11   N     0  1 3 4 5 6 7 10 11 12 14 15
pci_link3: Links after initial validation:
Index  IRQ  Rtd  Ref  IRQs
    0   11   N     0  1 3 4 5 6 7 10 11 12 14 15
pci_link3: Links after disable:
Index  IRQ  Rtd  Ref  IRQs
    0  255   N     0  1 3 4 5 6 7 10 11 12 14 15
ACPI timer: 1/1 1/1 1/1 1/1 1/1 1/1 1/1 1/1 1/1 1/1 -> 10
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
acpi_button0: <Power Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff,0x4000-0x407f,0x4080-0x40ff,0x5000-0x500f,0x6000-0x607f on acpi0
pci0: <ACPI PCI bus> on pcib0
pci0: physical bus=0
found-> vendor=0x1106, dev=0x0691, revid=0xc4
        bus=0, slot=0, func=0
        class=06-00-00, hdrtype=0x00, mfdev=0
        cmdreg=0x0006, statreg=0x2210, cachelnsz=0 (dwords)
        lattimer=0x08 (240 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 2  supports D0 D3  current D0
        map[10]: type 3, range 32, base f0000000, size 25, enabled
found-> vendor=0x1106, dev=0x8598, revid=0x00
        bus=0, slot=1, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0007, statreg=0x2230, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x0c (3000 ns), maxlat=0x00 (0 ns)
found-> vendor=0x1106, dev=0x0686, revid=0x40
        bus=0, slot=7, func=0
        class=06-01-00, hdrtype=0x00, mfdev=1
        cmdreg=0x0087, statreg=0x0210, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 2  supports D0 D3  current D0
found-> vendor=0x1106, dev=0x0571, revid=0x06
        bus=0, slot=7, func=1
        class=01-01-8a, hdrtype=0x00, mfdev=0
        cmdreg=0x0007, statreg=0x0290, cachelnsz=0 (dwords)
        lattimer=0x20 (960 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 2  supports D0 D3  current D0
        map[20]: type 4, range 32, base 0000d800, size  4, enabled
found-> vendor=0x1106, dev=0x3038, revid=0x1a
        bus=0, slot=7, func=2
        class=0c-03-00, hdrtype=0x00, mfdev=0
        cmdreg=0x0007, statreg=0x0210, cachelnsz=8 (dwords)
        lattimer=0x20 (960 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=d, irq=11
        powerspec 2  supports D0 D3  current D0
        map[20]: type 4, range 32, base 0000d000, size  5, enabled
pcib0: matched entry for 0.7.INTD (src \\_SB_.PCI0.LNKD:0)
ioapic0: Changing trigger for pin 11 to level
ioapic0: Changing polarity for pin 11 to low
pcib0: slot 7 INTD routed to irq 11 via \\_SB_.PCI0.LNKD
found-> vendor=0x1106, dev=0x3038, revid=0x1a
        bus=0, slot=7, func=3
        class=0c-03-00, hdrtype=0x00, mfdev=0
        cmdreg=0x0007, statreg=0x0210, cachelnsz=8 (dwords)
        lattimer=0x20 (960 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=d, irq=11
        powerspec 2  supports D0 D3  current D0
        map[20]: type 4, range 32, base 0000d400, size  5, enabled
pcib0: matched entry for 0.7.INTD (src \\_SB_.PCI0.LNKD:0)
pcib0: slot 7 INTD routed to irq 11 via \\_SB_.PCI0.LNKD
found-> vendor=0x1106, dev=0x3057, revid=0x40
        bus=0, slot=7, func=4
        class=06-80-00, hdrtype=0x00, mfdev=0
        cmdreg=0x0000, statreg=0x0290, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 2  supports D0 D3  current D0
found-> vendor=0x1011, dev=0x0024, revid=0x03
        bus=0, slot=9, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0107, statreg=0x0290, cachelnsz=8 (dwords)
        lattimer=0x20 (960 ns), mingnt=0x06 (1500 ns), maxlat=0x00 (0 ns)
found-> vendor=0x8086, dev=0x0960, revid=0x05
        bus=0, slot=10, func=0
        class=06-04-00, hdrtype=0x01, mfdev=1
        cmdreg=0x0007, statreg=0x0280, cachelnsz=8 (dwords)
        lattimer=0x20 (960 ns), mingnt=0x06 (1500 ns), maxlat=0x02 (500 ns)
found-> vendor=0x1069, dev=0x0010, revid=0x05
        bus=0, slot=10, func=1
        class=01-04-00, hdrtype=0x00, mfdev=1
        cmdreg=0x0006, statreg=0x2290, cachelnsz=8 (dwords)
        lattimer=0x20 (960 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=a, irq=11
        powerspec 2  supports D0 D3  current D0
        map[10]: type 3, range 32, base fa000000, size 23, enabled
pcib0: matched entry for 0.10.INTA
pcib0: slot 10 INTA hardwired to IRQ 19
found-> vendor=0x168c, dev=0x0013, revid=0x01
        bus=0, slot=11, func=0
        class=02-00-00, hdrtype=0x00, mfdev=0
        cmdreg=0x0006, statreg=0x0290, cachelnsz=8 (dwords)
        lattimer=0x20 (960 ns), mingnt=0x0a (2500 ns), maxlat=0x1c (7000 ns)
        intpin=a, irq=5
        powerspec 2  supports D0 D3  current D0
        map[10]: type 1, range 32, base fa900000, size 16, enabled
pcib0: matched entry for 0.11.INTA
pcib0: slot 11 INTA hardwired to IRQ 16
found-> vendor=0x1102, dev=0x0002, revid=0x07
        bus=0, slot=12, func=0
        class=04-01-00, hdrtype=0x00, mfdev=1
        cmdreg=0x0005, statreg=0x0290, cachelnsz=0 (dwords)
        lattimer=0x20 (960 ns), mingnt=0x02 (500 ns), maxlat=0x14 (5000 ns)
        intpin=a, irq=11
        powerspec 1  supports D0 D1 D2 D3  current D0
        map[10]: type 4, range 32, base 0000dc00, size  5, enabled
pcib0: matched entry for 0.12.INTA
pcib0: slot 12 INTA hardwired to IRQ 17
found-> vendor=0x1102, dev=0x7002, revid=0x07
        bus=0, slot=12, func=1
        class=09-80-00, hdrtype=0x00, mfdev=1
        cmdreg=0x0005, statreg=0x0290, cachelnsz=0 (dwords)
        lattimer=0x20 (960 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        powerspec 1  supports D0 D1 D2 D3  current D0
        map[10]: type 4, range 32, base 0000e000, size  3, enabled
agp0: <VIA 82C691 (Apollo Pro) host to PCI bridge> mem 0xf0000000-0xf1ffffff at device 0.0 on pci0
agp0: Reserved 0x2000000 bytes for rid 0x10 type 3 at 0xf0000000
agp0: allocating GATT for aperture of size 256M
pcib1: <PCI-PCI bridge> at device 1.0 on pci0
pcib1:   secondary bus     1
pcib1:   subordinate bus   1
pcib1:   I/O decode        0xf000-0xfff
pcib1:   memory decode     0xf4000000-0xf6ffffff
pcib1:   prefetched decode 0xf2000000-0xf3ffffff
pci1: <PCI bus> on pcib1
pci1: physical bus=1
found-> vendor=0x102b, dev=0x0525, revid=0x85
        bus=1, slot=0, func=0
        class=03-00-00, hdrtype=0x00, mfdev=0
        cmdreg=0x0007, statreg=0x0290, cachelnsz=8 (dwords)
        lattimer=0x20 (960 ns), mingnt=0x10 (4000 ns), maxlat=0x20 (8000 ns)
        intpin=a, irq=5
        powerspec 2  supports D0 D3  current D0
        map[10]: type 3, range 32, base f2000000, size 25, enabled
pcib1: (null) requested memory range 0xf2000000-0xf3ffffff: good
        map[14]: type 1, range 32, base f4000000, size 14, enabled
pcib1: (null) requested memory range 0xf4000000-0xf4003fff: good
        map[18]: type 1, range 32, base f5000000, size 23, enabled
pcib1: (null) requested memory range 0xf5000000-0xf57fffff: good
pcib0: matched entry for 0.1.INTA
pcib0: slot 1 INTA hardwired to IRQ 16
pcib1: slot 0 INTA is routed to irq 16
drm0: <Matrox G400/G450 (AGP)> mem 0xf2000000-0xf3ffffff,0xf4000000-0xf4003fff,0xf5000000-0xf57fffff irq 16 at device 0.0 on pci1
info: [drm] AGP at 0xf0000000 32MB
info: [drm] Initialized mga 3.1.0 20021029 on minor 0
isab0: <PCI-ISA bridge> at device 7.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <VIA 82C686B UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xd800-0xd80f at device 7.1 on pci0
atapci0: Reserved 0x10 bytes for rid 0x20 type 4 at 0xd800
ata0: <ATA channel 0> on atapci0
atapci0: Reserved 0x8 bytes for rid 0x10 type 4 at 0x1f0
atapci0: Reserved 0x1 bytes for rid 0x14 type 4 at 0x3f6
ata0: reset tp1 mask=03 ostat0=50 ostat1=50
ata0: stat0=0x50 err=0x01 lsb=0x00 msb=0x00
ata0: stat1=0x50 err=0x01 lsb=0x00 msb=0x00
ata0: reset tp2 stat0=50 stat1=50 devices=0x3<ATA_SLAVE,ATA_MASTER>
ata0: [MPSAFE]
ata1: <ATA channel 1> on atapci0
atapci0: Reserved 0x8 bytes for rid 0x18 type 4 at 0x170
atapci0: Reserved 0x1 bytes for rid 0x1c type 4 at 0x376
ata1: reset tp1 mask=03 ostat0=50 ostat1=50
ata1: stat0=0x00 err=0x01 lsb=0x14 msb=0xeb
ata1: stat1=0x50 err=0x01 lsb=0x00 msb=0x00
ata1: reset tp2 stat0=00 stat1=50 devices=0x6<ATAPI_MASTER,ATA_SLAVE>
ata1: [MPSAFE]
uhci0: <VIA 83C572 USB controller> port 0xd000-0xd01f irq 11 at device 7.2 on pci0
uhci0: Reserved 0x20 bytes for rid 0x20 type 4 at 0xd000
uhci0: [GIANT-LOCKED]
usb0: <VIA 83C572 USB controller> on uhci0
usb0: USB revision 1.0
uhub0: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
uhci1: <VIA 83C572 USB controller> port 0xd400-0xd41f irq 11 at device 7.3 on pci0
uhci1: Reserved 0x20 bytes for rid 0x20 type 4 at 0xd400
uhci1: [GIANT-LOCKED]
usb1: <VIA 83C572 USB controller> on uhci1
usb1: USB revision 1.0
uhub1: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1: 2 ports with 2 removable, self powered
pci0: <bridge> at device 7.4 (no driver attached)
pcib2: <PCI-PCI bridge> at device 9.0 on pci0
pcib2:   secondary bus     2
pcib2:   subordinate bus   2
pcib2:   I/O decode        0xc000-0xcfff
pcib2:   memory decode     0xf7000000-0xf8ffffff
pcib2:   prefetched decode 0xfa800000-0xfa8fffff
pci2: <PCI bus> on pcib2
pci2: physical bus=2
found-> vendor=0x8086, dev=0x1229, revid=0x05
        bus=2, slot=4, func=0
        class=02-00-00, hdrtype=0x00, mfdev=0
        cmdreg=0x0007, statreg=0x0290, cachelnsz=8 (dwords)
        lattimer=0x20 (960 ns), mingnt=0x08 (2000 ns), maxlat=0x38 (14000 ns)
        intpin=a, irq=10
        powerspec 1  supports D0 D1 D2 D3  current D0
        map[10]: type 3, range 32, base fa800000, size 12, enabled
pcib2: (null) requested memory range 0xfa800000-0xfa800fff: good
        map[14]: type 4, range 32, base 0000c000, size  5, enabled
pcib2: (null) requested I/O range 0xc000-0xc01f: in range
        map[18]: type 1, range 32, base f8000000, size 20, enabled
pcib2: (null) requested memory range 0xf8000000-0xf80fffff: good
pcib0: matched entry for 0.9.INTA
pcib0: slot 9 INTA hardwired to IRQ 18
pcib2: slot 4 INTA is routed to irq 18
found-> vendor=0x8086, dev=0x1229, revid=0x05
        bus=2, slot=5, func=0
        class=02-00-00, hdrtype=0x00, mfdev=0
        cmdreg=0x0007, statreg=0x0290, cachelnsz=8 (dwords)
        lattimer=0x20 (960 ns), mingnt=0x08 (2000 ns), maxlat=0x38 (14000 ns)
        intpin=a, irq=11
        powerspec 1  supports D0 D1 D2 D3  current D0
        map[10]: type 3, range 32, base fa801000, size 12, enabled
pcib2: (null) requested memory range 0xfa801000-0xfa801fff: good
        map[14]: type 4, range 32, base 0000c400, size  5, enabled
pcib2: (null) requested I/O range 0xc400-0xc41f: in range
        map[18]: type 1, range 32, base f8100000, size 20, enabled
pcib2: (null) requested memory range 0xf8100000-0xf81fffff: good
pcib0: matched entry for 0.9.INTB
pcib0: slot 9 INTB hardwired to IRQ 19
pcib2: slot 5 INTA is routed to irq 19
fxp0: <Intel 82558 Pro/100 Ethernet> port 0xc000-0xc01f mem 0xfa800000-0xfa800fff,0xf8000000-0xf80fffff irq 18 at device 4.0 on pci2
fxp0: Reserved 0x1000 bytes for rid 0x10 type 3 at 0xfa800000
fxp0: using memory space register mapping
fxp0: PCI IDs: 8086 1229 0e11 b01f 0005
fxp0: Dynamic Standby mode is disabled
miibus0: <MII bus> on fxp0
inphy0: <i82555 10/100 media interface> on miibus0
inphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp0: bpf attached
fxp0: Ethernet address: 00:50:8b:68:a0:18
fxp0: [MPSAFE]
fxp1: <Intel 82558 Pro/100 Ethernet> port 0xc400-0xc41f mem 0xfa801000-0xfa801fff,0xf8100000-0xf81fffff irq 19 at device 5.0 on pci2
fxp1: Reserved 0x1000 bytes for rid 0x10 type 3 at 0xfa801000
fxp1: using memory space register mapping
fxp1: PCI IDs: 8086 1229 0e11 b01f 0005
fxp1: Dynamic Standby mode is disabled
miibus1: <MII bus> on fxp1
inphy1: <i82555 10/100 media interface> on miibus1
inphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp1: bpf attached
fxp1: Ethernet address: 00:50:8b:68:a0:19
fxp1: [MPSAFE]
pcib3: <PCI-PCI bridge> at device 10.0 on pci0
pcib3:   secondary bus     3
pcib3:   subordinate bus   3
pcib3:   I/O decode        0xf000-0xfff
pcib3:   memory decode     0xfff00000-0xfffff
pcib3:   prefetched decode 0xfff00000-0xfffff
pci3: <PCI bus> on pcib3
pci3: physical bus=3
mlx0: <Mylex version 4 RAID interface> mem 0xfa000000-0xfa7fffff irq 19 at device 10.1 on pci0
mlx0: Reserved 0x800000 bytes for rid 0x10 type 3 at 0xfa000000
mlx0: [GIANT-LOCKED]
mlx0: DAC960PTL1, 1 channel, firmware 4.08-0-33, 8MB RAM
mlx0:   Hardware ID                 0x03020116
mlx0:   Firmware ID                 0x30210804
mlx0:   Configured/Actual channels  1/1
mlx0:   Max Targets                 16
mlx0:   Max Tags                    252
mlx0:   Max System Drives           32
mlx0:   Max Arms                    8
mlx0:   Max Spans                   4
mlx0:   DRAM/cache/flash/NVRAM size 8388608/6684672/524288/32768
mlx0:   DRAM type                   17
mlx0:   Clock Speed                 40ns
mlx0:   Hardware Speed              360ns
mlx0:   Max Commands                124
mlx0:   Max SG Entries              33
mlx0:   Max DP                      504
mlx0:   Max IOD                     1024
mlx0:   Max Comb                    2048
mlx0:   Latency                     12s
mlx0:   SCSI Timeout                6s
mlx0:   Min Free Lines              18
mlx0:   Rate Constant               50
mlx0:   MAXBLK                      1024
mlx0:   Blocking Factor             8 sectors
mlx0:   Cache Line Size             128 blocks
mlx0:   SCSI Capability             40MHz, 16 bit
mlx0:   Firmware Build Number       0
mlx0:   Fault Management Type       4
mlx0:   Features                    0
mlxd0: <Mylex System Drive> on mlx0
mlxd0: 34732MB (71131136 sectors) RAID 5 (online)
ath0: <Atheros 5212> mem 0xfa900000-0xfa90ffff irq 16 at device 11.0 on pci0
ath0: Reserved 0x10000 bytes for rid 0x10 type 3 at 0xfa900000
ath0: [MPSAFE]
ath0: bpf attached
ath0: Ethernet address: 00:09:5b:84:d0:e2
ath0: bpf attached
ath0: bpf attached
ath0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps
ath0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
ath0: mac 5.6 phy 4.1 radio 1.7
ath0: Use hw queue 1 for WME_AC_BE traffic
ath0: Use hw queue 0 for WME_AC_BK traffic
ath0: Use hw queue 2 for WME_AC_VI traffic
ath0: Use hw queue 3 for WME_AC_VO traffic
ath0: Use hw queue 8 for CAB traffic
ath0: Use hw queue 9 for beacons
pcm0: <Creative EMU10K1> port 0xdc00-0xdc1f irq 17 at device 12.0 on pci0
pcm0: Reserved 0x20 bytes for rid 0x10 type 4 at 0xdc00
emu: setmap (376000, 800), nseg=1, error=0
emu: setmap (375000, 1000), nseg=1, error=0
pcm0: <TriTech TR28602 AC97 Codec (id = 0x54524123)>
pcm0: Codec features 5 bit master volume, no 3D Stereo Enhancement
pcm0: [MPSAFE]
emu: setmap (2f8000, 1000), nseg=1, error=0
emu: setmap (21d000, 1000), nseg=1, error=0
emu: setmap (21b000, 1000), nseg=1, error=0
emu: setmap (299000, 1000), nseg=1, error=0
pcm0: sndbuf_setmap 297000, 1000; 0xc4d4d000 -> 297000
pcm0: sndbuf_setmap 215000, 1000; 0xc4d4b000 -> 215000
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: ic_type 90 part_id 80
fdc0: [MPSAFE]
fdc0: [FAST]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
sio0: irq maps: 0xc001 0xc011 0xc001 0xc001
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
sio0: type 16550A
sio1: irq maps: 0xc001 0xc009 0xc001 0xc001
sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
sio1: type 16550A
ppc0: using extended I/O port range
ppc0: EPP SPP
ppc0: <Standard parallel printer port> port 0x378-0x37f irq 7 on acpi0
ppc0: Generic chipset (EPP/NIBBLE) in COMPATIBLE mode
ppbus0: <Parallel port bus> on ppc0
plip0: <PLIP network interface> on ppbus0
plip0: bpf attached
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
psmcpnp0: <PS/2 mouse port> irq 12 on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
atkbd: the current kbd controller command byte 0047
atkbd: keyboard ID 0x41ab (2)
kbd0 at atkbd0
kbd0: atkbd0, AT 101/102 (2), config:0x0, flags:0x3d0000
atkbd0: [GIANT-LOCKED]
psm0: current command byte:0047
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model MouseMan+, device ID 0-70, 3 buttons
psm0: config:00000000, flags:00000008, packet size:3
psm0: syncmask:08, syncbits:00
ata: ata0 already exists; skipping it
ata: ata1 already exists; skipping it
atkbdc: atkbdc0 already exists; skipping it
fdc: fdc0 already exists; skipping it
ppc: ppc0 already exists; skipping it
sio: sio0 already exists; skipping it
sio: sio1 already exists; skipping it
pnp_identify: Trying Read_Port at 203
pnp_identify: Trying Read_Port at 243
pnp_identify: Trying Read_Port at 283
pnp_identify: Trying Read_Port at 2c3
pnp_identify: Trying Read_Port at 303
pnp_identify: Trying Read_Port at 343
pnp_identify: Trying Read_Port at 383
pnp_identify: Trying Read_Port at 3c3
PNP Identify complete
sc: sc0 already exists; skipping it
vga: vga0 already exists; skipping it
isa_probe_children: disabling PnP devices
isa_probe_children: probing non-PnP devices
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc8fff,0xcc000-0xce7ff on isa0
adv0: not probed (disabled)
aha0: not probed (disabled)
aic0: not probed (disabled)
bt0: not probed (disabled)
cs0: not probed (disabled)
ed0: not probed (disabled)
fe0: not probed (disabled)
ie0: not probed (disabled)
lnc0: not probed (disabled)
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sc0: fb0, kbd0, terminal emulator: sc (syscons terminal)
sio2: not probed (disabled)
sio3: not probed (disabled)
sn0: not probed (disabled)
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
vt0: not probed (disabled)
isa_probe_children: probing PnP devices
Device configuration finished.
Reducing kern.maxvnodes 102175 -> 100000
lapic: Divisor 2, Frequency 66644466 hz
Timecounter "TSC" frequency 1399537802 Hz quality -100
Timecounters tick every 1.200 msec
crypto: <crypto device>
crypto: assign driver 0, flags 6
crypto: driver 0 registers alg 1 flags 0 maxoplen 0
crypto: driver 0 registers alg 2 flags 0 maxoplen 0
crypto: driver 0 registers alg 3 flags 0 maxoplen 0
crypto: driver 0 registers alg 4 flags 0 maxoplen 0
crypto: driver 0 registers alg 5 flags 0 maxoplen 0
crypto: driver 0 registers alg 17 flags 0 maxoplen 0
crypto: driver 0 registers alg 6 flags 0 maxoplen 0
crypto: driver 0 registers alg 7 flags 0 maxoplen 0
crypto: driver 0 registers alg 15 flags 0 maxoplen 0
crypto: driver 0 registers alg 8 flags 0 maxoplen 0
crypto: driver 0 registers alg 16 flags 0 maxoplen 0
crypto: driver 0 registers alg 9 flags 0 maxoplen 0
crypto: driver 0 registers alg 10 flags 0 maxoplen 0
crypto: driver 0 registers alg 13 flags 0 maxoplen 0
crypto: driver 0 registers alg 14 flags 0 maxoplen 0
crypto: driver 0 registers alg 11 flags 0 maxoplen 0
crypto: driver 0 registers alg 18 flags 0 maxoplen 0
Linux ELF exec handler installed
Fast IPsec: Initialized Security Association Processing.
lo0: bpf attached
pflog0: bpf attached
pfsync0: bpf attached
GEOM: new disk mlxd0
ata0-slave: pio=PIO4 wdma=WDMA2 udma=UDMA100 cable=80 wire
ata0-master: pio=PIO4 wdma=WDMA2 udma=UDMA100 cable=80 wire
atapicam: atapicam0 already exists; skipping it
ad0: setting PIO4 on VIA 82C686B chip
ad0: setting UDMA100 on VIA 82C686B chip
ad0: 58644MB <Maxtor 5T060H6 TAH71DP0> at ata0-master UDMA100
ad0: 120103200 sectors [119150C/16H/63S] 16 sectors/interrupt 1 depth queue
ad1: setting PIO4 on VIA 82C686B chip
ad1: setting UDMA100 on VIA 82C686B chip
ad1: 152627MB <WDC WD1600BB-98DWA0 15.05R15> at ata0-slave UDMA100
ad1: 312581808 sectors [310101C/16H/63S] 16 sectors/interrupt 1 depth queue
ata1-slave: pio=PIO4 wdma=WDMA2 udma=UDMA100 cable=80 wire
ata1-master: pio=PIO4 wdma=WDMA2 udma=UDMA33 cable=40 wire
acd0: setting PIO4 on VIA 82C686B chip
acd0: setting UDMA33 on VIA 82C686B chip
acd0: <PLEXTOR DVDR PX-712A/1.04> DVDR drive at ata1 as master
acd0: read 6890KB/s (6890KB/s) write 8268KB/s (8268KB/s), 8192KB buffer, UDMA33
acd0: Reads: CDR, CDRW, CDDA stream, DVDROM, DVDR, packet
acd0: Writes: CDR, CDRW, DVDR, test write, burnproof
acd0: Audio: play, 256 volume levels
acd0: Mechanism: ejectable tray, unlocked
acd0: Medium: no/blank disc
ad3: setting PIO4 on VIA 82C686B chip
ad3: setting UDMA100 on VIA 82C686B chip
ad3: 152627MB <WDC WD1600BB-22GUA0 08.02D08> at ata1-slave UDMA100
ad3: 312581808 sectors [310101C/16H/63S] 16 sectors/interrupt 1 depth queue
SMP: AP CPU #1 Launched!
cpu1 AP:
     ID: 0x01000000   VER: 0x00040011 LDR: 0x02000000 DFR: 0x0fffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000200ef therm: 0x00000000 err: 0x00010000 pcm: 0x00010000
ioapic0: routing intpin 1 (ISA IRQ 1) to cluster 0
ioapic0: routing intpin 3 (ISA IRQ 3) to cluster 0
ioapic0: routing intpin 4 (ISA IRQ 4) to cluster 0
ioapic0: routing intpin 6 (ISA IRQ 6) to cluster 0
ioapic0: routing intpin 7 (ISA IRQ 7) to cluster 0
ioapic0: routing intpin 9 (ISA IRQ 9) to cluster 0
ioapic0: routing intpin 11 (ISA IRQ 11) to cluster 0
ioapic0: routing intpin 12 (ISA IRQ 12) to cluster 0
ioapic0: routing intpin 13 (ISA IRQ 13) to cluster 0
ioapic0: routing intpin 14 (ISA IRQ 14) to cluster 0
ioapic0: routing intpin 15 (ISA IRQ 15) to cluster 0
ioapic0: routing intpin 16 (PCI IRQ 16) to cluster 0
ioapic0: routing intpin 17 (PCI IRQ 17) to cluster 0
ioapic0: routing intpin 18 (PCI IRQ 18) to cluster 0
ioapic0: routing intpin 19 (PCI IRQ 19) to cluster 0
Expensive timeout(9) function: 0xc06c1540(0) 0.006520381 s
GEOM: new disk ad0
GEOM: new disk ad1
GEOM: new disk ad3
Trying to mount root from ufs:/dev/mlxd0s2a

--
| /"\   ASCII ribbon   |  GnuPG Key ID | c7e4 d91d 64e2 6321 9988 |
| \ / campaign against |    0xb248b614 | f27a 4e5b 06ce b248 b614 |
|  X    HTML in email  |       .the next sentence is a lie.       |
| / \     and news     |     .the previous sentence was true.     |

. (260 bytes) Download Attachment
attachment1 (194 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: RELENG_6 vm_fault panic on filesystem mount

Xin LI-5
Hi, Joerg,

On 11/19/05, Joerg Pernfuss <[hidden email]> wrote:
[...]

> From the output of the backtrace (below) I figure the problem for me
> lies in /usr/src/sys/ufs/ufs/ufs_dirhash.c line 232; this block:
>
>      if (ep->d_reclen == 0 || ep->d_reclen >
>          DIRBLKSIZ - (pos & (DIRBLKSIZ - 1))) {
>              /* Corrupted directory. */
>              brelse(bp);
>              goto fail;
>      }
>
> Which is consistent with the very first panic message I got saying
> something about 'Fatal Trap 12, page fault while in kernel mode,
>  ufs_dirhash: bad dir' - sadly, I didn't write down that one.
> Hasn't reappeared since then.
Unfortunately the "ufs_dirhash: bad dir" in most cases indicate some
bugs elsewhere, or some hardware hazard.  I have recently upgraded
several production boxes at lab and suggested many other guys to
upgrade to 6.0-RELEASE, and I am very eager to see if there is some
problems that we can catch and fix.

If this is a production then my suggestion would be trying to remove
something that you do not need for everyday use from the kernel
configuration, and if it is not then my suggestion would be enabling
INVARIANTS and makeoptions=-g to see if something strange happen, and
possibly some backtraces would inspire us to catch the bug.

Thanks,
--
Xin LI <[hidden email]> http://www.delphij.net

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: RELENG_6 vm_fault panic on filesystem mount

Joerg Pernfuss
On Sat, 19 Nov 2005 01:58:46 +0800
Xin LI <[hidden email]> wrote:

> Hi, Joerg,
>
> If this is a production then my suggestion would be trying to remove
> something that you do not need for everyday use from the kernel
> configuration, and if it is not then my suggestion would be enabling
> INVARIANTS and makeoptions=-g to see if something strange happen, and
> possibly some backtraces would inspire us to catch the bug.
>
> Thanks,

Hi,

interestingly it doesn't show up in the verbose dmesg, but curently
my kernel is configured with:

makeoptions     DEBUG=-g
options         ADAPTIVE_GIANT
options         MUTEX_DEBUG
options         WITNESS
options         GDB
options         SYSCTL_DEBUG
options         DEBUG_MEMGUARD
options         KTRACE
options         KTRACE_REQUEST_POOL=101
options         INVARIANTS
options         INVARIANT_SUPPORT
options         DIAGNOSTIC
options         KDB_STOP_NMI

I also do boot /boot/kernel/kernel.debug. As for the backtraces, I have
a few of them, but they are all identical to the one I sent in with my
first mail.

I'll add KDB, KDB_TRACE and DDB and look if I get different output.

Joerg

--
| /"\   ASCII ribbon   |  GnuPG Key ID | c7e4 d91d 64e2 6321 9988 |
| \ / campaign against |    0xb248b614 | f27a 4e5b 06ce b248 b614 |
|  X    HTML in email  |       .the next sentence is a lie.       |
| / \     and news     |     .the previous sentence was true.     |

attachment0 (194 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: RELENG_6 vm_fault panic on filesystem mount

Robert N. M. Watson-2
In reply to this post by Joerg Pernfuss

On Fri, 18 Nov 2005, Joerg Pernfuss wrote:

> #6  0xc078806a in calltrap () at /usr/src/sys/i386/i386/exception.s:139
> #7  0xc070c00c in ufsdirhash_build (ip=0xc5426948) at /usr/src/sys/ufs/ufs/ufs_dirhash.c:232
> #8  0xc070f5c3 in ufs_lookup (ap=0xeabb6824) at /usr/src/sys/ufs/ufs/ufs_lookup.c:192
> #9  0xc070d8f4 in ufs_extattr_lookup (start_dvp=0xc5414dd0, lockparent=0x2, dirname=0xd71f3000 "\002", vp=0xd71f3000, td=0xc5558780) at /usr/src/sys/
> ufs/ufs/ufs_extattr.c:274
> #10 0xc070dfd6 in ufs_extattr_autostart (mp=0xc4ca3000, td=0xc5558780) at /usr/src/sys/ufs/ufs/ufs_extattr.c:463
> #11 0xc0706fa6 in ffs_mount (mp=0xc4ca3000, td=0xc5558780) at /usr/src/sys/ufs/ffs/ffs_vfsops.c:779
> #12 0xc0640d57 in vfs_donmount (td=0xc5558780, fsflags=0x8008, fsoptions=0xeabb6bf4) at /usr/src/sys/kern/vfs_mount.c:739
> #13 0xc06427c0 in kernel_mount (ma=0xc5235240, flags=0x0) at pcpu.h:162

The UFS1 extended attribute code performs directory listings, lookups, and
file operations very early in the "life cycle" of a UFS file system in
order to identify attribute backing files.  We could be looking at a bug
or new negative interaction between the extended attribute code in UFS1,
dirhash, and the changes to VFS required to get SMP VFS support in 6.x.
In principle, however, the EA code waits until everything is "ready to go"
before starting on file system I/O.  Are you actively using UFS1
attributes on that file system?  Could I ask you to boot to single user
mode, try mounting the file system, then try compiling a kernel without
UFS_EXTATTR and UFS_EXTATTR_AUTOSTART, boot to single user mode, and see
if you can mount the file system successfully?  I.e., compare mounting
with and without extended attributes, but on a "quiet" file system so any
existing extended attributes remain in sync.

Robert N M Watson
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: RELENG_6 vm_fault panic on filesystem mount

Joerg Pernfuss
On Fri, 18 Nov 2005 18:59:34 +0000 (GMT)
Robert Watson <[hidden email]> wrote:

>
> On Fri, 18 Nov 2005, Joerg Pernfuss wrote:
>
> > #6  0xc078806a in calltrap ()
> > at /usr/src/sys/i386/i386/exception.s:139 #7  0xc070c00c in
> > ufsdirhash_build (ip=0xc5426948)
> > at /usr/src/sys/ufs/ufs/ufs_dirhash.c:232 #8  0xc070f5c3 in
> > ufs_lookup (ap=0xeabb6824) at /usr/src/sys/ufs/ufs/ufs_lookup.c:192
> > #9  0xc070d8f4 in ufs_extattr_lookup (start_dvp=0xc5414dd0,
> > lockparent=0x2, dirname=0xd71f3000 "\002", vp=0xd71f3000,
> > td=0xc5558780) at /usr/src/sys/ ufs/ufs/ufs_extattr.c:274 #10
> > 0xc070dfd6 in ufs_extattr_autostart (mp=0xc4ca3000, td=0xc5558780)
> > at /usr/src/sys/ufs/ufs/ufs_extattr.c:463 #11 0xc0706fa6 in
> > ffs_mount (mp=0xc4ca3000, td=0xc5558780)
> > at /usr/src/sys/ufs/ffs/ffs_vfsops.c:779 #12 0xc0640d57 in
> > vfs_donmount (td=0xc5558780, fsflags=0x8008, fsoptions=0xeabb6bf4)
> > at /usr/src/sys/kern/vfs_mount.c:739 #13 0xc06427c0 in kernel_mount
> > (ma=0xc5235240, flags=0x0) at pcpu.h:162
>
> The UFS1 extended attribute code performs directory listings,
> lookups, and file operations very early in the "life cycle" of a UFS
> file system in order to identify attribute backing files.  We could
> be looking at a bug or new negative interaction between the extended
> attribute code in UFS1, dirhash, and the changes to VFS required to
> get SMP VFS support in 6.x. In principle, however, the EA code waits
> until everything is "ready to go" before starting on file system
> I/O.  Are you actively using UFS1 attributes on that file system?
No, it is just my nfs exported distfiles collection. Nothing special
I was aware of until recently.

> Could I ask you to boot to single user mode, try mounting the file
> system, then try compiling a kernel without UFS_EXTATTR and
> UFS_EXTATTR_AUTOSTART, boot to single user mode, and see if you can
> mount the file system successfully?  I.e., compare mounting with and
> without extended attributes, but on a "quiet" file system so any
> existing extended attributes remain in sync.

No problem, the kernel is already building.

I'll post the output as soon as I have it.

Joerg
--
| /"\   ASCII ribbon   |  GnuPG Key ID | c7e4 d91d 64e2 6321 9988 |
| \ / campaign against |    0xb248b614 | f27a 4e5b 06ce b248 b614 |
|  X    HTML in email  |       .the next sentence is a lie.       |
| / \     and news     |     .the previous sentence was true.     |

attachment0 (194 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: RELENG_6 vm_fault panic on filesystem mount

Joerg Pernfuss
In reply to this post by Robert N. M. Watson-2
On Fri, 18 Nov 2005 18:59:34 +0000 (GMT)
Robert Watson <[hidden email]> wrote:

> Could I ask you to boot to single user mode, try mounting the file
> system, then try compiling a kernel without UFS_EXTATTR and
> UFS_EXTATTR_AUTOSTART, boot to single user mode, and see if you can
> mount the file system successfully?  I.e., compare mounting with and
> without extended attributes, but on a "quiet" file system so any
> existing extended attributes remain in sync.

Alright, I built two kernels, both with makeoptions DEBUG=-g,
ADAPTIVE_GIANT, MUTEX_DEBUG, WITNESS, KDB, KDB_TRACE, DDB, DDB_NUMSYM,
GDB, SYSCTL_DEBUG, DEBUG_MEMGUARD, KTRACE, KTRACE_REUQEST_POOL=101,
INVARIANTS, INVARIANTS_SUPPORT, DIAGNOSTIC, KDB_STOP_NMI.

One of them with UFS_ACLS, UFS_EXTATTR, UFS_EXTATTR_AUTOSTART and the
other without (both with UFS_DIRHASH).

Booting the one without the extattr options, I was able to mount the
filesystem without getting a panic, but encountered a LOR upon
umount.


lock order reversal:
 1st 0xc50416dc vnode interlock (vnode interlock) @
         /usr/src/sys/kern/vfs_subr.c:2430
 2nd 0xc1060144 system map (system map) @ /usr/src/sys/vm/vm_kern.c:295
KDB: stack acktrace:
witness_checkorder(c1060144,9,c081f0f8,127,eaad4990) at 0xc060f8df =
        witness_checkorder+0x5bf
_mtx_lock_flags(c1060144,0,c081f0f8,127,321) at 0xc05da594 =
        _mtx_lock_flags+0x54
_vm_map_lock(c10600c0,c081f0f8,127,8,c106c468) at 0xc072a8e5 =
        _vm_map_lock+0x35
kmem_malloc(c10600c0,1000,101,101,8) at 0xc0729cdc = kmem_malloc+0x3c
slab_zalloc(9,c081e180,8a2,c4fae780,c4fae7f8) at 0xc07201d2 =
  slab_zalloc+0x82
uma_zone_slab(c106c468,8,c081e180,8a2,0) at 0xc072071e =
  uma_zalloc_internal+0x3e
bucket_alloc(c10440a8,0,c081e180,95e,c10440a0) at 0xc0720c09 =
        bucket_alloc+0x29
uma_zfree_arg(c1042000,c503121c,0,eaad4ae4,c06e98f8) at 0xc0721d86 =
        uma_zfree_arg+0x2d6
mac_labelzone_free(c503121c,c5041660,eaad4b00,c064b9fe,c5041660)
        at 0xc06e04d0 = mac_labelzone_free+0x20
mac_destroy_vnode(c5041660,0,c080fb66,2ff,c080fb66) at 0xc06e98f8 =
        mac_destroy_vnode+0x18
vdropl(c0849ee0,eaad4b28,c080fb66,8e8,c4f87444) at 0xc064b9fe =
        vdropl+0x11e
vflush(c4f87400,0,0,c4fae780,c081ccbc) at 0xc064da58 = vflush+0x378
ffs_flushfiles(c4f87400,0,c4fae780,c070b376,0) at 0xc070a19a =
        ffs_flushfiles+0x4a
softdep_flushfiles(c4f87400,0,c4fae780,c4f6ea00,0) at 0xc07096f3 =
        softdep_flushfiles+0x33
ffs_unmount(c4f87400,8000000,c4fae780,c4fae780,0) at 0xc070b470 =
        ffs_unmount+0x40
dounmount(c4f87400,8000000,c4fae780,37e,42262023) at 0xc0647b97 =
        dounmount+0x1e7
unmount(c4fae780,eaad4d04,c0828274,3c6,2) at 0xc0648091 =
        unmount+0x211
syscall(3b,3b,3b,804aa92,804d6a1) at 0xc07a40ec = syscall+0x14c
Xint0x80_syscall() at 0xc078e9df = Xint0x80_syscall+0x1f
--- syscall (22, FreeBSD ELF32, unmount), eip = 0x480c1c3f,
        esp = 0xbfbfe40c, ebp = 0xbfbfe4c8 ---


Booting the kernel with the extattr options, the system panic'ed
when I tried to mount the filesystem.


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address   = 0xd76f7004
fault code              = supervisor read, page not present
instruction pointer     = 0x20:0xc0712adc
stack pointer           = 0x28:0xe5e10680
frame pointer           = 0x28:0xe5e106d8
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, press 1, def32 1, gran 1
processor flags         = interrupt enabled, resume, IOPL = 0
current process         = 657 (mount)
[thread pid 657 tid 100055 ]
Stopped at      0xc0712adc = ufs_dirhash_build+0x6ac:
    movzwl  0x4(%ecx),%ebx
db> trace
Tracing pid 657 tid 100055 td 0xc4c93d80
ufsdirhash_build(c53b38c4,c4c93d80,c1044b48,351,e5e1070c) at 0xc0712adc =
  ufs_dirhash_build+0x6ac
ufs_lookup(e5e10824,c4f8f000,400,e5e10810,0) at 0xc0716093 =
        ufs_lookup+0xf3
ufs_extattr_lookup(c08204f9,e5e10860,c4c93d80,c4c93d80,c5246800)
        at 0xc07143c4 = ufs_extattr_lookup+0xf4
ufs_extattr_autostart(c4d41400,c4c93d80,c559f000,c558c300,c558c300)
        at 0xc0714aa6 = ufs_extattr_autostart+0x76
ffs_mount(c4d41400,c4v93d80,e5e10acc,2c7,0) at 0xc070da76 =
        ffs_mount+0x2066
vfs_donmount(e5e10bf4,e5e10d04,c4f9cd00,e,c0648ac2) at 0xc0647637 =
        vfs_donmount+0xb07
kernel_mount(c4f48840,0,e5e10c38,6c,bfbfeb96) at 0xc06490a0 =
        kernel_mount+0xb0
ffs_cmount(c4f48840,bfbfddc0,0,c4c93d80,0) at 0xc070a16d =
        ffs_cmount+0x8d
mount(c4c93d80,e5e10d04,c082b2a7,3c6,4) at 0xc0648de5 = mount+0x175
syscall(3b,3b,3b,bfbfddbc,bfbfe854) at 0xc07a6c2c = syscall+0x14c
Xint0x80_syscall() at 0xc079151f = Xint0x80_syscall+0x1f
--- syscall (21, FreeBSD ELF32, mount), eip = 0x480c2c5f,
        esp = 0xbfbfdd9c, ebp = 0xbfbfde48 ---
db> show alllocks
Process 657 (mount) thread 0xc4c93d80 (100055)
exclusive sleep mutex Giant r = 1 (0xc0887780) locked @
        /usr/src/sys/kern/vfs_lookup.c:197
db>

The fs was fsck'ed before each mounted and always reported clean.
I use 'src/sys/ufs/ufs/ufs_extattr.c,v 1.81.2.1 2005/10/15 18:32:55'.

I hope this new information helps you.

Joerg

--
| /"\   ASCII ribbon   |  GnuPG Key ID | c7e4 d91d 64e2 6321 9988 |
| \ / campaign against |    0xb248b614 | f27a 4e5b 06ce b248 b614 |
|  X    HTML in email  |       .the next sentence is a lie.       |
| / \     and news     |     .the previous sentence was true.     |

attachment0 (194 bytes) Download Attachment