[Bug 252608] [panic] [usb] [if_ure] Kernel fault on ure disconnect

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

[Bug 252608] [panic] [usb] [if_ure] Kernel fault on ure disconnect

bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252608

            Bug ID: 252608
           Summary: [panic] [usb] [if_ure] Kernel fault on ure disconnect
           Product: Base System
           Version: CURRENT
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: usb
          Assignee: [hidden email]
          Reporter: [hidden email]

My USB-C dock with Realtek NIC losts power for a second, but that was enough to
panic the entire system...

Jan 10 11:01:11 Fryzen495 kernel: ugen1.2: <VIA Labs, Inc. USB2.0 Hub> at
usbus1 (disconnected)
Jan 10 11:01:11 Fryzen495 kernel: uhub3: at uhub1, port 1, addr 1
(disconnected)
Jan 10 11:01:11 Fryzen495 kernel: ugen1.3: <Realtek USB-C Dock Ethernet> at
usbus1 (disconnected)
Jan 10 11:01:11 Fryzen495 kernel: ure0: at uhub3, port 1, addr 2 (disconnected)
Jan 10 11:02:58 Fryzen495 syslogd: kernel boot file is /boot/kernel/kernel
Jan 10 11:02:58 Fryzen495 kernel: panic: vm_fault_lookup: fault on nofault
entry, addr: 0xfffffe00fa8a1000
Jan 10 11:02:58 Fryzen495 kernel: cpuid = 3
Jan 10 11:02:58 Fryzen495 kernel: time = 1610272871
Jan 10 11:02:58 Fryzen495 kernel: KDB: stack backtrace:
Jan 10 11:02:58 Fryzen495 kernel: db_trace_self_wrapper() at
db_trace_self_wrapper+0x2b/frame 0xfffffe00b4b635a0
Jan 10 11:02:58 Fryzen495 kernel: vpanic() at vpanic+0x181/frame
0xfffffe00b4b635f0
Jan 10 11:02:58 Fryzen495 kernel: panic() at panic+0x43/frame
0xfffffe00b4b63650
Jan 10 11:02:58 Fryzen495 kernel: vm_fault() at vm_fault+0x1331/frame
0xfffffe00b4b63750
Jan 10 11:02:58 Fryzen495 kernel: vm_fault_trap() at vm_fault_trap+0xb1/frame
0xfffffe00b4b637a0
Jan 10 11:02:58 Fryzen495 kernel: trap_pfault() at trap_pfault+0x1f6/frame
0xfffffe00b4b63800
Jan 10 11:02:58 Fryzen495 kernel: trap() at trap+0x27d/frame 0xfffffe00b4b63910
Jan 10 11:02:58 Fryzen495 kernel: calltrap() at calltrap+0x8/frame
0xfffffe00b4b63910
Jan 10 11:02:58 Fryzen495 kernel: --- trap 0xc, rip = 0xffffffff80686ecd, rsp =
0xfffffe00b4b639e0, rbp = 0xfffffe00b4b639e0 ---
Jan 10 11:02:58 Fryzen495 kernel: memcpy_std() at memcpy_std+0x9d/frame
0xfffffe00b4b639e0
Jan 10 11:02:58 Fryzen495 kernel: usbd_copy_in() at usbd_copy_in+0x4d/frame
0xfffffe00b4b63a20
Jan 10 11:02:58 Fryzen495 kernel: ure_bulk_write_callback() at
ure_bulk_write_callback+0x43f/frame 0xfffffe00b4b63ad0
Jan 10 11:02:58 Fryzen495 kernel: usbd_callback_wrapper() at
usbd_callback_wrapper+0x6df/frame 0xfffffe00b4b63b30
Jan 10 11:02:58 Fryzen495 kernel: usb_command_wrapper() at
usb_command_wrapper+0xb5/frame 0xfffffe00b4b63b50
Jan 10 11:02:58 Fryzen495 kernel: usb_callback_proc() at
usb_callback_proc+0xb9/frame 0xfffffe00b4b63b70
Jan 10 11:02:58 Fryzen495 kernel: usb_process() at usb_process+0x106/frame
0xfffffe00b4b63bb0
Jan 10 11:02:58 Fryzen495 kernel: fork_exit() at fork_exit+0x7d/frame
0xfffffe00b4b63bf0
Jan 10 11:02:58 Fryzen495 kernel: fork_trampoline() at
fork_trampoline+0xe/frame 0xfffffe00b4b63bf0
Jan 10 11:02:58 Fryzen495 kernel: --- trap 0, rip = 0, rsp = 0, rbp = 0 ---
Jan 10 11:02:58 Fryzen495 kernel: KDB: enter: panic

As you can see, the usb process is still running can called
ure_bulk_write_callback, but the ure0 device was detached previously. I got a
crash dump, please let me know if you need further data.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-usb
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 252608] [panic] [usb] [if_ure] Kernel fault on ure disconnect

bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252608

Hans Petter Selasky <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #1 from Hans Petter Selasky <[hidden email]> ---
There is likely a missing drain of the USB transfers.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-usb
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 252608] [panic] [usb] [if_ure] Kernel fault on ure disconnect

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252608

--- Comment #2 from Ali Abdallah <[hidden email]> ---
Do you think there is an easy way to avoid the panic?

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-usb
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 252608] [panic] [usb] [if_ure] Kernel fault on ure disconnect

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252608

--- Comment #3 from Hans Petter Selasky <[hidden email]> ---
I suspect "sc->sc_txavail" is not properly managed at shutdown and that freed
USB transfers are kicked alive again.

Adding jmg@

Likely it is just more safe to kick all the transfers when there is a packet
for TX and not have some many USB TX transfers.

--HPS

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-usb
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 252608] [panic] [usb] [if_ure] Kernel fault on ure disconnect

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252608

Hans Petter Selasky <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|New                         |In Progress

--- Comment #4 from Hans Petter Selasky <[hidden email]> ---
> Do you think there is an easy way to avoid the panic?

Yes,

Try the attached patch.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-usb
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 252608] [panic] [usb] [if_ure] Kernel fault on ure disconnect

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252608

--- Comment #5 from Hans Petter Selasky <[hidden email]> ---
Created attachment 221491
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=221491&action=edit
Fix race

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-usb
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 252608] [panic] [usb] [if_ure] Kernel fault on ure disconnect

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252608

--- Comment #6 from Ali Abdallah <[hidden email]> ---
(In reply to Hans Petter Selasky from comment #5)
Thanks very much for the patch, will give it a try and report back.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-usb
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 252608] [panic] [usb] [if_ure] Kernel fault on ure disconnect

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252608

--- Comment #7 from Ali Abdallah <[hidden email]> ---
It crashes as well with the patch from comment 5

--- trap 0xc, rip = 0xffffffff803b28f1, rsp = 0xfffffe00c83604c0, rbp =
0xfffffe00c8360540 ---                                                          
__mtx_lock_sleep() at __mtx_lock_sleep+0xd1/frame 0xfffffe00c8360540            
usbd_do_request_flags() at usbd_do_request_flags+0x740/frame 0xfffffe00c83605d0
usbd_do_request_proc() at usbd_do_request_proc+0x60/frame 0xfffffe00c8360630    
ure_miibus_readreg() at ure_miibus_readreg+0x11c/frame 0xfffffe00c83606a0      
rgephy_status() at rgephy_status+0x80/frame 0xfffffe00c83606f0                  
rgephy_service() at rgephy_service+0x417/frame 0xfffffe00c8360750              
mii_pollstat() at mii_pollstat+0x46/frame 0xfffffe00c8360780                    
ure_ifmedia_sts() at ure_ifmedia_sts+0x41/frame 0xfffffe00c83607b0              
ifmedia_ioctl() at ifmedia_ioctl+0x16a/frame 0xfffffe00c83607e0                
ifhwioctl() at ifhwioctl+0x2af/frame 0xfffffe00c83608e0                        
ifioctl() at ifioctl+0x2eb/frame 0xfffffe00c8360990                            
kern_ioctl() at kern_ioctl+0x26d/frame 0xfffffe00c8360a00                      
sys_ioctl() at sys_ioctl+0xf2/frame 0xfffffe00c8360ac0

Use after free of the if_ure mutex.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-usb
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 252608] [panic] [usb] [if_ure] Kernel fault on ure disconnect

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252608

--- Comment #8 from Hans Petter Selasky <[hidden email]> ---
Ali,

This is another issue, and is expected. Can we solve that separately?

Do you see any more TX-path panics?

--HPS

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-usb
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 252608] [panic] [usb] [if_ure] Kernel fault on ure disconnect

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252608

--- Comment #9 from Ali Abdallah <[hidden email]> ---
> This is another issue, and is expected. Can we solve that separately?

Yes absolutely, thanks.

> Do you see any more TX-path panics?

Not anymore (Will report here if I get another tx-path panic).

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-usb
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 252608] [panic] [usb] [if_ure] Kernel fault on ure disconnect

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252608

--- Comment #10 from Hans Petter Selasky <[hidden email]> ---
@bz: ether_ifdetach() nor if_free() is draining if_ioctl. We should have a
solution in place for all kernel drivers!

--HPS

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-usb
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 252608] [panic] [usb] [if_ure] Kernel fault on ure disconnect

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252608

--- Comment #11 from [hidden email] ---
A commit in branch main references this bug:

URL:
https://cgit.FreeBSD.org/src/commit/?id=6e5baec33c1032f4fbf713d601a34b4658b918a2

commit 6e5baec33c1032f4fbf713d601a34b4658b918a2
Author:     Hans Petter Selasky <[hidden email]>
AuthorDate: 2021-01-12 13:13:14 +0000
Commit:     Hans Petter Selasky <[hidden email]>
CommitDate: 2021-01-12 16:57:58 +0000

    Fix for use-after-free in if_ure(4) driver.

    When detaching the if_ure(4) driver, the TX active USB transfer array may
    point to freed USB transfers. Given that the number of USB transfers is
    very low, simply start all transfers every time there is a packet to
    keep safe from use-after-free.

    PR: 252608
    MFC after: 1 week
    Sponsored by: Mellanox Technologies // NVIDIA Networking

 sys/dev/usb/net/if_ure.c    | 49 ++++-----------------------------------------
 sys/dev/usb/net/if_urereg.h |  9 ---------
 2 files changed, 4 insertions(+), 54 deletions(-)

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-usb
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 252608] [panic] [usb] [if_ure] Kernel fault on ure disconnect

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252608

--- Comment #12 from Hans Petter Selasky <[hidden email]> ---
Created attachment 221501
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=221501&action=edit
Patch for second issue - please test

Ali,

Please test this patch!

--HPS

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-usb
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 252608] [panic] [usb] [if_ure] Kernel fault on ure disconnect

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252608

--- Comment #13 from Ali Abdallah <[hidden email]> ---
(In reply to Hans Petter Selasky from comment #12)
Thanks, really appreciated your quick responses.

With the patch from comment 12, it is still panicking with the following trace

--- trap 0xc, rip = 0xffffffff80e85515, rsp = 0xfffffe00eaf936b0, rbp =
0xfffffe00eaf936f0 ---
rgephy_status() at rgephy_status+0x95/frame 0xfffffe00eaf936f0
rgephy_service() at rgephy_service+0x417/frame 0xfffffe00eaf93750
mii_pollstat() at mii_pollstat+0x46/frame 0xfffffe00eaf93780
ure_ifmedia_sts() at ure_ifmedia_sts+0x44/frame 0xfffffe00eaf937b0
ifmedia_ioctl() at ifmedia_ioctl+0x16a/frame 0xfffffe00eaf937e0
ifhwioctl() at ifhwioctl+0x2af/frame 0xfffffe00eaf938e0
ifioctl() at ifioctl+0x2eb/frame 0xfffffe00eaf93990
kern_ioctl() at kern_ioctl+0x26d/frame 0xfffffe00eaf93a00
sys_ioctl() at sys_ioctl+0xf2/frame 0xfffffe00eaf93ac0

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-usb
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 252608] [panic] [usb] [if_ure] Kernel fault on ure disconnect

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252608

Hans Petter Selasky <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
 Attachment #221501|0                           |1
        is obsolete|                            |

--- Comment #14 from Hans Petter Selasky <[hidden email]> ---
Created attachment 221506
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=221506&action=edit
Patch for second issue - please test

Please test updated patch.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-usb
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 252608] [panic] [usb] [if_ure] Kernel fault on ure disconnect

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252608

--- Comment #15 from Ali Abdallah <[hidden email]> ---
(In reply to Hans Petter Selasky from comment #14)
I can confirm that my system is no longer panicking when the dock is unplugged.
Thanks!!!

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-usb
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 252608] [panic] [usb] [if_ure] Kernel fault on ure disconnect

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252608

Mark Linimon <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |panic

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-usb
To unsubscribe, send any mail to "[hidden email]"