panic: Assertion in_epoch(net_epoch_preempt) failed at ... src/sys/net/if.c:3694

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

panic: Assertion in_epoch(net_epoch_preempt) failed at ... src/sys/net/if.c:3694

David Wolfskill
This was on my laptop (build machine, which was updated in parallel,
but uses a different NIC, had no issues) after a src update from
r353176 to r353298; laptop's NIC is wlan(4) (which is iwn(4), in
this case).

The ddb "dump" command recorded a dump, which I managed to capture; the
information may be found at
http://www.catwhisker.org/~david/FreeBSD/head/r353298/

Here's the backtrace (from
http://www.catwhisker.org/~david/FreeBSD/head/r353298/core.txt.0):

panic: Assertion in_epoch(net_epoch_preempt) failed at /usr/src/sys/net/if.c:3694
cpuid = 3
time = 1570535588
KDB: stack backtrace:
db_trace_self_wrapper() at 0xffffffff8049ebab = db_trace_self_wrapper+0x2b/frame 0xfffffe103196a8b0
vpanic() at 0xffffffff80b979bd = vpanic+0x19d/frame 0xfffffe103196a900
panic() at 0xffffffff80b97753 = panic+0x43/frame 0xfffffe103196a960
if_delmulti_ifma_flags() at 0xffffffff80ca9231 = if_delmulti_ifma_flags+0x141/frame 0xfffffe103196a990
inm_release_task() at 0xffffffff80d3211c = inm_release_task+0x1ac/frame 0xfffffe103196a9f0
gtaskqueue_run_locked() at 0xffffffff80be1c59 = gtaskqueue_run_locked+0xf9/frame 0xfffffe103196aa40
gtaskqueue_thread_loop() at 0xffffffff80be1a18 = gtaskqueue_thread_loop+0x88/frame 0xfffffe103196aa70
fork_exit() at 0xffffffff80b55714 = fork_exit+0x84/frame 0xfffffe103196aab0
fork_trampoline() at 0xffffffff8101f28e = fork_trampoline+0xe/frame 0xfffffe103196aab0
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---
KDB: enter: panic


I was unable to boot from (yesterday's) kernel.old -- loader whined
something about a header mismatch (I think -- there was a fair bit
of text-overwriting).  I was able to extract the dump by rebooting
from the same kernel, but disabling the NIC first, then switching
to a vty, logging in, and I managed to run "sync" a few times before
it panicked again.

I expect that for tomorrow, I'll end up "cloning" the stable/12
slice over the top of the head slice, then upgrade from stable/12
to head again.

Peace,
david
--
David H. Wolfskill [hidden email]
I am amazed that anyone would condone what Trunp has publicly admitted he did.

See http://www.catwhisker.org/~david/publickey.gpg for my public key.

signature.asc (631 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: panic: Assertion in_epoch(net_epoch_preempt) failed at ... src/sys/net/if.c:3694

Julian Elischer-5
Is there a paper or good description of the epoch concept? (more
readable than epoch(9)).
Haven't been to enough BSD confs recently.

julian
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: panic: Assertion in_epoch(net_epoch_preempt) failed at ... src/sys/net/if.c:3694

Walter Parker
See the Wikipedia page at https://en.wikipedia.org/wiki/Unix_time


Walter

On Tue, Oct 8, 2019 at 2:37 PM Julian Elischer <[hidden email]> wrote:

> Is there a paper or good description of the epoch concept? (more
> readable than epoch(9)).
> Haven't been to enough BSD confs recently.
>
> julian
> _______________________________________________
> [hidden email] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "[hidden email]"
>


--
The greatest dangers to liberty lurk in insidious encroachment by men of
zeal, well-meaning but without understanding.   -- Justice Louis D. Brandeis
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: panic: Assertion in_epoch(net_epoch_preempt) failed at ... src/sys/net/if.c:3694

Matthew Macy-3
On Tue, Oct 8, 2019 at 14:42 Walter Parker <[hidden email]> wrote:

> See the Wikipedia page at https://en.wikipedia.org/wiki/Unix_time
>
>

Unrelated.
http://concurrencykit.org/slides.html

And see also Keir Fraser’s thesis where the idea originated.

>
> Walter
>
> On Tue, Oct 8, 2019 at 2:37 PM Julian Elischer <[hidden email]> wrote:
>
> > Is there a paper or good description of the epoch concept? (more
> > readable than epoch(9)).
> > Haven't been to enough BSD confs recently.
> >
> > julian
> > _______________________________________________
> > [hidden email] mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-current
> > To unsubscribe, send any mail to "
> [hidden email]"
> >
>
>
> --
> The greatest dangers to liberty lurk in insidious encroachment by men of
> zeal, well-meaning but without understanding.   -- Justice Louis D.
> Brandeis
> _______________________________________________
> [hidden email] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "[hidden email]"
>
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: panic: Assertion in_epoch(net_epoch_preempt) failed at ... src/sys/net/if.c:3694

Julian Elischer-5
On 10/8/19 2:45 PM, Matthew Macy wrote:

>
>
> On Tue, Oct 8, 2019 at 14:42 Walter Parker <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     See the Wikipedia page at https://en.wikipedia.org/wiki/Unix_time
>
>
>
> Unrelated.
> http://concurrencykit.org/slides.html
>
> And see also Keir Fraser’s thesis where the idea originated.
>
>
>     Walter
>
>     On Tue, Oct 8, 2019 at 2:37 PM Julian Elischer
>     <[hidden email] <mailto:[hidden email]>> wrote:
>
>     > Is there a paper or good description of the epoch concept? (more
>     > readable than epoch(9)).
>     > Haven't been to enough BSD confs recently.
>     >
>     > julian
>     > _______________________________________________
>     > [hidden email]
>     <mailto:[hidden email]> mailing list
>     > https://lists.freebsd.org/mailman/listinfo/freebsd-current
>     > To unsubscribe, send any mail to
>     "[hidden email]
>     <mailto:[hidden email]>"
>     >
>
>
>     --
>     The greatest dangers to liberty lurk in insidious encroachment
>     by men of
>     zeal, well-meaning but without understanding.   -- Justice Louis
>     D. Brandeis
>     _______________________________________________
>     [hidden email] <mailto:[hidden email]>
>     mailing list
>     https://lists.freebsd.org/mailman/listinfo/freebsd-current
>     To unsubscribe, send any mail to
>     "[hidden email]
>     <mailto:[hidden email]>"
>
thanks for the answers


_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: panic: Assertion in_epoch(net_epoch_preempt) failed at ... src/sys/net/if.c:3694

Julian Elischer-5
In reply to this post by Walter Parker
On 10/8/19 2:42 PM, Walter Parker wrote:
> See the Wikipedia page at https://en.wikipedia.org/wiki/Unix_time

but I was refering to the new "epoch" faciity in the kernel..  ( man 9
epoch  )

https://www.freebsd.org/cgi/man.cgi?query=epoch&apropos=0&sektion=9&manpath=FreeBSD+13-current&arch=default&format=html


>
>
> Walter
>
> On Tue, Oct 8, 2019 at 2:37 PM Julian Elischer <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Is there a paper or good description of the epoch concept? (more
>     readable than epoch(9)).
>     Haven't been to enough BSD confs recently.
>
>     julian
>     _______________________________________________
>     [hidden email] <mailto:[hidden email]>
>     mailing list
>     https://lists.freebsd.org/mailman/listinfo/freebsd-current
>     To unsubscribe, send any mail to
>     "[hidden email]
>     <mailto:[hidden email]>"
>
>
>
> --
> The greatest dangers to liberty lurk in insidious encroachment by
> men of zeal, well-meaning but without understanding.   -- Justice
> Louis D. Brandeis


_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: panic: Assertion in_epoch(net_epoch_preempt) failed at ... src/sys/net/if.c:3694

Matthew Macy-3
On Tue, Oct 8, 2019 at 15:01 Julian Elischer <[hidden email]> wrote:

> On 10/8/19 2:42 PM, Walter Parker wrote:
> > See the Wikipedia page at https://en.wikipedia.org/wiki/Unix_time
>
> but I was refering to the new "epoch" faciity in the kernel..  ( man 9
> epoch  )




That’s what one would assume given that the thread started with a panic in
said system. Epoch is built on concurrency kit’s EBR implementation. Hence
the link I shared.


>
>
> https://www.freebsd.org/cgi/man.cgi?query=epoch&apropos=0&sektion=9&manpath=FreeBSD+13-current&arch=default&format=html
>
>
> >
> >
> > Walter
> >
> > On Tue, Oct 8, 2019 at 2:37 PM Julian Elischer <[hidden email]
> > <mailto:[hidden email]>> wrote:
> >
> >     Is there a paper or good description of the epoch concept? (more
> >     readable than epoch(9)).
> >     Haven't been to enough BSD confs recently.
> >
> >     julian
> >     _______________________________________________
> >     [hidden email] <mailto:[hidden email]>
> >     mailing list
> >     https://lists.freebsd.org/mailman/listinfo/freebsd-current
> >     To unsubscribe, send any mail to
> >     "[hidden email]
> >     <mailto:[hidden email]>"
> >
> >
> >
> > --
> > The greatest dangers to liberty lurk in insidious encroachment by
> > men of zeal, well-meaning but without understanding.   -- Justice
> > Louis D. Brandeis
>
>
> _______________________________________________
> [hidden email] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "[hidden email]"
>
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: panic: Assertion in_epoch(net_epoch_preempt) failed at ... src/sys/net/if.c:3694

Ryan Stone-2
In reply to this post by Julian Elischer-5
I haven't found any good references on the subject, but here's my understanding:

- epoch_enter() and epoch_exit() are very inexpensive operations
(cheaper than mtx, rw_lock or rm_lock operations) that are use to mark
read-only critical sections
- epoch_wait() guarantees that no threads that were in the critical
section when it was first called are still in the critical section
when it completes

With this guarantee, you can safely destroy an object with the
following procedure:

1. Atomically remove all global pointers to the object (e.g. remove it
from any lists that the critical sections might look it up in).  This
must be done atomically because read-only threads can be concurrently
running in the critical section.  This guarantees that no more threads
can get a pointer to it.
2. Call epoch_wait() to drain all threads that already held pointers
to it before step 1.
3. You now hold the only pointer to the object, so you are free to
destroy it as you please.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: panic: Assertion in_epoch(net_epoch_preempt) failed at ... src/sys/net/if.c:3694

Julian Elischer-5
On 10/8/19 4:22 PM, Ryan Stone wrote:

> I haven't found any good references on the subject, but here's my understanding:
>
> - epoch_enter() and epoch_exit() are very inexpensive operations
> (cheaper than mtx, rw_lock or rm_lock operations) that are use to mark
> read-only critical sections
> - epoch_wait() guarantees that no threads that were in the critical
> section when it was first called are still in the critical section
> when it completes
>
> With this guarantee, you can safely destroy an object with the
> following procedure:
>
> 1. Atomically remove all global pointers to the object (e.g. remove it
> from any lists that the critical sections might look it up in).  This
> must be done atomically because read-only threads can be concurrently
> running in the critical section.  This guarantees that no more threads
> can get a pointer to it.
> 2. Call epoch_wait() to drain all threads that already held pointers
> to it before step 1.
> 3. You now hold the only pointer to the object, so you are free to
> destroy it as you please.

Ok thanks. Gottit.  thanks for the description.



_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: panic: Assertion in_epoch(net_epoch_preempt) failed at ... src/sys/net/if.c:3694

David Wolfskill
In reply to this post by David Wolfskill
On Tue, Oct 08, 2019 at 05:15:19AM -0700, David Wolfskill wrote:
> This was on my laptop (build machine, which was updated in parallel,
> but uses a different NIC, had no issues) after a src update from
> r353176 to r353298; laptop's NIC is wlan(4) (which is iwn(4), in
> this case).

After restoring the /boot/*.old set, I was able to reboot & update from
r353298 to r353336.  The subsequent reboot seemed OK ... until the
keyboard & mouse were enabled (under X11; I use xdm on the laptop).

Then I got a slightly different panic:

panic: Assertion in_epoch(net_epoch_preempt) failed at /usr/src/sys/net/if.c:356

Backtrace:

panic: Assertion in_epoch(net_epoch_preempt) failed at /usr/src/sys/net/if.c:356
cpuid = 0
time = 1570621553
KDB: stack backtrace:
db_trace_self_wrapper() at 0xffffffff8049ebab = db_trace_self_wrapper+0x2b/frame 0xfffffe1072956580
vpanic() at 0xffffffff80b978fd = vpanic+0x19d/frame 0xfffffe10729565d0
panic() at 0xffffffff80b97693 = panic+0x43/frame 0xfffffe1072956630
if_ref() at 0xffffffff80ca1b10 = if_ref/frame 0xfffffe1072956650
sysctl_ifdata() at 0xffffffff80cb247d = sysctl_ifdata+0x4d/frame 0xfffffe1072956760
sysctl_root_handler_locked() at 0xffffffff80ba714b = sysctl_root_handler_locked+0x7b/frame 0xfffffe10729567a0
sysctl_root() at 0xffffffff80ba64dc = sysctl_root+0x20c/frame 0xfffffe1072956820
userland_sysctl() at 0xffffffff80ba6bab = userland_sysctl+0x17b/frame 0xfffffe10729568d0
sys___sysctl() at 0xffffffff80ba69ef = sys___sysctl+0x5f/frame 0xfffffe1072956980
amd64_syscall() at 0xffffffff810480f4 = amd64_syscall+0x2d4/frame 0xfffffe1072956ab0
fast_syscall_common() at 0xffffffff8101eaa0 = fast_syscall_common+0x101/frame 0xfffffe1072956ab0
--- syscall (202, FreeBSD ELF64, sys___sysctl), rip = 0x8014d4d2a, rsp = 0x7fffffffeaa8, rbp = 0x7fffffffeae0 ---
KDB: enter: panic

> The ddb "dump" command recorded a dump, which I managed to capture; the
> information may be found at
> http://www.catwhisker.org/~david/FreeBSD/head/r353298/

This one is at http://www.catwhisker.org/~david/FreeBSD/head/r353336

As before, the (headless) build machine had no issues that I could see.

Peace,
david
--
David H. Wolfskill [hidden email]
I am amazed that anyone would condone what Trunp has publicly admitted he did.

See http://www.catwhisker.org/~david/publickey.gpg for my public key.

signature.asc (631 bytes) Download Attachment