kqueue(2) kevents for jails

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

kqueue(2) kevents for jails

Fabian Freyer
Hi!

I'm writing a jail management library [1], and am wondering if there's
any nice way to get notified of jail state changes (especially running
-> dying -> dead) as well as of parameter changes.

What are the opinions on adding a kevent(2) for these things?

Fabian

[1] https://github.com/fubarnetes/libjail-rs/
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: kqueue(2) kevents for jails

Konstantin Belousov
On Fri, Jan 04, 2019 at 02:55:05PM +0100, Fabian Freyer wrote:

> Hi!
>
> I'm writing a jail management library [1], and am wondering if there's
> any nice way to get notified of jail state changes (especially running
> -> dying -> dead) as well as of parameter changes.
>
> What are the opinions on adding a kevent(2) for these things?
>
> Fabian
>
> [1] https://github.com/fubarnetes/libjail-rs/
No, kevent(2) is not suitable mechanism to notify about jail state changes.
If anything in the existing system can be reused for such notifications,
it is devctl(4) notifications which are handled by devd(8).  Look at the
man pages and for existing notifications in kernel code, e.g.
sys/kern/kern_conf.c notify*() for how devfs does it.

It is both more natural and much easier integrated with the jail code.
Not least because jail creation/destruction is relatively low frequency
events with potentially rich secondary information that should be attached
to them.  Kevents are high-frequency, high-performance kind of events,
and only naturally tied to file descriptors.  There were lot of bugs in
integration of kevents with e.g. processes notifications, and API is
still somewhat racy.

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: kqueue(2) kevents for jails

Christian Barthel-5
In reply to this post by Fabian Freyer
Fabian Freyer <[hidden email]> writes:

> I'm writing a jail management library [1], and am wondering if there's
> any nice way to get notified of jail state changes (especially running
> -> dying -> dead) as well as of parameter changes.

I worked on something similar (not a library but more acting like a
daemon).   The way I managed Jails was by forking a jail(8) process
and collecting the exit status.  Not sure if that is possible for your
library case.

--
Christian Barthel
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: kqueue(2) kevents for jails

Fabian Freyer
On 1/4/19 6:20 PM, Christian Barthel wrote:
> I worked on something similar (not a library but more acting like a
> daemon).   The way I managed Jails was by forking a jail(8) process
> and collecting the exit status.  Not sure if that is possible for your
> library case.

Yes, I've thought about doing things like that too, like double-forking
and having the parent wait for the jailed child, but those all seem
dirty to me. Ideally, I'd like to register callbacks on jail state
change to clean up file systems etc.

On 1/4/19 5:14 PM, Konstantin Belousov wrote:
> No, kevent(2) is not suitable mechanism to notify about jail state changes.
> If anything in the existing system can be reused for such notifications,
> it is devctl(4) notifications which are handled by devd(8).  Look at the
> man pages and for existing notifications in kernel code, e.g.
> sys/kern/kern_conf.c notify*() for how devfs does it.

Can any running binary subscribe to devd(8) events or does that require
a configuration change in /etc/devd.conf?
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: kqueue(2) kevents for jails

Konstantin Belousov
On Fri, Jan 04, 2019 at 09:11:58PM +0100, Fabian Freyer wrote:
> On 1/4/19 5:14 PM, Konstantin Belousov wrote:
> > No, kevent(2) is not suitable mechanism to notify about jail state changes.
> > If anything in the existing system can be reused for such notifications,
> > it is devctl(4) notifications which are handled by devd(8).  Look at the
> > man pages and for existing notifications in kernel code, e.g.
> > sys/kern/kern_conf.c notify*() for how devfs does it.
>
> Can any running binary subscribe to devd(8) events or does that require
> a configuration change in /etc/devd.conf?

Only one reader is supported, effectively.  devctl(4) tries to limit opens
naively.  But then even if you have the file descriptor and fork or pass
it over unix domain socket, single event can be only read by one reader.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: kqueue(2) kevents for jails

Fabian Freyer


On 1/4/19 9:29 PM, Konstantin Belousov wrote:

> On Fri, Jan 04, 2019 at 09:11:58PM +0100, Fabian Freyer wrote:
>> On 1/4/19 5:14 PM, Konstantin Belousov wrote:
>>> No, kevent(2) is not suitable mechanism to notify about jail state changes.
>>> If anything in the existing system can be reused for such notifications,
>>> it is devctl(4) notifications which are handled by devd(8).  Look at the
>>> man pages and for existing notifications in kernel code, e.g.
>>> sys/kern/kern_conf.c notify*() for how devfs does it.
>>
>> Can any running binary subscribe to devd(8) events or does that require
>> a configuration change in /etc/devd.conf?
>
> Only one reader is supported, effectively.  devctl(4) tries to limit opens
> naively.  But then even if you have the file descriptor and fork or pass
> it over unix domain socket, single event can be only read by one reader.
>

Ah, I see, thanks! Is there any other nice notification mechanism that a
process could 'subscribe' to to be notified of an event?

I am still a bit confused as to why knotify would be such a bad fit,
maybe you could expand a bit on that?

> Not least because jail creation/destruction is relatively low frequency
> events with potentially rich secondary information that should be attached
> to them.  Kevents are high-frequency, high-performance kind of events,

Does this mean they cannot nicely be used for lower-frequency things?
I'm thinking of situations where jails may get spawned e.g.
per-network-request.

> and only naturally tied to file descriptors.

according to kevent(2),

EVFILT_PROC         Takes the process ID to monitor as the identifier

so there's also cases where it isn't tied to a file descriptor, but some
other descriptor (pid's don't seem to be too different to jid's?)

> There were lot of bugs in
> integration of kevents with e.g. processes notifications, and API is
> still somewhat racy

Is this a kevents issue or an integration problem?

In the end, might it be a good idea to add devctl(4) notifications as
well as kevent(2)?
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: kqueue(2) kevents for jails

Konstantin Belousov
On Fri, Jan 04, 2019 at 10:22:07PM +0100, Fabian Freyer wrote:

>
>
> On 1/4/19 9:29 PM, Konstantin Belousov wrote:
> > On Fri, Jan 04, 2019 at 09:11:58PM +0100, Fabian Freyer wrote:
> >> On 1/4/19 5:14 PM, Konstantin Belousov wrote:
> >>> No, kevent(2) is not suitable mechanism to notify about jail state changes.
> >>> If anything in the existing system can be reused for such notifications,
> >>> it is devctl(4) notifications which are handled by devd(8).  Look at the
> >>> man pages and for existing notifications in kernel code, e.g.
> >>> sys/kern/kern_conf.c notify*() for how devfs does it.
> >>
> >> Can any running binary subscribe to devd(8) events or does that require
> >> a configuration change in /etc/devd.conf?
> >
> > Only one reader is supported, effectively.  devctl(4) tries to limit opens
> > naively.  But then even if you have the file descriptor and fork or pass
> > it over unix domain socket, single event can be only read by one reader.
> >
>
> Ah, I see, thanks! Is there any other nice notification mechanism that a
> process could 'subscribe' to to be notified of an event?
devctl(4) is currently the best mechanism.

Apparently there is a functionality in devd(8) which I was not aware of.
In essence, it can operate as fan-out service, delivering kernel events to
all clients connected to /var/run/devd.seqpacket.pipe socket.  So despite
/dev/devctl* not allowing multiple clients, you would make your service
connect to devd(8) and get the events stream from there.

>
> I am still a bit confused as to why knotify would be such a bad fit,
> maybe you could expand a bit on that?
I have no idea what is knotify.

>
> > Not least because jail creation/destruction is relatively low frequency
> > events with potentially rich secondary information that should be attached
> > to them.  Kevents are high-frequency, high-performance kind of events,
>
> Does this mean they cannot nicely be used for lower-frequency things?
> I'm thinking of situations where jails may get spawned e.g.
> per-network-request.
They are not nice to be bolted on things which are not supposed to be
performance-critical.  Not least because _properly_ supporting kevents
adds significant maintainance cost on the code.

>
> > and only naturally tied to file descriptors.
>
> according to kevent(2),
>
> EVFILT_PROC         Takes the process ID to monitor as the identifier
>
> so there's also cases where it isn't tied to a file descriptor, but some
> other descriptor (pid's don't seem to be too different to jid's?)
Did you read what I wrote in previous message ?

>
> > There were lot of bugs in
> > integration of kevents with e.g. processes notifications, and API is
> > still somewhat racy
>
> Is this a kevents issue or an integration problem?
It is architectural problem.

>
> In the end, might it be a good idea to add devctl(4) notifications as
> well as kevent(2)?
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"