VNET jail and dhclient

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

VNET jail and dhclient

Goran Mekić
Hello,

TLDR: I can setup static IP or use dhcpcd to get address, but not dhclient.

Let me elaborate. I run 12-CURRENT on my laptop and use CBSD as jail manager (I don't think it matters).

# dhclient eth0
chroot
exiting.

This is what I found with truss: https://gist.github.com/anonymous/36a4e2bf1760198971934ff609a7d0de#file-gistfile1-txt-L227-L228. Selected lines are what I think is the problem. Offending line in the code is probably https://svnweb.freebsd.org/base/head/sbin/dhclient/dhclient.c?revision=317915&view=markup#l507. With that asumption, Oleg, CBSD author, noticed that the following "patch" works:

diff -ruN dhclient.c-o dhclient.c
--- dhclient.c-o        2017-10-08 13:06:59.134921000 +0300
+++ dhclient.c  2017-10-08 13:07:48.047004000 +0300
 -504,8 +504,8

        if (cap_rights_limit(routefd, &rights) < 0 && errno != ENOSYS)
                error("can't limit route socket: %m");

-       if (chroot(_PATH_VAREMPTY) == -1)
-               error("chroot");
+//     if (chroot(_PATH_VEREMPTY) == -1)
+//             error("chroot");
        if (chdir("/") == -1)
                error("chdir(\"/\")");

I just assume that commenting out capsicum part of code would do the same (didn't try it) as I can create files under /var/empty and perms look normal.

Does anyone have a fairly recent 12-CURRENT VNET jail running with dhclient? If yes, what jail manager, if any? Also, could you recommend the way continue from this point given I never worked with capsicum? Thank you!

Regards,
meka

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: VNET jail and dhclient

Oleg Ginzburg
in reply to
https://lists.freebsd.org/pipermail/freebsd-jail/2017-October/003444.html

comment: it looks like it's a regression in FreeBSD 12/Current,
because in FreeBSD 11 dhclient works fine:

--
jail1:/root@[15:16] # dhclient eth0
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 3
DHCPOFFER from 192.168.10.1
DHCPREQUEST on eth0 to 255.255.255.255 port 67
DHCPACK from 192.168.10.1
bound to 192.168.8.8 -- renewal in 900 seconds.

jail1:/root@[15:16] # uname -a
FreeBSD jail1.my.domain 11.0-RELEASE-p12 FreeBSD 11.0-RELEASE-p12 #0
r324489: Tue Oct 10 14:57:58 MSK 2017
[hidden email]:/usr/obj/usr/jails/src/src_11.0/src/sys/VIMAGE
amd64
--
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: VNET jail and dhclient

Kristof Provost-3
In reply to this post by Goran Mekić
On 9 Oct 2017, at 9:25, Goran Mekić wrote:
> Hello,
>
> TLDR: I can setup static IP or use dhcpcd to get address, but not dhclient.
>
> Let me elaborate. I run 12-CURRENT on my laptop and use CBSD as jail manager (I don't think it matters).
>
What version of CURRENT are you using?

> # dhclient eth0
> chroot
> exiting.
>
> This is what I found with truss: https://gist.github.com/anonymous/36a4e2bf1760198971934ff609a7d0de#file-gistfile1-txt-L227-L228. Selected lines are what I think is the problem. Offending line in the code is probably https://svnweb.freebsd.org/base/head/sbin/dhclient/dhclient.c?revision=317915&view=markup#l507. With that asumption, Oleg, CBSD author, noticed that the following "patch" works:
>
Is there any chance you don’t have /var/empty in your jail?

I do this to create a simple vnet jail:
sudo jail -c name=alcatraz persist vnet vnet.interface=epair0b
(in the jail) dhclient epair0b

And see:

fsync(0x9)                                       = 0 (0x0)
close(8)                                         = 0 (0x0)
socket(PF_ROUTE,SOCK_RAW,0)                      = 8 (0x8)
shutdown(8,SHUT_WR)                              = 0 (0x0)
cap_rights_limit(8,{ CAP_READ,CAP_EVENT })       = 0 (0x0)
chroot("/var/empty")                             = 0 (0x0)
chdir("/")                                       = 0 (0x0)
setgroups(0x1,0x800e2c1e4)                       = 0 (0x0)


I also see the DCHP request packets on the other end of the epair interface.

Regards,
Kristof

signature.asc (921 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: VNET jail and dhclient

Oleg Ginzburg
Hello!

On Tue, Oct 10, 2017 at 8:24 PM, Kristof Provost <[hidden email]> wrote:

> On 9 Oct 2017, at 9:25, Goran Mekić wrote:
> > Hello,
> >
> > TLDR: I can setup static IP or use dhcpcd to get address, but not
> dhclient.
> >
> > Let me elaborate. I run 12-CURRENT on my laptop and use CBSD as jail
> manager (I don't think it matters).
> >
> What version of CURRENT are you using?
>
> > # dhclient eth0
> > chroot
> > exiting.
> >
> > This is what I found with truss: https://gist.github.com/anonymous/
> 36a4e2bf1760198971934ff609a7d0de#file-gistfile1-txt-L227-L228. Selected
> lines are what I think is the problem. Offending line in the code is
> probably https://svnweb.freebsd.org/base/head/sbin/dhclient/
> dhclient.c?revision=317915&view=markup#l507. With that asumption, Oleg,
> CBSD author, noticed that the following "patch" works:
> >
> Is there any chance you don’t have /var/empty in your jail?
>
> I do this to create a simple vnet jail:
> sudo jail -c name=alcatraz persist vnet vnet.interface=epair0b
> (in the jail) dhclient epair0b
>
> And see:
> …
> fsync(0x9)                                       = 0 (0x0)
> close(8)                                         = 0 (0x0)
> socket(PF_ROUTE,SOCK_RAW,0)                      = 8 (0x8)
> shutdown(8,SHUT_WR)                              = 0 (0x0)
> cap_rights_limit(8,{ CAP_READ,CAP_EVENT })       = 0 (0x0)
> chroot("/var/empty")                             = 0 (0x0)
> chdir("/")                                       = 0 (0x0)
> setgroups(0x1,0x800e2c1e4)                       = 0 (0x0)
> …
>
> I also see the DCHP request packets on the other end of the epair
> interface.
>
> Regards,
> Kristof
>


What is your FreeBSD version? This problem reproduced on FreeBSD 12 only.
/var/empty is exist and trivial test:

#include <stdio.h>
#include <stdlib.h>

int main()
{
printf("%d\n",chroot("/var/empty");
}

works successfully.

I think I found something, but I do not understand why this is only
observed in jail and with commit change this.
The problem about which the Goran wrote can be fixed with:

# diff -ruN dhclient.c-orig dhclient.c
--- dhclient.c-orig     2017-10-10 23:51:52.451361000 +0000
+++ dhclient.c  2017-10-10 23:54:55.803404000 +0000
@@ -479,6 +479,7 @@

        fork_privchld(pipe_fd[0], pipe_fd[1]);

+       pidfile_close(pidfile);
        close(ifi->ufdesc);
        ifi->ufdesc = -1;
        close(ifi->wfdesc);




From pidfile(3) man page:

    The pidfile_close() function closes a pidfile.  It should be used after
     daemon fork()s to start a child process.


chroot(2) in dhclient return NOPERM (via global errno). it seems to be
related to open descriptor outside the chroot.

I'm not sure if this fd leak (due to pidfile_remove at the end of
dhclient),  nevertheless closing pid fd in my jail/FreeBSD12 before chroot
solve dhclient issue.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: VNET jail and dhclient

Goran Mekić
On Tue, Oct 10, 2017 at 09:10:37PM +0000, Oleg Ginzburg wrote:

> I think I found something, but I do not understand why this is only
> observed in jail and with commit change this.
> The problem about which the Goran wrote can be fixed with:
>
> # diff -ruN dhclient.c-orig dhclient.c
> --- dhclient.c-orig     2017-10-10 23:51:52.451361000 +0000
> +++ dhclient.c  2017-10-10 23:54:55.803404000 +0000
> @@ -479,6 +479,7 @@
>
>         fork_privchld(pipe_fd[0], pipe_fd[1]);
>
> +       pidfile_close(pidfile);
>         close(ifi->ufdesc);
>         ifi->ufdesc = -1;
>         close(ifi->wfdesc);
>
>
>
>
> From pidfile(3) man page:
>
>     The pidfile_close() function closes a pidfile.  It should be used after
>      daemon fork()s to start a child process.
>
>
> chroot(2) in dhclient return NOPERM (via global errno). it seems to be
> related to open descriptor outside the chroot.
>
> I'm not sure if this fd leak (due to pidfile_remove at the end of
> dhclient),  nevertheless closing pid fd in my jail/FreeBSD12 before chroot
> solve dhclient issue.
I can confirm Oleg's patch works for me. Weird one, for sure!

signature.asc (849 bytes) Download Attachment