vnet jail shutdown crashes system

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

vnet jail shutdown crashes system

David Mehler
Hello,

I've finally created a vnet jail on FreeBSD 12.1 that will get out to
the internet. Whenever I atempt to shut it down the system crashes, I
have no idea why.

I found an example and adapted and pounded on it until I got it
working. Here's my configuration. On the host:

/etc/rc.conf fragment:
cloned_interfaces="bridge0"
ifconfig_bridge0="inet 192.168.122.1/24 addm vtnet0 up"

#cat /etc/jail.conf
exec.clean;
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
mount.devfs;
allow.raw_sockets;
        exec.system_user = "root";
        exec.jail_user = "root";
vnet;

jail1 {
    host.hostname = jail1.lan;
    path = "/jails/jail1";
    devfs_ruleset = "5";
    vnet.interface = "epair0b";
    exec.prestart = "ifconfig epair0 create up";
    exec.prestart += "ifconfig bridge0 addm epair0a";
    exec.poststop = "ifconfig bridge0 deletem epair0a";
    exec.poststop += "ifconfig epair0a destroy";
        exec.consolelog = "/var/log/jail_jail1_console.log";
}

ifconfig fragment:
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:e7:79:f2:c4:00
        inet 192.168.122.1 netmask 0xffffff00 broadcast 192.168.122.255
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 4 priority 128 path cost 2000
        member: vtnet0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 1 priority 128 path cost 2000
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>
epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST>
metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:ad:9b:f9:5e:0a
        inet6 fe80::ad:9bff:fef9:5e0a%epair0a prefixlen 64 scopeid 0x4
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>

In the vnet jail:
# cat /etc/rc.conf
hostname="jail1.lan"
ifconfig_epair0b="inet 192.168.122.50 netmask 255.255.255.0"
defaultrouter="192.168.122.1"

I wish I knew why stopping this jail takes the whole system down,
suggestions welcome.
Thanks.
Dave.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: vnet jail shutdown crashes system

Kyle Evans-3
On Sun, Jun 7, 2020 at 8:59 PM David Mehler <[hidden email]> wrote:

>
> Hello,
>
> I've finally created a vnet jail on FreeBSD 12.1 that will get out to
> the internet. Whenever I atempt to shut it down the system crashes, I
> have no idea why.
>
> [... snip ...]
>
> I wish I knew why stopping this jail takes the whole system down,
> suggestions welcome.

Without seeing a panic message, this is almost certainly the ol' epair
teardown problem. I'm working toward a solution for it, but I'm still
needing to get people to review the prerequisite change to add a busy
mechanism to ifnet so that we can coordinate correctly.

You can work around it by destroying the epair interface visible there
on the host before you destroy the jail (and perhaps there's a better
way), though that's not necessarily ideal if you need networking to
work through a graceful shutdown.

Thanks,

Kyle Evans
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Ole
Reply | Threaded
Open this post in threaded view
|

Re: vnet jail shutdown crashes system

Ole
In reply to this post by David Mehler
Hello Dave,

I had the same problem. I found out, that the system will crash, if I do
the 'ifconfig epair0a destroy' direct after the 'jail  -r'.

My solution is to sleep 2 seconds after the 'jail  -r' command.

Maybe a little bit dirty.

Ole



Sun, 7 Jun 2020 21:59:03 -0400 - David Mehler <[hidden email]>:

> Hello,
>
> I've finally created a vnet jail on FreeBSD 12.1 that will get out to
> the internet. Whenever I atempt to shut it down the system crashes, I
> have no idea why.
>
> I found an example and adapted and pounded on it until I got it
> working. Here's my configuration. On the host:
>
> /etc/rc.conf fragment:
> cloned_interfaces="bridge0"
> ifconfig_bridge0="inet 192.168.122.1/24 addm vtnet0 up"
>
> #cat /etc/jail.conf
> exec.clean;
> exec.start = "/bin/sh /etc/rc";
> exec.stop = "/bin/sh /etc/rc.shutdown";
> mount.devfs;
> allow.raw_sockets;
>         exec.system_user = "root";
>         exec.jail_user = "root";
> vnet;
>
> jail1 {
>     host.hostname = jail1.lan;
>     path = "/jails/jail1";
>     devfs_ruleset = "5";
>     vnet.interface = "epair0b";
>     exec.prestart = "ifconfig epair0 create up";
>     exec.prestart += "ifconfig bridge0 addm epair0a";
>     exec.poststop = "ifconfig bridge0 deletem epair0a";
>     exec.poststop += "ifconfig epair0a destroy";
>         exec.consolelog = "/var/log/jail_jail1_console.log";
> }
>
> ifconfig fragment:
> bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0
> mtu 1500 ether 02:e7:79:f2:c4:00
>         inet 192.168.122.1 netmask 0xffffff00 broadcast
> 192.168.122.255 id 00:00:00:00:00:00 priority 32768 hellotime 2
> fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
>         root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
>         member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
>                 ifmaxaddr 0 port 4 priority 128 path cost 2000
>         member: vtnet0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
>                 ifmaxaddr 0 port 1 priority 128 path cost 2000
>         groups: bridge
>         nd6 options=9<PERFORMNUD,IFDISABLED>
> epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST>
> metric 0 mtu 1500
>         options=8<VLAN_MTU>
>         ether 02:ad:9b:f9:5e:0a
>         inet6 fe80::ad:9bff:fef9:5e0a%epair0a prefixlen 64 scopeid 0x4
>         groups: epair
>         media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
>         status: active
>         nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
>
> In the vnet jail:
> # cat /etc/rc.conf
> hostname="jail1.lan"
> ifconfig_epair0b="inet 192.168.122.50 netmask 255.255.255.0"
> defaultrouter="192.168.122.1"
>
> I wish I knew why stopping this jail takes the whole system down,
> suggestions welcome.
> Thanks.
> Dave.
> _______________________________________________
> [hidden email] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-jail
> To unsubscribe, send any mail to
> "[hidden email]"

attachment0 (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: vnet jail shutdown crashes system

David Mehler
Hello,

Thanks to everyone who offered suggestions on my vnet jail. I'm
passing this on, a friend of mine sent me his configuration, which he
modified from mine, it does not crash on vnet jail shutdown and takes
down the interfaces both epair0a and epair0b. My rc.conf in the
original post is unchanged. Here's the revised and working
/etc/jail.conf:

#cat jail.conf
exec.clean;
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
mount.devfs;
allow.raw_sockets;
        #exec.system_user = "root";
        #exec.jail_user = "root";
vnet;

jail1 {
    host.hostname = jail1.lan;
    path = "/jails/jail1";
    devfs_ruleset = "4";
        mount.devfs;
        vnet = "new";
    vnet.interface = "epair0b";
    exec.prestart = "ifconfig epair0 create up";
    exec.prestart += "ifconfig bridge0 addm epair0a";
        exec.start  = "/bin/sh /etc/rc";
        exec.start += "ifconfig epair0b inet 192.168.122.50 netmask
255.255.255.0";
        exec.start += "route add default 192.168.122.1";
        exec.stop   = "/bin/sh /etc/rc.shutdown";
    exec.poststop = "ifconfig bridge0 deletem epair0a";
    exec.poststop += "ifconfig epair0a destroy";
        exec.consolelog = "/var/log/jail_jail1_console.log";
}


I have no idea why this works but it is here I'm passing it on hoping
it helps others with vnet and/or track down the epair teardown issue.

Thanks and HTH
Dave.


On 6/9/20, Ole <[hidden email]> wrote:

> Hello Dave,
>
> I had the same problem. I found out, that the system will crash, if I do
> the 'ifconfig epair0a destroy' direct after the 'jail  -r'.
>
> My solution is to sleep 2 seconds after the 'jail  -r' command.
>
> Maybe a little bit dirty.
>
> Ole
>
>
>
> Sun, 7 Jun 2020 21:59:03 -0400 - David Mehler <[hidden email]>:
>
>> Hello,
>>
>> I've finally created a vnet jail on FreeBSD 12.1 that will get out to
>> the internet. Whenever I atempt to shut it down the system crashes, I
>> have no idea why.
>>
>> I found an example and adapted and pounded on it until I got it
>> working. Here's my configuration. On the host:
>>
>> /etc/rc.conf fragment:
>> cloned_interfaces="bridge0"
>> ifconfig_bridge0="inet 192.168.122.1/24 addm vtnet0 up"
>>
>> #cat /etc/jail.conf
>> exec.clean;
>> exec.start = "/bin/sh /etc/rc";
>> exec.stop = "/bin/sh /etc/rc.shutdown";
>> mount.devfs;
>> allow.raw_sockets;
>>         exec.system_user = "root";
>>         exec.jail_user = "root";
>> vnet;
>>
>> jail1 {
>>     host.hostname = jail1.lan;
>>     path = "/jails/jail1";
>>     devfs_ruleset = "5";
>>     vnet.interface = "epair0b";
>>     exec.prestart = "ifconfig epair0 create up";
>>     exec.prestart += "ifconfig bridge0 addm epair0a";
>>     exec.poststop = "ifconfig bridge0 deletem epair0a";
>>     exec.poststop += "ifconfig epair0a destroy";
>>         exec.consolelog = "/var/log/jail_jail1_console.log";
>> }
>>
>> ifconfig fragment:
>> bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0
>> mtu 1500 ether 02:e7:79:f2:c4:00
>>         inet 192.168.122.1 netmask 0xffffff00 broadcast
>> 192.168.122.255 id 00:00:00:00:00:00 priority 32768 hellotime 2
>> fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
>>         root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
>>         member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
>>                 ifmaxaddr 0 port 4 priority 128 path cost 2000
>>         member: vtnet0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
>>                 ifmaxaddr 0 port 1 priority 128 path cost 2000
>>         groups: bridge
>>         nd6 options=9<PERFORMNUD,IFDISABLED>
>> epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST>
>> metric 0 mtu 1500
>>         options=8<VLAN_MTU>
>>         ether 02:ad:9b:f9:5e:0a
>>         inet6 fe80::ad:9bff:fef9:5e0a%epair0a prefixlen 64 scopeid 0x4
>>         groups: epair
>>         media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
>>         status: active
>>         nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
>>
>> In the vnet jail:
>> # cat /etc/rc.conf
>> hostname="jail1.lan"
>> ifconfig_epair0b="inet 192.168.122.50 netmask 255.255.255.0"
>> defaultrouter="192.168.122.1"
>>
>> I wish I knew why stopping this jail takes the whole system down,
>> suggestions welcome.
>> Thanks.
>> Dave.
>> _______________________________________________
>> [hidden email] mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-jail
>> To unsubscribe, send any mail to
>> "[hidden email]"
>
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"