Need help with VNET, Jail and IPv6

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Need help with VNET, Jail and IPv6

Jacques Foucry
Hello all,

On my hosted machine I already have many "classical" jails.

But I would like to switch to modern schema with Bridge and vnet.

With IPv4 I have no problem. In fact is almost like without Bridge/VNET:

ifconfig em0bridge    
em0bridge: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:36:b3:c1:8a:00
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 1 priority 128 path cost 20000
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>

nyjail{
   host.hostname="mywebite.fr";
   path="/jails/mywebsite";
 
   allow.mount=true;
   allow.raw_sockets;
   vnet;
   vnet.interface   = "e0b_${name}";

   exec.prestart   += "jib addm ${name} em0";
   exec.poststop   += "jib destroy ${name}";

   exec.start      = "/sbin/ifconfig e0b_${name} 10.1.1.28/24";
   exec.start     += "/sbin/route add default 10.1.1.254";

   exec.poststart += "/sbin/ifconfig e0a_${name} 10.1.1.254/24";
   
   exec.poststop  += "/sbin/ifconfig e0b_${name} -vnet ${name}";
   exec.poststop  += "/sbin/ifconfig deletem e0b_${name}";
   exec.poststop  += "sleep 2";
   exec.poststop  += "/sbin/ifconfig e0b_${name} destroy";

   persist=true;
   mount.fstab="/etc/fstab.${name}";
}

With pf I could connect througt ssh to my jail:

jails_net = "{192.168.12.0/24 10.1.1.0/24}"

nat on $ext_if from $jails_net to any -> ($ext_if)

myjail_v4="10.1.1.28"
myjail_v6="2a01:4f9:4a:1fd8::28"
myjail_ports = 2228

rdr on $ext_if inet proto tcp from any to $ext_if port $myjail_ports -> $myjail_v4
rdr on $ext_if inet6 proto tcp from any to $ext_if port $myjail_ports -> $myjail_v6

pass in log quick on $ext_if proto tcp from any to $myjail_v4 port
pass in log quick on $ext_if proto tcp from any to $myjail_v6 port


The old fashion mail use em0_alias for IPv6

em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=81009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWFILTER>
        ether b4:2e:99:6a:80:9d
        inet6 2a01:4f9:4a:1fd8::2 prefixlen 64
        inet6 fe80::b62e:99ff:fe6a:809d%em0 prefixlen 64 scopeid 0x1
        inet6 2a01:4f9:4a:1fd8::5 prefixlen 64
        inet6 2a01:4f9:4a:1fd8::16 prefixlen 64
        inet6 2a01:4f9:4a:1fd8::14 prefixlen 64
        inet6 2a01:4f9:4a:1fd8::15 prefixlen 64
        inet6 2a01:4f9:4a:1fd8::21 prefixlen 64
        inet6 2a01:4f9:4a:1fd8::25 prefixlen 64
        inet6 2a01:4f9:4a:1fd8::29 prefixlen 64
        inet6 2a01:4f9:4a:1fd8::17 prefixlen 64
        inet6 2a01:4f9:4a:1fd8::11 prefixlen 64
        inet6 2a01:4f9:4a:1fd8::12 prefixlen 64
        inet6 2a01:4f9:4a:1fd8::18 prefixlen 64
        inet6 2a01:4f9:4a:1fd8::22 prefixlen 64
        inet6 2a01:4f9:4a:1fd8::19 prefixlen 64
        inet6 2a01:4f9:4a:1fd8::28 prefixlen 64
        inet 95.217.83.231 netmask 0xffffffc0 broadcast 95.217.83.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

My goal is first to have on jail (myjail) working with IPv4 and IPv6 then,
slowly migrate the old jail to the new way.

So, I need help to configure myjail to have IPv6 working:
- configure a IPv6 on e0b_myjail is easy, but which defaultrouter6 did I use?
- did the bridge have an IPv6 to be the defaultrouter6? I try with no luck.
- did I need some configuration on PF?

Thanks for reading me (I sure I not really clear) and for your advice.

Btw, after I successfully configure myjail (and the other one) I will wrote a
how-to.

--
Jacques Foucry
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Need help with VNET, Jail and IPv6

Mina Galić

> Hello all,

Hi Jacques,


> On my hosted machine I already have many "classical" jails.
>
> But I would like to switch to modern schema with Bridge and vnet.
>
> With IPv4 I have no problem. In fact is almost like without Bridge/VNET:

For:

    https://alpha.pkgbase.live/

instead of libioc I just used jail.conf. With:

    https://antranigv.am/weblog_en/posts/vnet-jail-howto/

as basis for the IPv4 setup.

> My goal is first to have on jail (myjail) working with IPv4 and IPv6 then,
>
> slowly migrate the old jail to the new way.
>
> So, I need help to configure myjail to have IPv6 working:
>
> -   configure a IPv6 on e0b_myjail is easy, but which defaultrouter6 did I use?
> -   did the bridge have an IPv6 to be the defaultrouter6? I try with no luck.
> -   did I need some configuration on PF?
>
>     Thanks for reading me (I sure I not really clear) and for your advice.
>
>     Btw, after I successfully configure myjail (and the other one) I will wrote a  how-to.
>

Okay, let's see if I can hit all beats:

Here's the paste of webserver.jail.conf, rc.conf (highlights) and pf.conf

    https://gist.github.com/87ba10c1c5611ed32367d5d48ef5f402

I'll explain some of the important bits:

my ISP binds the IPv4 to the MAC, but not the IPv6, go figure.
That's why I leave the IPv4 address on the main interface, instead of fiddling with MAC addresses and moving it to the bridge.

On the bridge, we have the IPv6 and the IPv4 NAT; That's handy, as it also means we only need one interface for both IPv4 and IPv6.

    cloned_interfaces="bridge0"
    # jail NAT and Network access
    ifconfig_bridge0="inet 192.168.17.1/24"
    gateway_enable="YES"

note that we explicitly enable link-local addresses, because, as per spec, they are needed to make IPv6 work:

    # working IPv6 setup needs link-local addresses (according to the spec)
    ipv6_activate_all_interfaces="YES"
    ifconfig_bridge0_ipv6="inet6 2a01:4f9:c010:c64c::1/64 auto_linklocal"
    ipv6_defaultrouter="fe80::1%vtnet0"
    # enable IPv6 gateway
    ipv6_gateway_enable="YES"

and in the jail.conf it's really just about adding the IPv6 addresses to the interfaces, too!

    vnet.interface = "$jepair";

    exec.prestart   = "ifconfig epair${id} create up";
    exec.prestart  += "ifconfig epair${id}a up descr vnet-${name}";
    exec.prestart  += "ifconfig $bridge addm epair${id}a up";

    exec.start      = "/sbin/ifconfig lo0 127.0.0.1 up";
    exec.start     += "/sbin/ifconfig epair${id}b ${ipaddr}";
    exec.start     += "/sbin/ifconfig epair${id}b inet6 ${ip6addr}";
    exec.start     += "/sbin/route add default ${gw}";
    exec.start     += "/sbin/route add -inet6 default ${gw6}";
    exec.start     += "/bin/sh /etc/rc";

I also highly recommend adding IPv6 nameservers to your resolv.conf; that way, if you broke your IPv4 setup, you still have working IPv6!

Being NAT, IPv4 routing is obviously happening via the host.
Aaaaand, given that my ISP uses fe80::1 as the default gateway, the only way to make jails' IPv6 routing work was by routing it thru the host.

as for pf, it's only used for NAT.
No firewalling, and I'm not doing anything to IPv6.

That's all from me, i hope it helps.

>     --
>
>     Jacques Foucry


best of luck,

Mina
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Need help with VNET, Jail and IPv6

Jacques Foucry
Le mardi 05 janv. 2021 à 21:39:27 (+0000), Mina Galić à écrit:

>
> > Hello all,
>
> Hi Jacques,
>
>
> > On my hosted machine I already have many "classical" jails.
> >
> > But I would like to switch to modern schema with Bridge and vnet.
> >
> > With IPv4 I have no problem. In fact is almost like without Bridge/VNET:
>
> For:
>
>     https://alpha.pkgbase.live/
>
> instead of libioc I just used jail.conf. With:
>
>     https://antranigv.am/weblog_en/posts/vnet-jail-howto/


I already read this and succesfully made a IPv4 jail with this tuto.

>
> as basis for the IPv4 setup.
>
> > My goal is first to have on jail (myjail) working with IPv4 and IPv6 then,
> >
> > slowly migrate the old jail to the new way.
> >
> > So, I need help to configure myjail to have IPv6 working:
> >
> > -   configure a IPv6 on e0b_myjail is easy, but which defaultrouter6 did I use?
> > -   did the bridge have an IPv6 to be the defaultrouter6? I try with no luck.
> > -   did I need some configuration on PF?
> >
> >     Thanks for reading me (I sure I not really clear) and for your advice.
> >
> >     Btw, after I successfully configure myjail (and the other one) I will wrote a  how-to.
> >
>
> Okay, let's see if I can hit all beats:
>
> Here's the paste of webserver.jail.conf, rc.conf (highlights) and pf.conf
>
>     https://gist.github.com/87ba10c1c5611ed32367d5d48ef5f402

Thanks, that really clear.

>
> I'll explain some of the important bits:
>
> my ISP binds the IPv4 to the MAC, but not the IPv6, go figure.
> That's why I leave the IPv4 address on the main interface, instead of fiddling with MAC addresses and moving it to the bridge.
>
> On the bridge, we have the IPv6 and the IPv4 NAT; That's handy, as it also means we only need one interface for both IPv4 and IPv6.
>
>     cloned_interfaces="bridge0"
>     # jail NAT and Network access
>     ifconfig_bridge0="inet 192.168.17.1/24"
>     gateway_enable="YES"
>
> note that we explicitly enable link-local addresses, because, as per spec, they are needed to make IPv6 work:
>
>     # working IPv6 setup needs link-local addresses (according to the spec)
>     ipv6_activate_all_interfaces="YES"
>     ifconfig_bridge0_ipv6="inet6 2a01:4f9:c010:c64c::1/64 auto_linklocal"
>     ipv6_defaultrouter="fe80::1%vtnet0"

Why vtnet instead of vnet ? Is there a difference that I did saw?

>     # enable IPv6 gateway
>     ipv6_gateway_enable="YES"
>
> and in the jail.conf it's really just about adding the IPv6 addresses to the interfaces, too!
>
>     vnet.interface = "$jepair";
>
>     exec.prestart   = "ifconfig epair${id} create up";
>     exec.prestart  += "ifconfig epair${id}a up descr vnet-${name}";
>     exec.prestart  += "ifconfig $bridge addm epair${id}a up";
>
>     exec.start      = "/sbin/ifconfig lo0 127.0.0.1 up";
>     exec.start     += "/sbin/ifconfig epair${id}b ${ipaddr}";
>     exec.start     += "/sbin/ifconfig epair${id}b inet6 ${ip6addr}";
>     exec.start     += "/sbin/route add default ${gw}";
>     exec.start     += "/sbin/route add -inet6 default ${gw6}";
>     exec.start     += "/bin/sh /etc/rc";
>
> I also highly recommend adding IPv6 nameservers to your resolv.conf; that way, if you broke your IPv4 setup, you still have working IPv6!

That a good advice too :-)
 
> Being NAT, IPv4 routing is obviously happening via the host.
> Aaaaand, given that my ISP uses fe80::1 as the default gateway, the only way to make jails' IPv6 routing work was by routing it thru the host.
 
> as for pf, it's only used for NAT.
> No firewalling, and I'm not doing anything to IPv6.
>
> That's all from me, i hope it helps.


Sure it'a help, thanks for your advices, your time and expertise.

--
Jacques Foucry
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"