Fixed ip's for jails

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Fixed ip's for jails

Heinz Gies
Good afternoon everyone,
we are currently contemplating how to handle multi-tenant Jails best to allow ProjectFiFo to administer FreeBSD Jails along with Solaris Zones.

The hurdle we have run into is the following:

With vnet Jails the owner could change the IP, making it impractical for multi-tenancy.

With no-vnet Jails, all the jails would share the same network stack removing a layer of isolation and risking a noisy neighbour problem.

There are a few possible solutions it seems.

Allen suggested using the firewall to restrict the traffic from a vnet in the global zone (host system, not sure how BSD calls it). The top of mind issue with this is that it would block multicast.

An alternative Kevin came up with was putting a jail inside another jail, with the outer jail being a vimage jail and the inner jail using a static (non-vnet IP). This would also mean later on beehive inside a jail would be easier as it could follow the same logic. On the other hand, I am a bit worried about unforeseen consequences of this approach. Also, I am not 100% positive whether the inner jail would use the vnet network stack for it is IP and not the global one.

Thank you for your input.

Cheers,
Heinz


Heinz N. Gies
Project-FiFo
Cloud Orchestration
Web:

project-fifo.net<https://project-fifo.net/>

Docs:

Documentation<https://docs.project-fifo.net/en/latest/index.html>

Tickets:
GPG:

Ticket Tracker<https://project-fifo.atlassian.net/>
452B6F98<https://project-fifo.net/heinz.gpg>


_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Fixed ip's for jails

James Gritton-2
On 2017-07-06 12:57, Heinz Gies wrote:

> Good afternoon everyone,
> we are currently contemplating how to handle multi-tenant Jails best
> to allow ProjectFiFo to administer FreeBSD Jails along with Solaris
> Zones.
>
> The hurdle we have run into is the following:
>
> With vnet Jails the owner could change the IP, making it impractical
> for multi-tenancy.
>
> With no-vnet Jails, all the jails would share the same network stack
> removing a layer of isolation and risking a noisy neighbour problem.
>
> There are a few possible solutions it seems.
>
> Allen suggested using the firewall to restrict the traffic from a vnet
> in the global zone (host system, not sure how BSD calls it). The top
> of mind issue with this is that it would block multicast.
>
> An alternative Kevin came up with was putting a jail inside another
> jail, with the outer jail being a vimage jail and the inner jail using
> a static (non-vnet IP). This would also mean later on beehive inside a
> jail would be easier as it could follow the same logic. On the other
> hand, I am a bit worried about unforeseen consequences of this
> approach. Also, I am not 100% positive whether the inner jail would
> use the vnet network stack for it is IP and not the global one.
>
> Thank you for your input.

I'm for the jail within a jail.  Rest assured that the child jail will
be inside the vnet that the parent jail was created with.  The outer
jail need not be running anything - just run enough to set up the
network, and set its "persist" parameter.  Then it's nothing but an
empty shell to hold the network for the inner jail, with almost no
overhead.  The inner jail doesn't need any IP restrictions, which
inherits (but can't change) the parent's addresses.  Something like
this:

foo {
   vnet;
   vnet.interface = foo0;
   persist;
}

foo.client {
   <normal jail parameters>;
}

As for unforeseen consequences, I don't foresee any ;-).

- Jamie
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Fixed ip's for jails

Heinz Gies
Thanks a lot Jamie :)
With a bit of experimenting it ended up working pr


Heinz N. Gies
Project-FiFo
Cloud Orchestration
Web:
project-fifo.net <https://project-fifo.net/>
Docs:
Documentation <https://docs.project-fifo.net/en/latest/index.html>
Tickets:
GPG:
Ticket Tracker <https://project-fifo.atlassian.net/>
452B6F98 <https://project-fifo.net/heinz.gpg>
 

On 07.07.17, 17:18, "James Gritton" <[hidden email]> wrote:

    On 2017-07-06 12:57, Heinz Gies wrote:
    > Good afternoon everyone,
    > we are currently contemplating how to handle multi-tenant Jails best
    > to allow ProjectFiFo to administer FreeBSD Jails along with Solaris
    > Zones.
    >
    > The hurdle we have run into is the following:
    >
    > With vnet Jails the owner could change the IP, making it impractical
    > for multi-tenancy.
    >
    > With no-vnet Jails, all the jails would share the same network stack
    > removing a layer of isolation and risking a noisy neighbour problem.
    >
    > There are a few possible solutions it seems.
    >
    > Allen suggested using the firewall to restrict the traffic from a vnet
    > in the global zone (host system, not sure how BSD calls it). The top
    > of mind issue with this is that it would block multicast.
    >
    > An alternative Kevin came up with was putting a jail inside another
    > jail, with the outer jail being a vimage jail and the inner jail using
    > a static (non-vnet IP). This would also mean later on beehive inside a
    > jail would be easier as it could follow the same logic. On the other
    > hand, I am a bit worried about unforeseen consequences of this
    > approach. Also, I am not 100% positive whether the inner jail would
    > use the vnet network stack for it is IP and not the global one.
    >
    > Thank you for your input.
   
    I'm for the jail within a jail.  Rest assured that the child jail will
    be inside the vnet that the parent jail was created with.  The outer
    jail need not be running anything - just run enough to set up the
    network, and set its "persist" parameter.  Then it's nothing but an
    empty shell to hold the network for the inner jail, with almost no
    overhead.  The inner jail doesn't need any IP restrictions, which
    inherits (but can't change) the parent's addresses.  Something like
    this:
   
    foo {
       vnet;
       vnet.interface = foo0;
       persist;
    }
   
    foo.client {
       <normal jail parameters>;
    }
   
    As for unforeseen consequences, I don't foresee any ;-).
   
    - Jamie
   

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Fixed ip's for jails

Heinz Gies
In reply to this post by James Gritton-2
Thanks a lot Jamie :),
With a bit of experimenting it ended up working quite well! I wanted to give some feedback so your advice doesn’t end up on a black hole.

This is what it ended up with, outer jail with vnet inner jail with a inherited NIC.

https://github.com/project-fifo/r-vmadm/pull/20

Cheers,
Heinz


Heinz N. Gies
Project-FiFo
Cloud Orchestration
Web:
project-fifo.net <https://project-fifo.net/>
Docs:
Documentation <https://docs.project-fifo.net/en/latest/index.html>
Tickets:
GPG:
Ticket Tracker <https://project-fifo.atlassian.net/>
452B6F98 <https://project-fifo.net/heinz.gpg>
 

On 07.07.17, 17:18, "James Gritton" <[hidden email]> wrote:

    On 2017-07-06 12:57, Heinz Gies wrote:
    > Good afternoon everyone,
    > we are currently contemplating how to handle multi-tenant Jails best
    > to allow ProjectFiFo to administer FreeBSD Jails along with Solaris
    > Zones.
    >
    > The hurdle we have run into is the following:
    >
    > With vnet Jails the owner could change the IP, making it impractical
    > for multi-tenancy.
    >
    > With no-vnet Jails, all the jails would share the same network stack
    > removing a layer of isolation and risking a noisy neighbour problem.
    >
    > There are a few possible solutions it seems.
    >
    > Allen suggested using the firewall to restrict the traffic from a vnet
    > in the global zone (host system, not sure how BSD calls it). The top
    > of mind issue with this is that it would block multicast.
    >
    > An alternative Kevin came up with was putting a jail inside another
    > jail, with the outer jail being a vimage jail and the inner jail using
    > a static (non-vnet IP). This would also mean later on beehive inside a
    > jail would be easier as it could follow the same logic. On the other
    > hand, I am a bit worried about unforeseen consequences of this
    > approach. Also, I am not 100% positive whether the inner jail would
    > use the vnet network stack for it is IP and not the global one.
    >
    > Thank you for your input.
   
    I'm for the jail within a jail.  Rest assured that the child jail will
    be inside the vnet that the parent jail was created with.  The outer
    jail need not be running anything - just run enough to set up the
    network, and set its "persist" parameter.  Then it's nothing but an
    empty shell to hold the network for the inner jail, with almost no
    overhead.  The inner jail doesn't need any IP restrictions, which
    inherits (but can't change) the parent's addresses.  Something like
    this:
   
    foo {
       vnet;
       vnet.interface = foo0;
       persist;
    }
   
    foo.client {
       <normal jail parameters>;
    }
   
    As for unforeseen consequences, I don't foresee any ;-).
   
    - Jamie
   

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"