delegating ZFS of jail's root directory

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

delegating ZFS of jail's root directory

Michael W. Lucas-2
Hi,

Two more book research questions, sorry. If the answer is "it doesn't
work that way," cool, I'll document and move on. It looks like ZFS
delegation isn't widely used.

1) It seems I can successfully delegate managing ZFS datasets to a jail,
sort of. A restart removes my ability to destroy and rename datasets I
created, though.

2) I can't delegate the jail's root to the jail. Obvious question: CAN
you delegate a jail's root dataset, or am I chasing an impossibility
here?

Details:

Real hardware, running yesterday's -current:

FreeBSD storm 13.0-CURRENT FreeBSD 13.0-CURRENT r343219 GENERIC  amd64


Here's my jail.conf.

exec.start="sh /etc/rc";
exec.stop="sh /etc/rc.shutdown";

filedump {
  host.hostname="filedump.mwl.io";
  ip4.addr="203.0.113.224";
  path="/jail/filedump/zroot";
  persist=true;
  mount.devfs=true;
  allow.mount=true;
  allow.mount.zfs=true;
  enforce_statfs=1;
  exec.poststart="/sbin/zfs jail filedump jail/filedump/zroot";
  exec.poststop="/sbin/zfs unjail filedump jail/filedump/zroot";
}

/jail/filedump/zroot contains FreeBSD 12.0 base.tgz extract.

# ls /jail/filedump/zroot/
.cshrc          dev             media           root            var
.profile        etc             mnt             sbin
COPYRIGHT       jail            net             sys
bin             lib             proc            tmp
boot            libexec         rescue          usr

Initial ZFS "jailed" parameter:

# zfs get -r jailed jail/filedump
NAME                          PROPERTY  VALUE   SOURCE
jail/filedump                 jailed    off     default
jail/filedump/zroot           jailed    off     default
jail/filedump/zroot/cdr       jailed    on      local
jail/filedump/zroot/home      jailed    on      local
jail/filedump/zroot/home/mwl  jailed    on      inherited from jail/filedump/zroot/home


Running "service jail start filedump" gives me a working jail. I can
create and destroy datasets.

root@filedump:~ # zfs create jail/filedump/zroot/home/abc
root@filedump:~ # zfs destroy jail/filedump/zroot/home/abc

Gonna recreate that dataset for testing purposes:

root@filedump:~ # zfs create jail/filedump/zroot/home/abc

Now back to the host, restart the jail, and:

root@filedump:~ # zfs destroy jail/filedump/zroot/home/abc
cannot unmount '/jail/filedump/zroot/home/abc': Operation not permitted

I created this dataset within the jail, and can manage it only so long
as it's the same jail instance. A restart wrecks my ability to manage
the dataset.



Second problem:

I would also like to delegate management of the jail's root fileset,
so on the host I run:

# zfs set jailed=on jail/filedump/zroot
# service jail start filedump
Starting jails: cannot start jail  "filedump":
jail: filedump: mount.devfs: /jail/filedump/zroot/dev: No such file or directory
.

Which--of course, the root dir isn't mounted, so /dev can't be mounted.


I'm vaguely confident I've heard of people delegating management of
the root dataset to the jail, though I can't find it. Am I
misremembering?

Thanks,

==ml

--
Michael W. Lucas https://mwl.io/
author of: Absolute OpenBSD, SSH Mastery, git commit murder,
Immortal Clay, PGP & GPG, Absolute FreeBSD, etc, etc, etc...
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: delegating ZFS of jail's root directory

Willem Jan Withagen-2
On 21-1-2019 17:42, Michael W. Lucas wrote:

> Hi,
>
> Two more book research questions, sorry. If the answer is "it doesn't
> work that way," cool, I'll document and move on. It looks like ZFS
> delegation isn't widely used.
>
> 1) It seems I can successfully delegate managing ZFS datasets to a jail,
> sort of. A restart removes my ability to destroy and rename datasets I
> created, though.
>
> 2) I can't delegate the jail's root to the jail. Obvious question: CAN
> you delegate a jail's root dataset, or am I chasing an impossibility
> here?
>
> Details:
>
> Real hardware, running yesterday's -current:
>
> FreeBSD storm 13.0-CURRENT FreeBSD 13.0-CURRENT r343219 GENERIC  amd64
>
>
> Here's my jail.conf.
>
> exec.start="sh /etc/rc";
> exec.stop="sh /etc/rc.shutdown";
>
> filedump {
>    host.hostname="filedump.mwl.io";
>    ip4.addr="203.0.113.224";
>    path="/jail/filedump/zroot";
>    persist=true;
>    mount.devfs=true;
>    allow.mount=true;
>    allow.mount.zfs=true;
>    enforce_statfs=1;
>    exec.poststart="/sbin/zfs jail filedump jail/filedump/zroot";
>    exec.poststop="/sbin/zfs unjail filedump jail/filedump/zroot";
> }
>
> /jail/filedump/zroot contains FreeBSD 12.0 base.tgz extract.
>
> # ls /jail/filedump/zroot/
> .cshrc          dev             media           root            var
> .profile        etc             mnt             sbin
> COPYRIGHT       jail            net             sys
> bin             lib             proc            tmp
> boot            libexec         rescue          usr
>
> Initial ZFS "jailed" parameter:
>
> # zfs get -r jailed jail/filedump
> NAME                          PROPERTY  VALUE   SOURCE
> jail/filedump                 jailed    off     default
> jail/filedump/zroot           jailed    off     default
> jail/filedump/zroot/cdr       jailed    on      local
> jail/filedump/zroot/home      jailed    on      local
> jail/filedump/zroot/home/mwl  jailed    on      inherited from jail/filedump/zroot/home
>
>
> Running "service jail start filedump" gives me a working jail. I can
> create and destroy datasets.
>
> root@filedump:~ # zfs create jail/filedump/zroot/home/abc
> root@filedump:~ # zfs destroy jail/filedump/zroot/home/abc
>
> Gonna recreate that dataset for testing purposes:
>
> root@filedump:~ # zfs create jail/filedump/zroot/home/abc
>
> Now back to the host, restart the jail, and:
>
> root@filedump:~ # zfs destroy jail/filedump/zroot/home/abc
> cannot unmount '/jail/filedump/zroot/home/abc': Operation not permitted
>
> I created this dataset within the jail, and can manage it only so long
> as it's the same jail instance. A restart wrecks my ability to manage
> the dataset.
>
>
>
> Second problem:
>
> I would also like to delegate management of the jail's root fileset,
> so on the host I run:
>
> # zfs set jailed=on jail/filedump/zroot
> # service jail start filedump
> Starting jails: cannot start jail  "filedump":
> jail: filedump: mount.devfs: /jail/filedump/zroot/dev: No such file or directory
> .
>
> Which--of course, the root dir isn't mounted, so /dev can't be mounted.
>
>
> I'm vaguely confident I've heard of people delegating management of
> the root dataset to the jail, though I can't find it. Am I
> misremembering?

Hi Michael,

I think I asked that question a some time ago, to be able to run a
ceph-setup script in a jail....

The basic answer was that the jail needs to have access to /dev/zfs in
the jail to be effectively controlling zfs. But then I think you
delegate the whole set of zfs capabilities to the jail.

Which in my case was not a problem. But if you want to use a jail as
separation of control, then this will be way too liberal.

There is a set of configs for devfs in /etc. See `man -k devfs`
But I've not used this in the end.

--WjW



_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: delegating ZFS of jail's root directory

Michael W. Lucas-2
On Tue, Jan 22, 2019 at 11:39:57AM +0100, Willem Jan Withagen wrote:

> On 21-1-2019 17:42, Michael W. Lucas wrote:
> > Hi,
> >
> > Two more book research questions, sorry. If the answer is "it doesn't
> > work that way," cool, I'll document and move on. It looks like ZFS
> > delegation isn't widely used.
> >
> > 1) It seems I can successfully delegate managing ZFS datasets to a jail,
> > sort of. A restart removes my ability to destroy and rename datasets I
> > created, though.
> >
> > 2) I can't delegate the jail's root to the jail. Obvious question: CAN
> > you delegate a jail's root dataset, or am I chasing an impossibility
> > here?
> >
> > Details:
> >
> > Real hardware, running yesterday's -current:
> >
> > FreeBSD storm 13.0-CURRENT FreeBSD 13.0-CURRENT r343219 GENERIC  amd64
> >
> >
> > Here's my jail.conf.
> >
> > exec.start="sh /etc/rc";
> > exec.stop="sh /etc/rc.shutdown";
> >
> > filedump {
> >    host.hostname="filedump.mwl.io";
> >    ip4.addr="203.0.113.224";
> >    path="/jail/filedump/zroot";
> >    persist=true;
> >    mount.devfs=true;
> >    allow.mount=true;
> >    allow.mount.zfs=true;
> >    enforce_statfs=1;
> >    exec.poststart="/sbin/zfs jail filedump jail/filedump/zroot";
> >    exec.poststop="/sbin/zfs unjail filedump jail/filedump/zroot";
> > }
> >
> > /jail/filedump/zroot contains FreeBSD 12.0 base.tgz extract.
> >
> > # ls /jail/filedump/zroot/
> > .cshrc          dev             media           root            var
> > .profile        etc             mnt             sbin
> > COPYRIGHT       jail            net             sys
> > bin             lib             proc            tmp
> > boot            libexec         rescue          usr
> >
> > Initial ZFS "jailed" parameter:
> >
> > # zfs get -r jailed jail/filedump
> > NAME                          PROPERTY  VALUE   SOURCE
> > jail/filedump                 jailed    off     default
> > jail/filedump/zroot           jailed    off     default
> > jail/filedump/zroot/cdr       jailed    on      local
> > jail/filedump/zroot/home      jailed    on      local
> > jail/filedump/zroot/home/mwl  jailed    on      inherited from jail/filedump/zroot/home
> >
> >
> > Running "service jail start filedump" gives me a working jail. I can
> > create and destroy datasets.
> >
> > root@filedump:~ # zfs create jail/filedump/zroot/home/abc
> > root@filedump:~ # zfs destroy jail/filedump/zroot/home/abc
> >
> > Gonna recreate that dataset for testing purposes:
> >
> > root@filedump:~ # zfs create jail/filedump/zroot/home/abc
> >
> > Now back to the host, restart the jail, and:
> >
> > root@filedump:~ # zfs destroy jail/filedump/zroot/home/abc
> > cannot unmount '/jail/filedump/zroot/home/abc': Operation not permitted
> >
> > I created this dataset within the jail, and can manage it only so long
> > as it's the same jail instance. A restart wrecks my ability to manage
> > the dataset.
> >
> >
> >
> > Second problem:
> >
> > I would also like to delegate management of the jail's root fileset,
> > so on the host I run:
> >
> > # zfs set jailed=on jail/filedump/zroot
> > # service jail start filedump
> > Starting jails: cannot start jail  "filedump":
> > jail: filedump: mount.devfs: /jail/filedump/zroot/dev: No such file or directory
> > .
> >
> > Which--of course, the root dir isn't mounted, so /dev can't be mounted.
> >
> >
> > I'm vaguely confident I've heard of people delegating management of
> > the root dataset to the jail, though I can't find it. Am I
> > misremembering?
>
> Hi Michael,
>
> I think I asked that question a some time ago, to be able to run a
> ceph-setup script in a jail....
>
> The basic answer was that the jail needs to have access to /dev/zfs in the
> jail to be effectively controlling zfs. But then I think you delegate the
> whole set of zfs capabilities to the jail.
>
> Which in my case was not a problem. But if you want to use a jail as
> separation of control, then this will be way too liberal.
>
> There is a set of configs for devfs in /etc. See `man -k devfs`
> But I've not used this in the end.

<facepalm> That fixes the first problem, thank you.

I still can't delegate the jail's root directory to the jail,
though. Once I set jailed=on to the jail's zroot, it's unmounted and
jail(8) can't find the jail's /dev to mount it.

There seems to be a chicken-and-egg problem here that I have no idea
how to resolve. Any suggestions?

==ml


--
Michael W. Lucas https://mwl.io/
author of: Absolute OpenBSD, SSH Mastery, git commit murder,
Immortal Clay, PGP & GPG, Absolute FreeBSD, etc, etc, etc...
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: delegating ZFS of jail's root directory

Miroslav Lachman
Michael W. Lucas wrote on 2019/01/22 17:23:
> On Tue, Jan 22, 2019 at 11:39:57AM +0100, Willem Jan Withagen wrote:
>> On 21-1-2019 17:42, Michael W. Lucas wrote:

>> Hi Michael,
>>
>> I think I asked that question a some time ago, to be able to run a
>> ceph-setup script in a jail....
>>
>> The basic answer was that the jail needs to have access to /dev/zfs in the
>> jail to be effectively controlling zfs. But then I think you delegate the
>> whole set of zfs capabilities to the jail.
>>
>> Which in my case was not a problem. But if you want to use a jail as
>> separation of control, then this will be way too liberal.
>>
>> There is a set of configs for devfs in /etc. See `man -k devfs`
>> But I've not used this in the end.
>
> <facepalm> That fixes the first problem, thank you.
>
> I still can't delegate the jail's root directory to the jail,
> though. Once I set jailed=on to the jail's zroot, it's unmounted and
> jail(8) can't find the jail's /dev to mount it.
>
> There seems to be a chicken-and-egg problem here that I have no idea
> how to resolve. Any suggestions?


What about to mount it with exec.prestart before the jail is created?
(I didn't tried it)

Miroslav Lachman
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"