FreeBSD/powerpc64 (POWER9) with ZFS on /

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

FreeBSD/powerpc64 (POWER9) with ZFS on /

freebsd-ppc mailing list
ZFS is by default not available for partitioning in bsdinstall. We can, however, use manual partitioning in shell to make ZFS partitions.

But what if we want to do ZFS on /? This is currently not supported out-of-the-box. The reason is that Petitboot, used on POWER9 to boot kernel, doesn't support loading modules (at least when booting FreeBSD). ZFS is by default distributed only as a module, to overcome licensing issues.

There are two ways to boot ZFS on root with Petitboot:
1. We can use initramfs, which will load necessary modules. It's said to be supported on FreeBSD/powerpc64, but no one knows how to create one for FreeBSD :)
2. We can actually compile a custom kernel with ZFS in it (not as a module). This the approach that I use. Unfortunately, I can't redistribute this kernel, because of ZFS's license. I created a custom installation iso, which uses my kernel with ZFS included.

Here are the steps to do it (on an installed powerpc64 system, but should be possible on other architectures after redefining TARGET and TARGET_ARCH):
1. cd /usr/src/sys/powerpc/conf
2. cat << EOF > ZFS
include GENERIC64
options ZFS
EOF
3. cd ../../../
3. make buildworld buildkernel KERNCONF=ZFS
4. cd release
5. make release

When it's done, there will be iso files in /usr/obj/usr/src/powerpc.powerpc64/release.

You can use it to install ZFS on root.

Note that you need a FAT32 partition to mount on /boot.
This partition needs to have /boot/etc/kboot.conf file, with a line like:
FreeBSD='/kernel/kernel kernelname=/boot/kernel/kernel vfs.root.mountfrom=zfs:zroot/ROOT/default'

You also need the appriopriate /etc/fstab:
/dev/nda0p1     /boot           msdosfs rw      2       2

Other than that, howto at https://wiki.freebsd.org/RootOnZFS/GPTZFSBoot#Create_Partitions (from point 2., omit 1.) is totally ok to follow.

Thanks to mmacy for informing me about this little trick with including ZFS to kernel.

--
 _________________________________________
/ It has been said that man is a rational \
| animal. All my life I have been         |
| searching for evidence which could      |
| support this.                           |
|                                         |
\ -- Bertrand Russell                     /
 -----------------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

signature.asc (915 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD/powerpc64 (POWER9) with ZFS on /

Bill Sorenson
On Mon, Nov 5, 2018, 3:45 AM Piotr Kubaj via freebsd-ppc
<[hidden email] wrote:

>
> ZFS is by default not available for partitioning in bsdinstall. We can, however, use manual partitioning in shell to make ZFS partitions.
>
> But what if we want to do ZFS on /? This is currently not supported out-of-the-box. The reason is that Petitboot, used on POWER9 to boot kernel, doesn't support loading modules (at least when booting FreeBSD). ZFS is by default distributed only as a module, to overcome licensing issues.
>
> There are two ways to boot ZFS on root with Petitboot:
> 1. We can use initramfs, which will load necessary modules. It's said to be supported on FreeBSD/powerpc64, but no one knows how to create one for FreeBSD :)
> 2. We can actually compile a custom kernel with ZFS in it (not as a module). This the approach that I use. Unfortunately, I can't redistribute this kernel, because of ZFS's license. I created a custom installation iso, which uses my kernel with ZFS included.
>
> Here are the steps to do it (on an installed powerpc64 system, but should be possible on other architectures after redefining TARGET and TARGET_ARCH):
> 1. cd /usr/src/sys/powerpc/conf
> 2. cat << EOF > ZFS
> include GENERIC64
> options ZFS
> EOF
> 3. cd ../../../
> 3. make buildworld buildkernel KERNCONF=ZFS
> 4. cd release
> 5. make release
>
> When it's done, there will be iso files in /usr/obj/usr/src/powerpc.powerpc64/release.
>
> You can use it to install ZFS on root.
>
> Note that you need a FAT32 partition to mount on /boot.
> This partition needs to have /boot/etc/kboot.conf file, with a line like:
> FreeBSD='/kernel/kernel kernelname=/boot/kernel/kernel vfs.root.mountfrom=zfs:zroot/ROOT/default'
>
> You also need the appriopriate /etc/fstab:
> /dev/nda0p1     /boot           msdosfs rw      2       2
>
> Other than that, howto at https://wiki.freebsd.org/RootOnZFS/GPTZFSBoot#Create_Partitions (from point 2., omit 1.) is totally ok to follow.
>
> Thanks to mmacy for informing me about this little trick with including ZFS to kernel.
>
> --
>  _________________________________________
> / It has been said that man is a rational \
> | animal. All my life I have been         |
> | searching for evidence which could      |
> | support this.                           |
> |                                         |
> \ -- Bertrand Russell                     /
>  -----------------------------------------
>         \   ^__^
>          \  (oo)\_______
>             (__)\       )\/\
>                 ||----w |
>                 ||     ||

I don't think there are any license issues distributing a FreeBSD
kernel with ZFS statically linked like there are with Linux. For a
long time it wasn't supported and ZFS will probably never be in
GENERIC but a custom kernel with ZFS built in should be freely
redistributable. I don't claim to be an expert on the license issues
so if someone else has better information feel free correct me.

-Bill Sorenson
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ppc
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD/powerpc64 (POWER9) with ZFS on /

Sean Bruno-7
In reply to this post by freebsd-ppc mailing list


On 11/5/18 2:45 AM, Piotr Kubaj wrote:

> ZFS is by default not available for partitioning in bsdinstall. We can,
> however, use manual partitioning in shell to make ZFS partitions.
>
> But what if we want to do ZFS on /? This is currently not supported
> out-of-the-box. The reason is that Petitboot, used on POWER9 to boot
> kernel, doesn't support loading modules (at least when booting FreeBSD).
> ZFS is by default distributed only as a module, to overcome licensing
> issues.
>
> There are two ways to boot ZFS on root with Petitboot:
> 1. We can use initramfs, which will load necessary modules. It's said to
> be supported on FreeBSD/powerpc64, but no one knows how to create one
> for FreeBSD :)
> 2. We can actually compile a custom kernel with ZFS in it (not as a
> module). This the approach that I use. Unfortunately, I can't
> redistribute this kernel, because of ZFS's license. I created a custom
> installation iso, which uses my kernel with ZFS included.
>
> Here are the steps to do it (on an installed powerpc64 system, but
> should be possible on other architectures after redefining TARGET and
> TARGET_ARCH):
> 1. cd /usr/src/sys/powerpc/conf
> 2. cat << EOF > ZFS
> include GENERIC64
> options ZFS
> EOF
> 3. cd ../../../
> 3. make buildworld buildkernel KERNCONF=ZFS
> 4. cd release
> 5. make release
>
> When it's done, there will be iso files in
> /usr/obj/usr/src/powerpc.powerpc64/release.
>
> You can use it to install ZFS on root.
>
> Note that you need a FAT32 partition to mount on /boot.
> This partition needs to have /boot/etc/kboot.conf file, with a line like:
> FreeBSD='/kernel/kernel kernelname=/boot/kernel/kernel
> vfs.root.mountfrom=zfs:zroot/ROOT/default'
>
> You also need the appriopriate /etc/fstab:
> /dev/nda0p1     /boot           msdosfs rw      2       2
>
> Other than that, howto at
> https://wiki.freebsd.org/RootOnZFS/GPTZFSBoot#Create_Partitions (from
> point 2., omit 1.) is totally ok to follow.
>
> Thanks to mmacy for informing me about this little trick with including
> ZFS to kernel.
>

I'd recommend adding a default to the kboot.conf so that your machine
doesn't hang at petitboot forever.

# cat /mnt/etc/kboot.conf
default=FreeBSD_boot
FreeBSD_boot='/kernel/kernel vfs.root.mountfrom=zfs:zroot
kernelname=/kernel/kernel'

The Power8 machines in the freebsd cluster use a statically built kernel
with no modules.

sean
(trimmed cc list)


signature.asc (631 bytes) Download Attachment