Making C++11 a hard requirement for FreeBSD

classic Classic list List threaded Threaded
34 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Making C++11 a hard requirement for FreeBSD

Warner Losh
I'd like to start a conversation about the viability of making C++11 a hard
requirement for bootstrapping FreeBSD and setting a specific deadline for
doing so.

This discussion is motivated by an ask from the jemalloc folks to use a
limited subset of C++11 inside of malloc in such a way that is C safe (so
the C programs wouldn't bloat with a C++ runtime). That's an ongoing
discussion in another forum, and isn't appropriate for this thread because
this has become a frequent request (but if you have opinions, please find
the thread in current@ about it). I don't know the timeline of their plans
to do this.

I'd like to take the rather vague plans we've had "before 12" and create a
timeline for removal of gcc 4.2 coupled with a requirement for support in
clang or a working external toolchain. This requirement would be coupled
with the requirement that the external toolchain support C++11 constructs.

I'd like to propose we do this 12/31/17. Any architectures that can't meet
this timeline will be disconnected from universe at that time and deleted
3/31/18.

It's my belief that i386, amd64, arm, aarch64, powerpc and powerpc64 are
ready for this change and mips* would be ready for this change with an
external clang toolchain. I'm unsure of riscv and sparc64, but suspect that
a newer version of gcc as an external toolchain could work.

Comments?

Warner
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Making C++11 a hard requirement for FreeBSD

Brooks Davis-2
On Thu, Oct 05, 2017 at 04:28:44PM -0700, Warner Losh wrote:

> I'd like to start a conversation about the viability of making C++11 a hard
> requirement for bootstrapping FreeBSD and setting a specific deadline for
> doing so.
>
> This discussion is motivated by an ask from the jemalloc folks to use a
> limited subset of C++11 inside of malloc in such a way that is C safe (so
> the C programs wouldn't bloat with a C++ runtime). That's an ongoing
> discussion in another forum, and isn't appropriate for this thread because
> this has become a frequent request (but if you have opinions, please find
> the thread in current@ about it). I don't know the timeline of their plans
> to do this.
>
> I'd like to take the rather vague plans we've had "before 12" and create a
> timeline for removal of gcc 4.2 coupled with a requirement for support in
> clang or a working external toolchain. This requirement would be coupled
> with the requirement that the external toolchain support C++11 constructs.
>
> I'd like to propose we do this 12/31/17. Any architectures that can't meet
> this timeline will be disconnected from universe at that time and deleted
> 3/31/18.
This deadline seems viable to me.

> It's my belief that i386, amd64, arm, aarch64, powerpc and powerpc64 are
> ready for this change and mips* would be ready for this change with an
> external clang toolchain. I'm unsure of riscv and sparc64, but suspect that
> a newer version of gcc as an external toolchain could work.

mips64 should be good to go with external clang and lld in LLVM 6.0
and the llvm-devel port should be ready before then (We need to get
multi-got support landed upstream and then Alex has a fix for the
remaining lld issues).  As it is, I run clang built mips64 binaries
daily.

I'm certain the riscv supporting GCC supports C++11, but we might need
to do some testing and tweak some make bits.

If someone wants to test sparc64, that's fine, but with Oracle laying
off the SPARC it's arguably more dead than IA64 was when we removed it
(Intel shipped the last design *this* May).

-- Brooks

signature.asc (465 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Making C++11 a hard requirement for FreeBSD

John Baldwin
In reply to this post by Warner Losh
On Thursday, October 05, 2017 04:28:44 PM Warner Losh wrote:

> I'd like to start a conversation about the viability of making C++11 a hard
> requirement for bootstrapping FreeBSD and setting a specific deadline for
> doing so.
>
> This discussion is motivated by an ask from the jemalloc folks to use a
> limited subset of C++11 inside of malloc in such a way that is C safe (so
> the C programs wouldn't bloat with a C++ runtime). That's an ongoing
> discussion in another forum, and isn't appropriate for this thread because
> this has become a frequent request (but if you have opinions, please find
> the thread in current@ about it). I don't know the timeline of their plans
> to do this.
>
> I'd like to take the rather vague plans we've had "before 12" and create a
> timeline for removal of gcc 4.2 coupled with a requirement for support in
> clang or a working external toolchain. This requirement would be coupled
> with the requirement that the external toolchain support C++11 constructs.
>
> I'd like to propose we do this 12/31/17. Any architectures that can't meet
> this timeline will be disconnected from universe at that time and deleted
> 3/31/18.
>
> It's my belief that i386, amd64, arm, aarch64, powerpc and powerpc64 are
> ready for this change and mips* would be ready for this change with an
> external clang toolchain. I'm unsure of riscv and sparc64, but suspect that
> a newer version of gcc as an external toolchain could work.

In-tree clang 5.0 for MIPS mostly works (modulo some small patches).  However,
it requires external ld.bfd.  I know that there ld.lld can link a working
mips64 world with some patches (notably the multigot patch).  mips works fine
with external GCC.  riscv is already using external GCC that is C++11-capable.

The problem with external GCC is that you can cross-build a world + kernel
just fine and get a working system via CROSS_TOOLCHAIN=foo-gcc.  However,
that system has no viable '/usr/bin/cc' once GCC 4.2 is removed.  bapt@
started on ports to cross-build binutils and gcc packages via the base/*
ports, but those are not yet finished / fully tested.  I don't think anyone
has thought about how release builds will work either with only an external
toolchain available.  (I'm pretty sure sparc64 has booted fine with
external GCC, it's just in the same boat as mips with regard to /usr/bin/cc.)

Also, if you svn rm contrib/gcc you will nuke all of our systems because we
still use 'crtstuff.c' from there on all architectures for part of the C
startup code.  emaste@ has looked at a replacement for that from NetBSD in
the past but I'm not sure what state that is in currently.

Another concern is fully replacing the compiler support libraries (libgcc and
friends).  Some of those also come from contrib/gcc.  For mips I have some
patches in review upstream to add mips to LLVM's libunwind (which allows mips
to use that for libgcc unwinding).  I think sparc64 might be the only other
architecture not using llvm libunwind.  (Fixing that is a much smaller lift
than fixing clang on sparc64 btw, and I've successfully used llvm libunwind
on mips worlds that are fully compiled with external GCC.)

That said, I definitely support the goal of requiring C++11.  I will happily
start using it myself in some userland bits (truss for example could benefit
from std::unordered_map<>) once it is available across the board.

12/31/17 might be a bit aggressive given the holidays at the end of the
quarter, but we can start with that and revisit if need be.

--
John Baldwin
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Making C++11 a hard requirement for FreeBSD

A. Wilcox
In reply to this post by Brooks Davis-2
On 05/10/17 18:41, Brooks Davis wrote:

> On Thu, Oct 05, 2017 at 04:28:44PM -0700, Warner Losh wrote:
>> It's my belief that i386, amd64, arm, aarch64, powerpc and powerpc64 are
>> ready for this change and mips* would be ready for this change with an
>> external clang toolchain. I'm unsure of riscv and sparc64, but suspect that
>> a newer version of gcc as an external toolchain could work.
>
> If someone wants to test sparc64, that's fine, but with Oracle laying
> off the SPARC it's arguably more dead than IA64 was when we removed it
> (Intel shipped the last design *this* May).
>
> -- Brooks
>
Thinking out loud:

That doesn't change the fact that sparc64 still exists, and with Oracle
laying off Solaris as well, FreeBSD becomes a "way out" for people
heavily invested (DC full of sparc64 gear, or such).  That could make
the sparc64 port not only more widely used, but more widely tested, with
more potential developers.

Or maybe not.

I can't see the future :)

--arw

--
A. Wilcox (awilfox)
Open-source programmer (C, C++, Python)
https://code.foxkit.us/u/awilfox/


signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Making C++11 a hard requirement for FreeBSD

Baptiste Daroussin-2
In reply to this post by John Baldwin
On Thu, Oct 05, 2017 at 11:47:57PM +0000, John Baldwin wrote:

> On Thursday, October 05, 2017 04:28:44 PM Warner Losh wrote:
> > I'd like to start a conversation about the viability of making C++11 a hard
> > requirement for bootstrapping FreeBSD and setting a specific deadline for
> > doing so.
> >
> > This discussion is motivated by an ask from the jemalloc folks to use a
> > limited subset of C++11 inside of malloc in such a way that is C safe (so
> > the C programs wouldn't bloat with a C++ runtime). That's an ongoing
> > discussion in another forum, and isn't appropriate for this thread because
> > this has become a frequent request (but if you have opinions, please find
> > the thread in current@ about it). I don't know the timeline of their plans
> > to do this.
> >
> > I'd like to take the rather vague plans we've had "before 12" and create a
> > timeline for removal of gcc 4.2 coupled with a requirement for support in
> > clang or a working external toolchain. This requirement would be coupled
> > with the requirement that the external toolchain support C++11 constructs.
> >
> > I'd like to propose we do this 12/31/17. Any architectures that can't meet
> > this timeline will be disconnected from universe at that time and deleted
> > 3/31/18.
> >
> > It's my belief that i386, amd64, arm, aarch64, powerpc and powerpc64 are
> > ready for this change and mips* would be ready for this change with an
> > external clang toolchain. I'm unsure of riscv and sparc64, but suspect that
> > a newer version of gcc as an external toolchain could work.
>
> In-tree clang 5.0 for MIPS mostly works (modulo some small patches).  However,
> it requires external ld.bfd.  I know that there ld.lld can link a working
> mips64 world with some patches (notably the multigot patch).  mips works fine
> with external GCC.  riscv is already using external GCC that is C++11-capable.
>
> The problem with external GCC is that you can cross-build a world + kernel
> just fine and get a working system via CROSS_TOOLCHAIN=foo-gcc.  However,
> that system has no viable '/usr/bin/cc' once GCC 4.2 is removed.  bapt@
> started on ports to cross-build binutils and gcc packages via the base/*
> ports, but those are not yet finished / fully tested.  I don't think anyone
> has thought about how release builds will work either with only an external
> toolchain available.  (I'm pretty sure sparc64 has booted fine with
> external GCC, it's just in the same boat as mips with regard to /usr/bin/cc.)
Actually I did test those and they were working (tested in qemu) they were
working fine. I have given up working on them due to the lack of interested by
the community (by interest I mean people really testing, working on it, not just
saying "hey nice sounds cool").

As for the boot when I initially worked on external toolchain sparc64 was my
guinea pig and so yes it worked an booted just fine.

>
> Also, if you svn rm contrib/gcc you will nuke all of our systems because we
> still use 'crtstuff.c' from there on all architectures for part of the C
> startup code.  emaste@ has looked at a replacement for that from NetBSD in
> the past but I'm not sure what state that is in currently.
>
> Another concern is fully replacing the compiler support libraries (libgcc and
> friends).  Some of those also come from contrib/gcc.  For mips I have some
> patches in review upstream to add mips to LLVM's libunwind (which allows mips
> to use that for libgcc unwinding).  I think sparc64 might be the only other
> architecture not using llvm libunwind.  (Fixing that is a much smaller lift
> than fixing clang on sparc64 btw, and I've successfully used llvm libunwind
> on mips worlds that are fully compiled with external GCC.)
>
> That said, I definitely support the goal of requiring C++11.  I will happily
> start using it myself in some userland bits (truss for example could benefit
> from std::unordered_map<>) once it is available across the board.
>
> 12/31/17 might be a bit aggressive given the holidays at the end of the
> quarter, but we can start with that and revisit if need be.
>
I'm fine with that date.

Best regards,
Bapt

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Making C++11 a hard requirement for FreeBSD

Poul-Henning Kamp
If we allow C++ in libc, it should not merely be for the convenience
of a few programmers, but because we have a vision for how it that
makes the world, or at least FreeBSD, a better place.

Having C++ in libc is no trivial detail, there is a number of areas
where this causes bootstrapping issues and conflicts.

We can solve those issues with unsightly local hacks, most
notably a bogo-malloc to malloc while C++ constructs jemalloc.

But hand on heart, we all know that is a bad idea, all of us have
been down that road before, and we also know that there is no way
to be a little bit pregnant.

The other way, the right way, to accomodate the jemalloc request
is to go all in.

Nothing in the ISO verbiage says that you cannot have C and C++
runtimes in the same library, as long as your linker knows the zip
code of it.

Libc as a combined C and C++ runtime can be implemented a lot cleaner
than a libc which hides C++ components in the closet.

So that is my input to this question:

Either we tell the jemalloc people "sorry, it's called libc for a
reason" or we decide to make our libc a native C *and* C++ runtime.

I see no sane or even possible "middle ground" or compromise position.

Poul-Henning


--
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[hidden email]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Making C++11 a hard requirement for FreeBSD

Adrian Chadd-4
On 6 October 2017 at 09:47, Poul-Henning Kamp <[hidden email]> wrote:

> If we allow C++ in libc, it should not merely be for the convenience
> of a few programmers, but because we have a vision for how it that
> makes the world, or at least FreeBSD, a better place.
>
> Having C++ in libc is no trivial detail, there is a number of areas
> where this causes bootstrapping issues and conflicts.
>
> We can solve those issues with unsightly local hacks, most
> notably a bogo-malloc to malloc while C++ constructs jemalloc.
>
> But hand on heart, we all know that is a bad idea, all of us have
> been down that road before, and we also know that there is no way
> to be a little bit pregnant.
>
> The other way, the right way, to accomodate the jemalloc request
> is to go all in.
>
> Nothing in the ISO verbiage says that you cannot have C and C++
> runtimes in the same library, as long as your linker knows the zip
> code of it.
>
> Libc as a combined C and C++ runtime can be implemented a lot cleaner
> than a libc which hides C++ components in the closet.
>
> So that is my input to this question:
>
> Either we tell the jemalloc people "sorry, it's called libc for a
> reason" or we decide to make our libc a native C *and* C++ runtime.
>
> I see no sane or even possible "middle ground" or compromise position.

I don't mind it as long as it's "no C++ runtime bloat please". But
yes, I also feel the pain of where you start that path and then
suddenly you find you're al in that path.

(I face this at work right now on linux platforms because a "little
C++" becomes .. not little.)




-adrian
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Making C++11 a hard requirement for FreeBSD

John Baldwin
In reply to this post by Poul-Henning Kamp
On Friday, October 06, 2017 04:47:48 PM Poul-Henning Kamp wrote:

> If we allow C++ in libc, it should not merely be for the convenience
> of a few programmers, but because we have a vision for how it that
> makes the world, or at least FreeBSD, a better place.
>
> Having C++ in libc is no trivial detail, there is a number of areas
> where this causes bootstrapping issues and conflicts.
>
> We can solve those issues with unsightly local hacks, most
> notably a bogo-malloc to malloc while C++ constructs jemalloc.
>
> But hand on heart, we all know that is a bad idea, all of us have
> been down that road before, and we also know that there is no way
> to be a little bit pregnant.
>
> The other way, the right way, to accomodate the jemalloc request
> is to go all in.
>
> Nothing in the ISO verbiage says that you cannot have C and C++
> runtimes in the same library, as long as your linker knows the zip
> code of it.
>
> Libc as a combined C and C++ runtime can be implemented a lot cleaner
> than a libc which hides C++ components in the closet.
>
> So that is my input to this question:
>
> Either we tell the jemalloc people "sorry, it's called libc for a
> reason" or we decide to make our libc a native C *and* C++ runtime.
>
> I see no sane or even possible "middle ground" or compromise position.

Hmm, I don't quite agree.  I think it's possible to use a restricted C++
(no rtti, no exceptions, no STL) such that you are only using language
features like templates or 'auto' without requiring runtime support.  I
think that is the requirement we would place on the jemalloc implementation
for it to remain in libc.  Right now the C++ runtime is split into a
couple of different pieces: libc++ (STL bits, roughly), libcxxrt (rtti
/ exception support), libgcc_s (either llvm libunwind or gcc for _Unwind_*
along with intrinsics from compiler-rt).  All of these are variable in
some sense (if you wanted to build a GCC-based system you might want to
use libstdc++ instead of libc++, libgcc_s already varies by platform,
and upstream in LLVM there is already a libcxxabi alternative to libcxxrt
plus the GNU libsupc++).

I think bundling any of those pieces into libc makes our system less
flexible and different from all the other UNIXy systems currently in
vogue.

--
John Baldwin
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Making C++11 a hard requirement for FreeBSD

Poul-Henning Kamp
--------
In message <[hidden email]>, John Baldwin writes:

>Hmm, I don't quite agree.  I think it's possible to use a restricted C++
>(no rtti, no exceptions, no STL) such that you are only using language
>features like templates or 'auto' without requiring runtime support.

That's what Bjarne used to call "C++ as a better C compiler".

If the jemalloc crew can stay inside that dotted line _and_ the C++
compilers still allow you to do so, then that could be an "not quite
pregnant yet" option.

>[...]

>Right now the C++ runtime is split into a
>couple of different pieces: libc++ (STL bits, roughly), libcxxrt (rtti
>/ exception support), libgcc_s (either llvm libunwind or gcc for _Unwind_*
>along with intrinsics from compiler-rt).
>[...]
>I think bundling any of those pieces into libc makes our system less
>flexible and different from all the other UNIXy systems currently in
>vogue.

That goes to my point about ld:  The standard doesn't say which
library file which bits of the C++ runtime have to go into, we
get to decide that if we want to, as long as we provide a ld(1)
which knows where to find things.

--
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[hidden email]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Making C++11 a hard requirement for FreeBSD

Konstantin Belousov
In reply to this post by John Baldwin
On Fri, Oct 06, 2017 at 04:21:05PM -0700, John Baldwin wrote:

> On Friday, October 06, 2017 04:47:48 PM Poul-Henning Kamp wrote:
> > If we allow C++ in libc, it should not merely be for the convenience
> > of a few programmers, but because we have a vision for how it that
> > makes the world, or at least FreeBSD, a better place.
> >
> > Having C++ in libc is no trivial detail, there is a number of areas
> > where this causes bootstrapping issues and conflicts.
> >
> > We can solve those issues with unsightly local hacks, most
> > notably a bogo-malloc to malloc while C++ constructs jemalloc.
> >
> > But hand on heart, we all know that is a bad idea, all of us have
> > been down that road before, and we also know that there is no way
> > to be a little bit pregnant.
> >
> > The other way, the right way, to accomodate the jemalloc request
> > is to go all in.
> >
> > Nothing in the ISO verbiage says that you cannot have C and C++
> > runtimes in the same library, as long as your linker knows the zip
> > code of it.
> >
> > Libc as a combined C and C++ runtime can be implemented a lot cleaner
> > than a libc which hides C++ components in the closet.
> >
> > So that is my input to this question:
> >
> > Either we tell the jemalloc people "sorry, it's called libc for a
> > reason" or we decide to make our libc a native C *and* C++ runtime.
> >
> > I see no sane or even possible "middle ground" or compromise position.
>
> Hmm, I don't quite agree.  I think it's possible to use a restricted C++
> (no rtti, no exceptions, no STL) such that you are only using language
> features like templates or 'auto' without requiring runtime support.  I
> think that is the requirement we would place on the jemalloc implementation
> for it to remain in libc.
This is a requirement not only on jemalloc, but also on the compilers.
I am much more worried about C++ compiler's runtime requirements than
the ability of the jemalloc developers to restrict used language features
to the subset which does not need some external support _at the current
compiler version_.

Seeing the route that clang took making C compiler unusable for normal
work, I am just sure that clang++ would cause a lot of troubles if we
ever try to rely on the undocumented and unpromised detail of the
current implementation.

> Right now the C++ runtime is split into a
> couple of different pieces: libc++ (STL bits, roughly), libcxxrt (rtti
> / exception support), libgcc_s (either llvm libunwind or gcc for _Unwind_*
> along with intrinsics from compiler-rt).  All of these are variable in
> some sense (if you wanted to build a GCC-based system you might want to
> use libstdc++ instead of libc++, libgcc_s already varies by platform,
> and upstream in LLVM there is already a libcxxabi alternative to libcxxrt
> plus the GNU libsupc++).
>
> I think bundling any of those pieces into libc makes our system less
> flexible and different from all the other UNIXy systems currently in
> vogue.
This also hits the ABI stability hard, see my other reply.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Making C++11 a hard requirement for FreeBSD

Konstantin Belousov
In reply to this post by Poul-Henning Kamp
On Sat, Oct 07, 2017 at 12:04:38AM +0000, Poul-Henning Kamp wrote:

> --------
> In message <[hidden email]>, John Baldwin writes:
>
> >Hmm, I don't quite agree.  I think it's possible to use a restricted C++
> >(no rtti, no exceptions, no STL) such that you are only using language
> >features like templates or 'auto' without requiring runtime support.
>
> That's what Bjarne used to call "C++ as a better C compiler".
>
> If the jemalloc crew can stay inside that dotted line _and_ the C++
> compilers still allow you to do so, then that could be an "not quite
> pregnant yet" option.
>
> >[...]
>
> >Right now the C++ runtime is split into a
> >couple of different pieces: libc++ (STL bits, roughly), libcxxrt (rtti
> >/ exception support), libgcc_s (either llvm libunwind or gcc for _Unwind_*
> >along with intrinsics from compiler-rt).
> >[...]
> >I think bundling any of those pieces into libc makes our system less
> >flexible and different from all the other UNIXy systems currently in
> >vogue.
>
> That goes to my point about ld:  The standard doesn't say which
> library file which bits of the C++ runtime have to go into, we
> get to decide that if we want to, as long as we provide a ld(1)
> which knows where to find things.
This is not how linkers work.

The language standard does not say that, but the platform standards do:
system ABI very much put the symbol origin in a stone. Dynamic libraries
expose symbols under versions, if we ever expose some symbol under the
specific version, we promise to do so to the end of times (or we break
the ABI promise of stability).

C++ runtime external symbols are already exported from the specific set
of libraries. There are some tricks like filter objects which could
somewhat alleviate the move, but maintaining them for third-party libs
is too much efforts to ask, for almost no visible effect until things
break.

And of course there are parts of the C++ ABI which are underspecified by
the Itanium ABI, which means that the ABI is not yet stable. Also we get
the tie for specific compiler implementation, which runtime is imported.
Since other compiler ABI is partially incompatible, but only partially,
the other compiler would be impossible to use: some symbols resolution
would come from our libc, some from the compiler libraries.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: future of sparc64 (was: Making C++11 a hard requirement for FreeBSD)

K. Macy
In reply to this post by A. Wilcox
On Sat, Oct 7, 2017 at 10:41 Mark Linimon <[hidden email]> wrote:

> On Thu, Oct 05, 2017 at 07:12:28PM -0500, A. Wilcox wrote:
> > That doesn't change the fact that sparc64 still exists, and with Oracle
> > laying off Solaris as well, FreeBSD becomes a "way out" for people
> > heavily invested (DC full of sparc64 gear, or such).
>
> I have thought for some time that we've been a "way out" for Solaris
> sites wanting to keep ZFS and not deal with licensing issues, and have
> worked to keep sparc64 alive.  (AFAIK FreeBSD is the only open source
> sparc64/zfs solution?)
>
> But here's the current problem.
>
> All gccs > 4.9 fail to build.  Looking at the logs AFAICT the failure
> is a floating-point exception as soon as the first built binary is run
> during the internal testing.
>
> Neither Marcel nor Gerald nor I have any insight on how to fix this.
> Gerald does state that those gccs build on other OSes, so this is almost
> certainly a FreBSD problem.
>
> The default ports compiler has recently moved to gcc5 and then again
> to gcc6.  The only reason gcc49 still exists in the Ports Collection is
> specifically for sparc64 ports.
>
> Recent llvms do not build.  I have no insight into that failure, either.
>
> So, the long and short is, even with using gcc4.2.1 as an external
> compiler, over time, fewer and fewer ports build as they adapt to the
> newer compilers.
>
> This is something I don't have the cycles to fix.  Unless someone else
> can step up and fix the compilers, we're close to the end of feasibility.
>
> In the meantime, I'll keep running package builds with gcc4.9 as long as
> it produces some kind of useful results.



My recollection of sparc64 from sun4v work was that unsupported operations
would trap in to the kernel which would in turn trap in to a user space
handler for floating point emulation. If someone wants to fix it that’s
where to look. I think that FreeBSD needs to always have one big-endian
arch and one arch that requires IOMMU. Bonus points if it fulfills both.
For a time that was sparc64. These days other arches meet that need. And at
this point the most recent hardware supported by the sparc64 port shipped
in ~2003. One could amortize the cost of a low end 2017 server in just the
power bill within a year. I don’t know how much work continuing to maintain
sparc64 really adds to non sparc64 enthusiasts. Nonetheless, it is non-zero.



-M
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: future of sparc64 (was: Making C++11 a hard requirement for FreeBSD)

Mark Linimon-2
On Sat, Oct 07, 2017 at 07:06:29PM +0000, K. Macy wrote:
> I think that FreeBSD needs to always have one big-endian arch

IMHO it keeps things honest.  fwiw, if you fix a port on sparc64 it
will usually fix it on powerpc64 and vice versa (~80% correlation).

But powerpc64 has a (hardware) future and sparc64 doesn't.  I run both
at home, but not the powerpc64 continuously.  (Actually first typed
"4U" it as "$U".  Same idea.)

So, I'm willing to help keep it going (and even loan a machine to the
effort), but I am overcommitted in other areas already.

mcl
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Making C++11 a hard requirement for FreeBSD

Peter Jeremy-8
In reply to this post by Warner Losh
On 2017-Oct-05 16:28:44 -0700, Warner Losh <[hidden email]> wrote:
>I'd like to start a conversation about the viability of making C++11 a hard
>requirement for bootstrapping FreeBSD and setting a specific deadline for
>doing so.

I don't have any specific objection and the suggested timeframe sounds
reasonable to me.

>I'd like to propose we do this 12/31/17. Any architectures that can't meet
>this timeline will be disconnected from universe at that time and deleted
>3/31/18.

Note that FreeBSD is an international project.  Using abbreviated US-format
dates can be ambiguous.  Please either use ISO standard dates (2017-12-31)
or a variant with the abbreviated month name (2017-Dec-31) to reduce the
scope for confusion.

--
Peter Jeremy

signature.asc (968 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Making C++11 a hard requirement for FreeBSD

Nathan Whitehorn-8
In reply to this post by Baptiste Daroussin-2


On 10/06/17 00:20, Baptiste Daroussin wrote:

> On Thu, Oct 05, 2017 at 11:47:57PM +0000, John Baldwin wrote:
>> On Thursday, October 05, 2017 04:28:44 PM Warner Losh wrote:
>>> I'd like to start a conversation about the viability of making C++11 a hard
>>> requirement for bootstrapping FreeBSD and setting a specific deadline for
>>> doing so.
>>>
>>> This discussion is motivated by an ask from the jemalloc folks to use a
>>> limited subset of C++11 inside of malloc in such a way that is C safe (so
>>> the C programs wouldn't bloat with a C++ runtime). That's an ongoing
>>> discussion in another forum, and isn't appropriate for this thread because
>>> this has become a frequent request (but if you have opinions, please find
>>> the thread in current@ about it). I don't know the timeline of their plans
>>> to do this.
>>>
>>> I'd like to take the rather vague plans we've had "before 12" and create a
>>> timeline for removal of gcc 4.2 coupled with a requirement for support in
>>> clang or a working external toolchain. This requirement would be coupled
>>> with the requirement that the external toolchain support C++11 constructs.
>>>
>>> I'd like to propose we do this 12/31/17. Any architectures that can't meet
>>> this timeline will be disconnected from universe at that time and deleted
>>> 3/31/18.
>>>
>>> It's my belief that i386, amd64, arm, aarch64, powerpc and powerpc64 are
>>> ready for this change and mips* would be ready for this change with an
>>> external clang toolchain. I'm unsure of riscv and sparc64, but suspect that
>>> a newer version of gcc as an external toolchain could work.
>> In-tree clang 5.0 for MIPS mostly works (modulo some small patches).  However,
>> it requires external ld.bfd.  I know that there ld.lld can link a working
>> mips64 world with some patches (notably the multigot patch).  mips works fine
>> with external GCC.  riscv is already using external GCC that is C++11-capable.
>>
>> The problem with external GCC is that you can cross-build a world + kernel
>> just fine and get a working system via CROSS_TOOLCHAIN=foo-gcc.  However,
>> that system has no viable '/usr/bin/cc' once GCC 4.2 is removed.  bapt@
>> started on ports to cross-build binutils and gcc packages via the base/*
>> ports, but those are not yet finished / fully tested.  I don't think anyone
>> has thought about how release builds will work either with only an external
>> toolchain available.  (I'm pretty sure sparc64 has booted fine with
>> external GCC, it's just in the same boat as mips with regard to /usr/bin/cc.)
> Actually I did test those and they were working (tested in qemu) they were
> working fine. I have given up working on them due to the lack of interested by
> the community (by interest I mean people really testing, working on it, not just
> saying "hey nice sounds cool").
>
> As for the boot when I initially worked on external toolchain sparc64 was my
> guinea pig and so yes it worked an booted just fine.

So far as I know, we never solved any of the infrastructural problems
associated with this concept:
1. Providing built releases with a /usr/bin/cc
2. Coversioning base and in-ports toolchain, including ensuring commit
atomicity between toolchains and libc
3. Adding a dependency on ports for src, including out-of-tree code that
has to be fetched from external servers
4. Getting make universe to do the right thing

We really need to solve those. If we go the external toolchain route,
which it is not clear to me is the best option, #2 and #1 are quite
complex problems. I think that, frankly, a deadline in two months to
solve this set of political problems we have had for two years is
probably crazy, but maybe making the status quo unsustainable will
finally force progress.

Also, are there any directions for how to build a toolchain from the
base ports? I haven't been able to figure it out -- the README starts
with "pkg install", but that doesn't get me very far on platforms
without binary packages, which are the ones this is meant to target.
-Nathan
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Making C++11 a hard requirement for FreeBSD

Warner Losh
On Sun, Oct 8, 2017 at 11:01 PM, Nathan Whitehorn <[hidden email]>
wrote:

>
>
> On 10/06/17 00:20, Baptiste Daroussin wrote:
>
>> On Thu, Oct 05, 2017 at 11:47:57PM +0000, John Baldwin wrote:
>>
>>> On Thursday, October 05, 2017 04:28:44 PM Warner Losh wrote:
>>>
>>>> I'd like to start a conversation about the viability of making C++11 a
>>>> hard
>>>> requirement for bootstrapping FreeBSD and setting a specific deadline
>>>> for
>>>> doing so.
>>>>
>>>> This discussion is motivated by an ask from the jemalloc folks to use a
>>>> limited subset of C++11 inside of malloc in such a way that is C safe
>>>> (so
>>>> the C programs wouldn't bloat with a C++ runtime). That's an ongoing
>>>> discussion in another forum, and isn't appropriate for this thread
>>>> because
>>>> this has become a frequent request (but if you have opinions, please
>>>> find
>>>> the thread in current@ about it). I don't know the timeline of their
>>>> plans
>>>> to do this.
>>>>
>>>> I'd like to take the rather vague plans we've had "before 12" and
>>>> create a
>>>> timeline for removal of gcc 4.2 coupled with a requirement for support
>>>> in
>>>> clang or a working external toolchain. This requirement would be coupled
>>>> with the requirement that the external toolchain support C++11
>>>> constructs.
>>>>
>>>> I'd like to propose we do this 12/31/17. Any architectures that can't
>>>> meet
>>>> this timeline will be disconnected from universe at that time and
>>>> deleted
>>>> 3/31/18.
>>>>
>>>> It's my belief that i386, amd64, arm, aarch64, powerpc and powerpc64 are
>>>> ready for this change and mips* would be ready for this change with an
>>>> external clang toolchain. I'm unsure of riscv and sparc64, but suspect
>>>> that
>>>> a newer version of gcc as an external toolchain could work.
>>>>
>>> In-tree clang 5.0 for MIPS mostly works (modulo some small patches).
>>> However,
>>> it requires external ld.bfd.  I know that there ld.lld can link a working
>>> mips64 world with some patches (notably the multigot patch).  mips works
>>> fine
>>> with external GCC.  riscv is already using external GCC that is
>>> C++11-capable.
>>>
>>> The problem with external GCC is that you can cross-build a world +
>>> kernel
>>> just fine and get a working system via CROSS_TOOLCHAIN=foo-gcc.  However,
>>> that system has no viable '/usr/bin/cc' once GCC 4.2 is removed.  bapt@
>>> started on ports to cross-build binutils and gcc packages via the base/*
>>> ports, but those are not yet finished / fully tested.  I don't think
>>> anyone
>>> has thought about how release builds will work either with only an
>>> external
>>> toolchain available.  (I'm pretty sure sparc64 has booted fine with
>>> external GCC, it's just in the same boat as mips with regard to
>>> /usr/bin/cc.)
>>>
>> Actually I did test those and they were working (tested in qemu) they were
>> working fine. I have given up working on them due to the lack of
>> interested by
>> the community (by interest I mean people really testing, working on it,
>> not just
>> saying "hey nice sounds cool").
>>
>> As for the boot when I initially worked on external toolchain sparc64 was
>> my
>> guinea pig and so yes it worked an booted just fine.
>>
>
> So far as I know, we never solved any of the infrastructural problems
> associated with this concept:
> 1. Providing built releases with a /usr/bin/cc
> 2. Coversioning base and in-ports toolchain, including ensuring commit
> atomicity between toolchains and libc
> 3. Adding a dependency on ports for src, including out-of-tree code that
> has to be fetched from external servers
> 4. Getting make universe to do the right thing
>
> We really need to solve those. If we go the external toolchain route,
> which it is not clear to me is the best option, #2 and #1 are quite complex
> problems. I think that, frankly, a deadline in two months to solve this set
> of political problems we have had for two years is probably crazy, but
> maybe making the status quo unsustainable will finally force progress.
>

External toolchains have been in flight for 5 or more years now. It's time
they land. Though the requirements for them have never included
cross-threading between /usr/src and /usr/ports like you suggest above, and
those sorts of things won't be sorted by my deadlines (which are closer to
3 months). Nor, imho, should they.


> Also, are there any directions for how to build a toolchain from the base
> ports? I haven't been able to figure it out -- the README starts with "pkg
> install", but that doesn't get me very far on platforms without binary
> packages, which are the ones this is meant to target.


External toolchains are by definition external. Not part of the tree nor
integrated into the tree. How you get them depends on the external
toolchain and is unspecified. You get a lower level of integration than you
do with in-tree toolchains, not the same level with reach-overs into some
external repo.

Warner
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: future of sparc64 (was: Making C++11 a hard requirement for FreeBSD)

Peter Jeremy-8
In reply to this post by K. Macy
On 2017-Oct-07 19:06:29 +0000, "K. Macy" <[hidden email]> wrote:

>On Sat, Oct 7, 2017 at 10:41 Mark Linimon <[hidden email]> wrote:
>
>> On Thu, Oct 05, 2017 at 07:12:28PM -0500, A. Wilcox wrote:
>> > That doesn't change the fact that sparc64 still exists, and with Oracle
>> > laying off Solaris as well, FreeBSD becomes a "way out" for people
>> > heavily invested (DC full of sparc64 gear, or such).
>>
>> I have thought for some time that we've been a "way out" for Solaris
>> sites wanting to keep ZFS and not deal with licensing issues, and have
>> worked to keep sparc64 alive.  (AFAIK FreeBSD is the only open source
>> sparc64/zfs solution?)
AFAIK Illumos still supports sparc64 and is probably an easier
migration for Solaris sites so I don't think that argument holds.

Also, we run into the same situation we had with Alpha - a basically dead
architecture that only runs on old equipment.  Unless there's a critical
mass of FreeBSD developers that are willing to keep it running, we're better
off killing it quickly, rather than letting it soak up developer effort.

>My recollection of sparc64 from sun4v work was that unsupported operations
>would trap in to the kernel which would in turn trap in to a user space
>handler for floating point emulation.

Yes.  I did some poking at that some time ago.  The userland package is
basically a complete single/double/quad precision IEEE FP implementation
(see /usr/src/lib/libc/sparc64/fpu).  I have a test suite for it but it
hasn't been committed and I'd need to check if it's developed any bitrot.

--
Peter Jeremy

signature.asc (968 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Making C++11 a hard requirement for FreeBSD

Nathan Whitehorn-8
In reply to this post by Warner Losh


On 10/08/17 22:26, Warner Losh wrote:

>
>
> On Sun, Oct 8, 2017 at 11:01 PM, Nathan Whitehorn
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>
>
>     On 10/06/17 00:20, Baptiste Daroussin wrote:
>
>         On Thu, Oct 05, 2017 at 11:47:57PM +0000, John Baldwin wrote:
>
>             On Thursday, October 05, 2017 04:28:44 PM Warner Losh wrote:
>
>                 I'd like to start a conversation about the viability
>                 of making C++11 a hard
>                 requirement for bootstrapping FreeBSD and setting a
>                 specific deadline for
>                 doing so.
>
>                 This discussion is motivated by an ask from the
>                 jemalloc folks to use a
>                 limited subset of C++11 inside of malloc in such a way
>                 that is C safe (so
>                 the C programs wouldn't bloat with a C++ runtime).
>                 That's an ongoing
>                 discussion in another forum, and isn't appropriate for
>                 this thread because
>                 this has become a frequent request (but if you have
>                 opinions, please find
>                 the thread in current@ about it). I don't know the
>                 timeline of their plans
>                 to do this.
>
>                 I'd like to take the rather vague plans we've had
>                 "before 12" and create a
>                 timeline for removal of gcc 4.2 coupled with a
>                 requirement for support in
>                 clang or a working external toolchain. This
>                 requirement would be coupled
>                 with the requirement that the external toolchain
>                 support C++11 constructs.
>
>                 I'd like to propose we do this 12/31/17. Any
>                 architectures that can't meet
>                 this timeline will be disconnected from universe at
>                 that time and deleted
>                 3/31/18.
>
>                 It's my belief that i386, amd64, arm, aarch64, powerpc
>                 and powerpc64 are
>                 ready for this change and mips* would be ready for
>                 this change with an
>                 external clang toolchain. I'm unsure of riscv and
>                 sparc64, but suspect that
>                 a newer version of gcc as an external toolchain could
>                 work.
>
>             In-tree clang 5.0 for MIPS mostly works (modulo some small
>             patches).  However,
>             it requires external ld.bfd.  I know that there ld.lld can
>             link a working
>             mips64 world with some patches (notably the multigot
>             patch).  mips works fine
>             with external GCC.  riscv is already using external GCC
>             that is C++11-capable.
>
>             The problem with external GCC is that you can cross-build
>             a world + kernel
>             just fine and get a working system via
>             CROSS_TOOLCHAIN=foo-gcc.  However,
>             that system has no viable '/usr/bin/cc' once GCC 4.2 is
>             removed.  bapt@
>             started on ports to cross-build binutils and gcc packages
>             via the base/*
>             ports, but those are not yet finished / fully tested.  I
>             don't think anyone
>             has thought about how release builds will work either with
>             only an external
>             toolchain available.  (I'm pretty sure sparc64 has booted
>             fine with
>             external GCC, it's just in the same boat as mips with
>             regard to /usr/bin/cc.)
>
>         Actually I did test those and they were working (tested in
>         qemu) they were
>         working fine. I have given up working on them due to the lack
>         of interested by
>         the community (by interest I mean people really testing,
>         working on it, not just
>         saying "hey nice sounds cool").
>
>         As for the boot when I initially worked on external toolchain
>         sparc64 was my
>         guinea pig and so yes it worked an booted just fine.
>
>
>     So far as I know, we never solved any of the infrastructural
>     problems associated with this concept:
>     1. Providing built releases with a /usr/bin/cc
>     2. Coversioning base and in-ports toolchain, including ensuring
>     commit atomicity between toolchains and libc
>     3. Adding a dependency on ports for src, including out-of-tree
>     code that has to be fetched from external servers
>     4. Getting make universe to do the right thing
>
>     We really need to solve those. If we go the external toolchain
>     route, which it is not clear to me is the best option, #2 and #1
>     are quite complex problems. I think that, frankly, a deadline in
>     two months to solve this set of political problems we have had for
>     two years is probably crazy, but maybe making the status quo
>     unsustainable will finally force progress.
>
>
> External toolchains have been in flight for 5 or more years now. It's
> time they land. Though the requirements for them have never included
> cross-threading between /usr/src and /usr/ports like you suggest
> above, and those sorts of things won't be sorted by my deadlines
> (which are closer to 3 months). Nor, imho, should they.

Well, sure. But the fact remains that we cannot build usable systems
with external toolchains right now. Those are real problems that need to
be solved somehow.

Let's focus on #1, the largest if not the only major problem. If I
build, say, a ppc64 system with an external toolchain right now, it
boots and runs fine. But the system is completely unusable since:
- There are no binary packages built for PPC64, because of project
policy preventing the use of native build systems
- You cannot cross-compile packages for PPC64, because of limitations in
QEMU
- There is no compiler installed in the base system, so you cannot
install any software from source code
- You cannot build the compiler from source, because you don't have one
to bootstrap from and can't get one pre-built because of the no-packages
problem.

We can't ship systems like this -- how is anyone expected to be able to
use them?

These are easy to fix -- for example, here are three possibilities --
but solutions been held up for *years* by project policy:
1. Allow Tier-2 packages to be built on native hardware without as much
centralized control, letting pkg install of the toolchain work. This
fixes ppc64, but maybe not embedded systems.
2. Have external-toolchain builds cross-build (or native-build) and
pre-install a standard compiler package from ports during make
buildworld, which keeps the user experience of current tier-2 and later
tier-1 systems.
3. Include a newer compiler fully in the tree or in some nearby
repository, which does the same.

If we break any of these policy impasses by the deadline, I am 100% for
dropping GCC 4.2. But we *have* to do something. Just axing code while
preventing a meaningful replacement isn't a solution.

>     Also, are there any directions for how to build a toolchain from
>     the base ports? I haven't been able to figure it out -- the README
>     starts with "pkg install", but that doesn't get me very far on
>     platforms without binary packages, which are the ones this is
>     meant to target.
>
>
> External toolchains are by definition external. Not part of the tree
> nor integrated into the tree. How you get them depends on the external
> toolchain and is unspecified. You get a lower level of integration
> than you do with in-tree toolchains, not the same level with
> reach-overs into some external repo.

I think you missed the point. I was asking how to build the port to test
it -- it doesn't build at present and I can't figure out how to make an
external toolchain via the "base" ports (the other toolchains in ports
work fine).
-Nathan

> Warner

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Making C++11 a hard requirement for FreeBSD

Warner Losh
On Oct 9, 2017 11:47 AM, "Nathan Whitehorn" <[hidden email]> wrote:



On 10/08/17 22:26, Warner Losh wrote:



On Sun, Oct 8, 2017 at 11:01 PM, Nathan Whitehorn <[hidden email]>
wrote:

>
>
> On 10/06/17 00:20, Baptiste Daroussin wrote:
>
>> On Thu, Oct 05, 2017 at 11:47:57PM +0000, John Baldwin wrote:
>>
>>> On Thursday, October 05, 2017 04:28:44 PM Warner Losh wrote:
>>>
>>>> I'd like to start a conversation about the viability of making C++11 a
>>>> hard
>>>> requirement for bootstrapping FreeBSD and setting a specific deadline
>>>> for
>>>> doing so.
>>>>
>>>> This discussion is motivated by an ask from the jemalloc folks to use a
>>>> limited subset of C++11 inside of malloc in such a way that is C safe
>>>> (so
>>>> the C programs wouldn't bloat with a C++ runtime). That's an ongoing
>>>> discussion in another forum, and isn't appropriate for this thread
>>>> because
>>>> this has become a frequent request (but if you have opinions, please
>>>> find
>>>> the thread in current@ about it). I don't know the timeline of their
>>>> plans
>>>> to do this.
>>>>
>>>> I'd like to take the rather vague plans we've had "before 12" and
>>>> create a
>>>> timeline for removal of gcc 4.2 coupled with a requirement for support
>>>> in
>>>> clang or a working external toolchain. This requirement would be coupled
>>>> with the requirement that the external toolchain support C++11
>>>> constructs.
>>>>
>>>> I'd like to propose we do this 12/31/17. Any architectures that can't
>>>> meet
>>>> this timeline will be disconnected from universe at that time and
>>>> deleted
>>>> 3/31/18.
>>>>
>>>> It's my belief that i386, amd64, arm, aarch64, powerpc and powerpc64 are
>>>> ready for this change and mips* would be ready for this change with an
>>>> external clang toolchain. I'm unsure of riscv and sparc64, but suspect
>>>> that
>>>> a newer version of gcc as an external toolchain could work.
>>>>
>>> In-tree clang 5.0 for MIPS mostly works (modulo some small patches).
>>> However,
>>> it requires external ld.bfd.  I know that there ld.lld can link a working
>>> mips64 world with some patches (notably the multigot patch).  mips works
>>> fine
>>> with external GCC.  riscv is already using external GCC that is
>>> C++11-capable.
>>>
>>> The problem with external GCC is that you can cross-build a world +
>>> kernel
>>> just fine and get a working system via CROSS_TOOLCHAIN=foo-gcc.  However,
>>> that system has no viable '/usr/bin/cc' once GCC 4.2 is removed.  bapt@
>>> started on ports to cross-build binutils and gcc packages via the base/*
>>> ports, but those are not yet finished / fully tested.  I don't think
>>> anyone
>>> has thought about how release builds will work either with only an
>>> external
>>> toolchain available.  (I'm pretty sure sparc64 has booted fine with
>>> external GCC, it's just in the same boat as mips with regard to
>>> /usr/bin/cc.)
>>>
>> Actually I did test those and they were working (tested in qemu) they were
>> working fine. I have given up working on them due to the lack of
>> interested by
>> the community (by interest I mean people really testing, working on it,
>> not just
>> saying "hey nice sounds cool").
>>
>> As for the boot when I initially worked on external toolchain sparc64 was
>> my
>> guinea pig and so yes it worked an booted just fine.
>>
>
> So far as I know, we never solved any of the infrastructural problems
> associated with this concept:
> 1. Providing built releases with a /usr/bin/cc
> 2. Coversioning base and in-ports toolchain, including ensuring commit
> atomicity between toolchains and libc
> 3. Adding a dependency on ports for src, including out-of-tree code that
> has to be fetched from external servers
> 4. Getting make universe to do the right thing
>
> We really need to solve those. If we go the external toolchain route,
> which it is not clear to me is the best option, #2 and #1 are quite complex
> problems. I think that, frankly, a deadline in two months to solve this set
> of political problems we have had for two years is probably crazy, but
> maybe making the status quo unsustainable will finally force progress.
>

External toolchains have been in flight for 5 or more years now. It's time
they land. Though the requirements for them have never included
cross-threading between /usr/src and /usr/ports like you suggest above, and
those sorts of things won't be sorted by my deadlines (which are closer to
3 months). Nor, imho, should they.


Well, sure. But the fact remains that we cannot build usable systems with
external toolchains right now. Those are real problems that need to be
solved somehow.


Sure we can. I've built a bootable i386 system with gcc 6. It is a solved
problem.

Let's focus on #1, the largest if not the only major problem. If I build,
say, a ppc64 system with an external toolchain right now, it boots and runs
fine. But the system is completely unusable since:
- There are no binary packages built for PPC64, because of project policy
preventing the use of native build systems


System is still usable w/o packages. People can still fire up custom
poudrier repos. We could also change project policy. This is is a specific
wrinkle for powerpc64 it seems.

- You cannot cross-compile packages for PPC64, because of limitations in
QEMU


Then we should fix those, like we did for arm and MIPS.

- There is no compiler installed in the base system, so you cannot install
any software from source code


You can install it as a package.

- You cannot build the compiler from source, because you don't have one to
bootstrap from and can't get one pre-built because of the no-packages
problem.


If you fix the other problems, this is solved.

We can't ship systems like this -- how is anyone expected to be able to use
them?


Most systems don't build software, so there are plenty of uses.

These are easy to fix -- for example, here are three possibilities -- but
solutions been held up for *years* by project policy:
1. Allow Tier-2 packages to be built on native hardware without as much
centralized control, letting pkg install of the toolchain work. This fixes
ppc64, but maybe not embedded systems.


If we can't build in QEMU then sure. It doesn't matter where the binaries
come from, so long as they work.

2. Have external-toolchain builds cross-build (or native-build) and
pre-install a standard compiler package from ports during make buildworld,
which keeps the user experience of current tier-2 and later tier-1 systems.


No need for this to be during buildworld. I see no benefit from that. You
can install it after installworld either native or to a destdir.

3. Include a newer compiler fully in the tree or in some nearby repository,
which does the same.


Yea, that's also possible. But then it isn't an external toolchain.

If we break any of these policy impasses by the deadline, I am 100% for
dropping GCC 4.2. But we *have* to do something. Just axing code while
preventing a meaningful replacement isn't a solution.



We have sometthing that works today with a few warts. We shouldn't let the
warts get in the way.

I'm happy to delay if there are specific items that have a definitive
timeline, but wanting 100% full integration from external toolchain with
full packages isn't a gating factor here.

Besides, I thought powerpc64 just worked with clang...

Warner

Also, are there any directions for how to build a toolchain from the base
> ports? I haven't been able to figure it out -- the README starts with "pkg
> install", but that doesn't get me very far on platforms without binary
> packages, which are the ones this is meant to target.


External toolchains are by definition external. Not part of the tree nor
integrated into the tree. How you get them depends on the external
toolchain and is unspecified. You get a lower level of integration than you
do with in-tree toolchains, not the same level with reach-overs into some
external repo.


I think you missed the point. I was asking how to build the port to test it
-- it doesn't build at present and I can't figure out how to make an
external toolchain via the "base" ports (the other toolchains in ports work
fine).
-Nathan

Warner
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

status of powerpc64 (was: Making C++11 a hard requirement for FreeBSD)

Mark Linimon-2
In reply to this post by Nathan Whitehorn-8
On Mon, Oct 09, 2017 at 10:47:13AM -0700, Nathan Whitehorn wrote:
> - There are no binary packages built for PPC64, because of project policy
> preventing the use of native build systems

I don't think this is 100% correct.

Although I am no longer on portmgr and thus don't work on the details,
my understanding is that:

 - "official" FreeBSD packages can only be issued from machines that
   are under the exclusive control of FreeBSD.org
 - the powerpc64 machine that falls under that category is not yet
   reliable enough

The first is a security concern.  The odds of that policy changing are
about the same as Elvis doing another concert.

IIUC the second is because we run package builds under virtualization
on freebsd.org's powerpc64 machine, and hit memory contstraints often
enough to make it "not quite ready for production".  You would have to
ask swills@ whether the latter is still true.

My own powerpc64 builds are constrained for other reasons[*].  When I
get back from my current road trip I am willing to build and make
available a subset of powerpc64 packages on the same basis I currently
do for sparc64, if that will help the situation.

> - You cannot cross-compile packages for PPC64, because of limitations
> in QEMU

s/limitations/a bug/

qemu/powerpc64 simply hangs when you start it up (as does qemu/sparc64).
Otherwise qemu is usable for cross-builds and is, in fact, how armv6,
mips, and mips64 package builds are currently done.

I know less about your other points, but, there needs to be some kind
of script or wrapper around the "do a base cross-build".  I had had one
set up for amd64 to sparc64 as a test but got distracted before I finished.
I recall it being non-trivial.

mcl

[*] a couple of kernel bugs which I think I know how to fix; not being
able to run diskful, and power consumption
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
12