External GCC Update

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

External GCC Update

John Baldwin
I was recently able to install base/binutils and base/gcc into an amd64 VM
and do a self-hosted build and install.  Some of the port patches have been
committed from this, but I have some source patches before the final ports
patches can be finished.

The source patches are here:
https://github.com/bsdjhb/freebsd/compare/master...base_gcc

They do a couple of things I'd like some feedback on:

1) MK_GDB no longer depends on MK_BINUTILS so that /usr/libexec/gdb can
   still be built/installed when WITHOUT_BINUTILS=yes is true

2) WITH_BASE_GCC and WITH_BASE_BINUTILS knobs can be set in src.conf to
   ensure that 'make delete-old' doesn't delete files installed by the
   base/* packages if you also set WITHOUT_BINUTILS=yes, and similar
   knobs (because you don't want to build/install the ones from src)

3) I add support for an /etc/src.conf.d dir that can hold files that get
   treated as if they are part of /etc/src.conf.  The current patch on
   github for this only fixes world and not yet kern.pre.mk and probably
   needs the most review if we want to go forward with this route.  With
   this, I plan to have the base/* packages install suitable files in this
   dir that disable build of the src-based components and also set
   WITH_BASE_<foo> to make sure 'delete-old' DTRT.

The file for base/binutils would be:

CROSS_BINUTILS_PREFIX=/usr/bin/
WITH_BASE_BINUTILS=yes
WITHOUT_BINUTILS=yes
WITHOUT_LLD_IS_LD=yes

The file for base/gcc would be:

XCC=/usr/bin/cc
XCXX=/usr/bin/c++
XCPP=/usr/bin/cpp
X_COMPILER_TYPE=gcc
WITH_BASE_GCC=yes
WITHOUT_GCC=yes
WITHOUT_CLANG_IS_CC=yes

Thoughts?

--
John Baldwin

                                                                            
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: External GCC Update

Rodney W. Grimes-4
> I was recently able to install base/binutils and base/gcc into an amd64 VM
> and do a self-hosted build and install.  Some of the port patches have been
> committed from this, but I have some source patches before the final ports
> patches can be finished.
>
> The source patches are here:
> https://github.com/bsdjhb/freebsd/compare/master...base_gcc

Phabricator?

> They do a couple of things I'd like some feedback on:
>
> 1) MK_GDB no longer depends on MK_BINUTILS so that /usr/libexec/gdb can
>    still be built/installed when WITHOUT_BINUTILS=yes is true
>
> 2) WITH_BASE_GCC and WITH_BASE_BINUTILS knobs can be set in src.conf to
>    ensure that 'make delete-old' doesn't delete files installed by the
>    base/* packages if you also set WITHOUT_BINUTILS=yes, and similar
>    knobs (because you don't want to build/install the ones from src)
>
> 3) I add support for an /etc/src.conf.d dir that can hold files that get
>    treated as if they are part of /etc/src.conf.  The current patch on
>    github for this only fixes world and not yet kern.pre.mk and probably
>    needs the most review if we want to go forward with this route.  With
>    this, I plan to have the base/* packages install suitable files in this
>    dir that disable build of the src-based components and also set
>    WITH_BASE_<foo> to make sure 'delete-old' DTRT.

This last one should probably be broke out and also discussed on -arch,
it effects more than just the tool chain.

> The file for base/binutils would be:
>
> CROSS_BINUTILS_PREFIX=/usr/bin/
> WITH_BASE_BINUTILS=yes
> WITHOUT_BINUTILS=yes
> WITHOUT_LLD_IS_LD=yes
>
> The file for base/gcc would be:
>
> XCC=/usr/bin/cc
> XCXX=/usr/bin/c++
> XCPP=/usr/bin/cpp
> X_COMPILER_TYPE=gcc
> WITH_BASE_GCC=yes
> WITHOUT_GCC=yes
> WITHOUT_CLANG_IS_CC=yes
>
> Thoughts?
>
> --
> John Baldwin
--
Rod Grimes                                                 [hidden email]
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: External GCC Update

John Baldwin
On 2/22/19 11:45 AM, Rodney W. Grimes wrote:
>> I was recently able to install base/binutils and base/gcc into an amd64 VM
>> and do a self-hosted build and install.  Some of the port patches have been
>> committed from this, but I have some source patches before the final ports
>> patches can be finished.
>>
>> The source patches are here:
>> https://github.com/bsdjhb/freebsd/compare/master...base_gcc
>
> Phabricator?

Eventually, wanted a first cut of the entire patchset in context to see if
folks run screaming or not.

--
John Baldwin

                                                                            
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: External GCC Update

Warner Losh
On Fri, Feb 22, 2019, 5:09 PM John Baldwin <[hidden email]> wrote:

> On 2/22/19 11:45 AM, Rodney W. Grimes wrote:
> >> I was recently able to install base/binutils and base/gcc into an amd64
> VM
> >> and do a self-hosted build and install.  Some of the port patches have
> been
> >> committed from this, but I have some source patches before the final
> ports
> >> patches can be finished.
> >>
> >> The source patches are here:
> >> https://github.com/bsdjhb/freebsd/compare/master...base_gcc
> >
> > Phabricator?
>
> Eventually, wanted a first cut of the entire patchset in context to see if
> folks run screaming or not.
>


Thank you. Phabricator isn't good with larger patches. Git let's me see
things in a number of different views that are hard with the one size fits
all phab ui.

Warner

--

> John Baldwin
>
>
>
> _______________________________________________
> [hidden email] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
> To unsubscribe, send any mail to "
> [hidden email]"
>
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: External GCC Update

Brandon Bergren
In reply to this post by John Baldwin


On Fri, Feb 22, 2019, at 1:01 PM, John Baldwin wrote:

> I was recently able to install base/binutils and base/gcc into an amd64 VM
> and do a self-hosted build and install.  Some of the port patches have been
> committed from this, but I have some source patches before the final ports
> patches can be finished.
>
> The source patches are here:
> https://github.com/bsdjhb/freebsd/compare/master...base_gcc
>
> They do a couple of things I'd like some feedback on:
>
> 1) MK_GDB no longer depends on MK_BINUTILS so that /usr/libexec/gdb can
>    still be built/installed when WITHOUT_BINUTILS=yes is true

Good thinking.

> 2) WITH_BASE_GCC and WITH_BASE_BINUTILS knobs can be set in src.conf to
>    ensure that 'make delete-old' doesn't delete files installed by the
>    base/* packages if you also set WITHOUT_BINUTILS=yes, and similar
>    knobs (because you don't want to build/install the ones from src)

I was really confused about the naming when I read through the diff. Bikeshedding but I think WITH_PORTS_BASE_BINUTILS / WITH_PORTS_BASE_GCC would help quite a lot cognitively, to differentiate between "base as in in-tree binutils" and "base as in the base/binutils port"

> 3) I add support for an /etc/src.conf.d dir that can hold files that get
>    treated as if they are part of /etc/src.conf.  The current patch on
>    github for this only fixes world and not yet kern.pre.mk and probably
>    needs the most review if we want to go forward with this route.  With
>    this, I plan to have the base/* packages install suitable files in this
>    dir that disable build of the src-based components and also set
>    WITH_BASE_<foo> to make sure 'delete-old' DTRT.

Not sure if I like this. Can't src.opts.mk just call `pkg info -e base/binutils` and so forth and use the exit result to adjust the defaults?

> The file for base/binutils would be:
>
> CROSS_BINUTILS_PREFIX=/usr/bin/
> WITH_BASE_BINUTILS=yes
> WITHOUT_BINUTILS=yes
> WITHOUT_LLD_IS_LD=yes
>
> The file for base/gcc would be:
>
> XCC=/usr/bin/cc
> XCXX=/usr/bin/c++
> XCPP=/usr/bin/cpp
> X_COMPILER_TYPE=gcc
> WITH_BASE_GCC=yes
> WITHOUT_GCC=yes
> WITHOUT_CLANG_IS_CC=yes

I don't like the concept of packages messing with anything related to src.conf. I have a bunch of conditional stuff in mine broken out by ${TARGET_ARCH} and extra config suddenly appearing would break a lot of my cross compiling stuff, even if it is in a separate *.d folder.

Seems to me that just influencing src.opts.mk's defaults would be more robust.

>
> Thoughts?
>
> --
> John Baldwin
>
>                                                                             
>

--
  Brandon Bergren
Technical Generalist
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: External GCC Update

Rodney W. Grimes-4
In reply to this post by Warner Losh
> On Fri, Feb 22, 2019, 5:09 PM John Baldwin <[hidden email]> wrote:
>
> > On 2/22/19 11:45 AM, Rodney W. Grimes wrote:
> > >> I was recently able to install base/binutils and base/gcc into an amd64
> > VM
> > >> and do a self-hosted build and install.  Some of the port patches have
> > been
> > >> committed from this, but I have some source patches before the final
> > ports
> > >> patches can be finished.
> > >>
> > >> The source patches are here:
> > >> https://github.com/bsdjhb/freebsd/compare/master...base_gcc
> > >
> > > Phabricator?
> >
> > Eventually, wanted a first cut of the entire patchset in context to see if
> > folks run screaming or not.

Huh? It is 5 files and not even 200 lines of diff???
My first Phab review for CPU topology was 10 files and over 300 lines.

>
> Thank you. Phabricator isn't good with larger patches. Git let's me see
> things in a number of different views that are hard with the one size fits
> all phab ui.

Its rather hypocritical for core to announce a "recomendation to do reviews,
and the tool of choice is phabricator" and then have 2 core team members
advocate a code review in git just a short time later.

This sets bad examples from the top :-(

> Warner
> > John Baldwin
--
Rod Grimes                                                 [hidden email]
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: External GCC Update

John Baldwin
On 2/22/19 8:05 PM, Rodney W. Grimes wrote:

>> On Fri, Feb 22, 2019, 5:09 PM John Baldwin <[hidden email]> wrote:
>>
>>> On 2/22/19 11:45 AM, Rodney W. Grimes wrote:
>>>>> I was recently able to install base/binutils and base/gcc into an amd64
>>> VM
>>>>> and do a self-hosted build and install.  Some of the port patches have
>>> been
>>>>> committed from this, but I have some source patches before the final
>>> ports
>>>>> patches can be finished.
>>>>>
>>>>> The source patches are here:
>>>>> https://github.com/bsdjhb/freebsd/compare/master...base_gcc
>>>>
>>>> Phabricator?
>>>
>>> Eventually, wanted a first cut of the entire patchset in context to see if
>>> folks run screaming or not.
>
> Huh? It is 5 files and not even 200 lines of diff???
> My first Phab review for CPU topology was 10 files and over 300 lines.

The "run screaming" is about the ideas, not the amount of code.  It's similar
to posting to arch@ to say "I have this idea and proof-of-concept, does this
look like the right path so I should spend time refining it into the a
review-ready product, or should I drop it".  I never said I would not use
phab, so stop putting words in my mouth.  This is not unusual project
practice to get "idea" review before detailed code review.

>> Thank you. Phabricator isn't good with larger patches. Git let's me see
>> things in a number of different views that are hard with the one size fits
>> all phab ui.
>
> Its rather hypocritical for core to announce a "recomendation to do reviews,
> and the tool of choice is phabricator" and then have 2 core team members
> advocate a code review in git just a short time later.
>
> This sets bad examples from the top :-(

No.  Phab is the preferred tool, but it is not the only tool as was clearly
noted in the recent recommendation.  As I said above, I will push the
actual patches for review to phab when the time comes, but they aren't
ready for that yet and I'm trying to get a review of the ideas to determine
how to spend my time.
--
John Baldwin

                                                                            
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: External GCC Update

John Baldwin
In reply to this post by Brandon Bergren
On 2/22/19 6:03 PM, Brandon Bergren wrote:

>
>
> On Fri, Feb 22, 2019, at 1:01 PM, John Baldwin wrote:
>> I was recently able to install base/binutils and base/gcc into an amd64 VM
>> and do a self-hosted build and install.  Some of the port patches have been
>> committed from this, but I have some source patches before the final ports
>> patches can be finished.
>>
>> The source patches are here:
>> https://github.com/bsdjhb/freebsd/compare/master...base_gcc
>>
>> They do a couple of things I'd like some feedback on:
>>
>> 1) MK_GDB no longer depends on MK_BINUTILS so that /usr/libexec/gdb can
>>    still be built/installed when WITHOUT_BINUTILS=yes is true
>
> Good thinking.
>
>> 2) WITH_BASE_GCC and WITH_BASE_BINUTILS knobs can be set in src.conf to
>>    ensure that 'make delete-old' doesn't delete files installed by the
>>    base/* packages if you also set WITHOUT_BINUTILS=yes, and similar
>>    knobs (because you don't want to build/install the ones from src)
>
> I was really confused about the naming when I read through the diff. Bikeshedding but I think WITH_PORTS_BASE_BINUTILS / WITH_PORTS_BASE_GCC would help quite a lot cognitively, to differentiate between "base as in in-tree binutils" and "base as in the base/binutils port"

In the future there won't be a "base as in in-tree binutils", only the
special ports from base/ that are used to build components of the base
system that come as packages, but I don't care strongly about the names.

>> 3) I add support for an /etc/src.conf.d dir that can hold files that get
>>    treated as if they are part of /etc/src.conf.  The current patch on
>>    github for this only fixes world and not yet kern.pre.mk and probably
>>    needs the most review if we want to go forward with this route.  With
>>    this, I plan to have the base/* packages install suitable files in this
>>    dir that disable build of the src-based components and also set
>>    WITH_BASE_<foo> to make sure 'delete-old' DTRT.
>
> Not sure if I like this. Can't src.opts.mk just call `pkg info -e base/binutils` and so forth and use the exit result to adjust the defaults?

That requires src.opts.mk to encode the policy that each package wants to
enforce rather than letting the package choose the policy it wants to
enforce.  I think we want the latter.
 

>> The file for base/binutils would be:
>>
>> CROSS_BINUTILS_PREFIX=/usr/bin/
>> WITH_BASE_BINUTILS=yes
>> WITHOUT_BINUTILS=yes
>> WITHOUT_LLD_IS_LD=yes
>>
>> The file for base/gcc would be:
>>
>> XCC=/usr/bin/cc
>> XCXX=/usr/bin/c++
>> XCPP=/usr/bin/cpp
>> X_COMPILER_TYPE=gcc
>> WITH_BASE_GCC=yes
>> WITHOUT_GCC=yes
>> WITHOUT_CLANG_IS_CC=yes
>
> I don't like the concept of packages messing with anything related to src.conf. I have a bunch of conditional stuff in mine broken out by ${TARGET_ARCH} and extra config suddenly appearing would break a lot of my cross compiling stuff, even if it is in a separate *.d folder.
>
> Seems to me that just influencing src.opts.mk's defaults would be more robust.

Hmm, cross compiling is indeed a bear.  My original version of this was to
have base/gcc install a special 'freebsd-gcc.mk' toolchain file to
/usr/share/toolchains and modify Makefile.inc1 to use this as the default
CROSS_TOOLCHAIN if present.  I mostly didn't like this because it would be
a single file that so you can't set separate policy if, for example, some
arch or install only wanted base/binutils and not base/gcc.  On the other
hand, it had the advantage that setting an explicit CROSS_TOOLCHAIN when you
are cross compiling would work correctly.

Perhaps I can rework this to use two files in /usr/share/toolchains and have
Makefile.inc1 explicitly include any files in that directory if
CROSS_TOOLCHAIN isn't set?
--
John Baldwin

                                                                            
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: External GCC Update

Brooks Davis-2
On Mon, Feb 25, 2019 at 10:50:40AM -0800, John Baldwin wrote:

> On 2/22/19 6:03 PM, Brandon Bergren wrote:
> >
> >
> > On Fri, Feb 22, 2019, at 1:01 PM, John Baldwin wrote:
> >> 3) I add support for an /etc/src.conf.d dir that can hold files that get
> >>    treated as if they are part of /etc/src.conf.  The current patch on
> >>    github for this only fixes world and not yet kern.pre.mk and probably
> >>    needs the most review if we want to go forward with this route.  With
> >>    this, I plan to have the base/* packages install suitable files in this
> >>    dir that disable build of the src-based components and also set
> >>    WITH_BASE_<foo> to make sure 'delete-old' DTRT.
> >
> > Not sure if I like this. Can't src.opts.mk just call `pkg info -e base/binutils` and so forth and use the exit result to adjust the defaults?
>
> That requires src.opts.mk to encode the policy that each package wants to
> enforce rather than letting the package choose the policy it wants to
> enforce.  I think we want the latter.
>  
> >> The file for base/binutils would be:
> >>
> >> CROSS_BINUTILS_PREFIX=/usr/bin/
> >> WITH_BASE_BINUTILS=yes
> >> WITHOUT_BINUTILS=yes
> >> WITHOUT_LLD_IS_LD=yes
> >>
> >> The file for base/gcc would be:
> >>
> >> XCC=/usr/bin/cc
> >> XCXX=/usr/bin/c++
> >> XCPP=/usr/bin/cpp
> >> X_COMPILER_TYPE=gcc
> >> WITH_BASE_GCC=yes
> >> WITHOUT_GCC=yes
> >> WITHOUT_CLANG_IS_CC=yes
> >
> > I don't like the concept of packages messing with anything related to src.conf. I have a bunch of conditional stuff in mine broken out by ${TARGET_ARCH} and extra config suddenly appearing would break a lot of my cross compiling stuff, even if it is in a separate *.d folder.
> >
> > Seems to me that just influencing src.opts.mk's defaults would be more robust.
>
> Hmm, cross compiling is indeed a bear.  My original version of this was to
> have base/gcc install a special 'freebsd-gcc.mk' toolchain file to
> /usr/share/toolchains and modify Makefile.inc1 to use this as the default
> CROSS_TOOLCHAIN if present.  I mostly didn't like this because it would be
> a single file that so you can't set separate policy if, for example, some
> arch or install only wanted base/binutils and not base/gcc.  On the other
> hand, it had the advantage that setting an explicit CROSS_TOOLCHAIN when you
> are cross compiling would work correctly.
>
> Perhaps I can rework this to use two files in /usr/share/toolchains and have
> Makefile.inc1 explicitly include any files in that directory if
> CROSS_TOOLCHAIN isn't set?
I think I like that option best.

Another way to deal with the two-files issue would be to have a
base/toolchain metaport with options that installs the consolidated file
you want.  That mirrors (somewhat) the setup in devel/*xtoolchain*, but
I'm not convinced it won't just lead to confusion.

-- Brooks

signature.asc (465 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: External GCC Update

David Chisnall-2
In reply to this post by John Baldwin
On 22/02/2019 19:00, John Baldwin wrote:
> 3) I add support for an /etc/src.conf.d dir that can hold files that get
>     treated as if they are part of /etc/src.conf.  The current patch on
>     github for this only fixes world and not yet kern.pre.mk and probably
>     needs the most review if we want to go forward with this route.  With
>     this, I plan to have the base/* packages install suitable files in this
>     dir that disable build of the src-based components and also set
>     WITH_BASE_<foo> to make sure 'delete-old' DTRT.

Having a file outside of both the source and build directories that
controls aspects of the build is already a cause of significant pain
when building multiple different configurations of the FreeBSD base
system.  This sounds as if it would make things considerably worse.

David
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: External GCC Update

John Baldwin
In reply to this post by Brooks Davis-2
On 2/25/19 12:24 PM, Brooks Davis wrote:

> On Mon, Feb 25, 2019 at 10:50:40AM -0800, John Baldwin wrote:
>> Hmm, cross compiling is indeed a bear.  My original version of this was to
>> have base/gcc install a special 'freebsd-gcc.mk' toolchain file to
>> /usr/share/toolchains and modify Makefile.inc1 to use this as the default
>> CROSS_TOOLCHAIN if present.  I mostly didn't like this because it would be
>> a single file that so you can't set separate policy if, for example, some
>> arch or install only wanted base/binutils and not base/gcc.  On the other
>> hand, it had the advantage that setting an explicit CROSS_TOOLCHAIN when you
>> are cross compiling would work correctly.
>>
>> Perhaps I can rework this to use two files in /usr/share/toolchains and have
>> Makefile.inc1 explicitly include any files in that directory if
>> CROSS_TOOLCHAIN isn't set?
>
> I think I like that option best.
>
> Another way to deal with the two-files issue would be to have a
> base/toolchain metaport with options that installs the consolidated file
> you want.  That mirrors (somewhat) the setup in devel/*xtoolchain*, but
> I'm not convinced it won't just lead to confusion.

I've rebased and repushed the 'base_gcc' branch again to follow this approach.
Rather than using a glob, it just hardcodes the two possible files.  I did
have to make one change which is that the helper files have to use 'export'
for the WITH/WITHOUT variables or they weren't being honored in child makes.
However, this approach works even for 'make buildenv' in my testing.

--
John Baldwin
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "[hidden email]"