Speculative: Rust for base system components

classic Classic list List threaded Threaded
132 messages Options
1234 ... 7
Reply | Threaded
Open this post in threaded view
|

Speculative: Rust for base system components

Eric McCorkle-2
Before I begin, I want to be clear that everything here is in the realm
of speculative, long-term discussion.  My goal is to start a
conversation, not to propose anything concrete right now.



I've talked at several conferences about the possibility of bringing the
Rust programming language into the base system, with the intent of
making it a viable implementation language for parts of the base system.
 I believe the potential security and stability benefits of doing this
are *substantial*.  This would obviously be a serious undertaking, but I
think the benefits are worth the cost.  For the rest of this mail, I'll
outline my position and present the important facts that support it.

I of course welcome any discussion, criticism, or input.

= Why Rust? =

Rust is a true systems programming language that incorporates results
from programming language research that took place in the late 90's and
2000's, and can be viewed as a culmination of this work.  (Specifically,
it draws on work on region types as well as substructural types, but
that's its own survey article).

This manifests most clearly in Rust's lifetime type system, which is
able to perform quite sophisticated reasoning about memory allocation
and ownership.  Rust programs do not require you to explicitly manage
memory, but neither do they use garbage collection.  Memory management
is automatically tracked by the lifetime types and allocations/frees are
added during compilation.  Moreover, threaded programs are able to use
the same type system features to track ownership.  This effectively
prevents a large class of memory- and concurrency-related errors *at the
type system level*, resulting in dramatically safer programs.  Rust
provides an "unsafe" language facility for accomplishing low-level
tasks, so you're not prevented from getting things done; however, almost
all programs can be implemented without relying on unsafe, and doing so
eliminates what has probably been the two worst classes of bugs in the
history of computing.

Because this is done at the type system level, it has no cost at
runtime.  This theme of "zero-cost abstractions" runs throughout the
language, which makes it suitable as a replacement for C in the long
term.  It also incorporates features befitting a modern programming
language such as parameterized types (aka generics), pattern-matching,
and trait-based polymorphism.  Finally, it can interface with C/C++
binaries and libraries relatively easily, making it a suitable systems
implementation language.

Because of these benefits, several projects (Mozilla and Tor) have begun
to reimplement their systems using Rust.  Additionally, there is a
master's thesis about writing FreeBSD device drivers using Rust out
there somewhere (forgive me for not chasing it down).

A secondary benefit is that Rust's packaging system, cargo, is built
around a more modern concept of modularity.  This is in line with
FreeBSD's goals of modularizing the base system and reducing redundancy.

= Downsides =

There are a couple of downsides, which I'll outline briefly, and provide
some counterargumentation for them.

* It's a big component

Adding the entire Rust platform would add more to the buildworld time,
which is already quite long due to LLVM.  OTOH, LLVM itself is a big
chunk of the rust build time, and we already build that as part of the
base system now.  Moreover, it stands to reason that the compiler would
end up being the majority of an open-source operating system's build
time, with second place being the kernel itself.  Compilers are an
active area of research with no signs of slowing down, and more hardware
platforms means more backends, which means the compiler ends up growing
over time, while most of the rest of the system doesn't.

* The language is still evolving

While this is true, the language has gotten *way* more mature in the
past year and a half, and I think it's ready to hold up major codebases
and production use, or at the very least it will be by the time we'd be
seriously looking at bringing it in.

* Rust binaries are large

This issue can be avoided with the proper use of build flags.  The
default settings use static linking for everything, and bring in the
entire runtime library (which isn't necessary in most cases).  See this
article:
https://lifthrasiir.github.io/rustlog/why-is-a-rust-executable-large.html.
 With LTO, the size can be cut down to that of a C++ binary, and with
dynamic linking, the executable size is comparable to a C program.  If
we were seriously using rust, we would probably change the default flags
to use dynamic linking/LTO by default, in fact.


= Alternatives =

There have been several alternatives that come up when I mention this.
I'll discuss the two big ones: go and C++

== C++ ==

C++ is already in the base system, of course.  It could be brought into
the kernel with some effort and restrictions.  The biggest argument for
Rust over C++ in my view is that Rust is a much simpler, cleaner
language.  C++ at this point is bewilderingly complex as a language, and
has so many bells and whistles that it almost requires an effort to
decide which subset of the language will be used.  This also tends to
slow down efforts to expand the language over time.

Rust by comparison is much leaner.  Its lifetime type system can be
intimidating, but the benefits are well worth getting used to it.  To
sum it up, C++ certainly isn't going anywhere, but I think Rust has a
great deal more vitality as a language.

== Go ==

Go is not really a "systems language" in the sense that C or Rust are.
It isn't a replacement for C so much as a replacement for python.

What's more, it is almost universally disliked by people with a
background in programming languages (I am one of those).  There's a
couple of reasons for this.  First, it omits a number of features that
modern languages *really* should include, parameterized types (generics)
being one of the most glaring.  Beyond that, it completely ignores
anything produced by modern PL research (unlike Rust, which benefits
from that work).  Second, one of its main design parameters was "it
should compile fast".  This as a design constraint basically guarantees
that you can't do anything interesting in compilation, which rules out
both advanced type systems as well as serious optimization.

Unlike Rust, which offers substantial security and stability benefits
through its type system, Go would do little more than just change the
syntax of a bunch of code.  Moreover, it's likely that there would be
performance consequences, given the design constraint of fast compilation.

= Summary =

Rust is a modern systems language with substantial benefits, and is
well-poised to become the heir apparent to C as the dominant systems
language going into the future.  For this reason, I think we should
seriously consider it as a language for the base system at some point in
the future, when the time is right and after sufficient exploration and
experimentation.


signature.asc (235 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Speculative: Rust for base system components

Mark Linimon-2
Until it runs on every arch we support, it's kind of academic to talk
about it.  IMHO.

mcl
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Speculative: Rust for base system components

Alan Somers-2
In reply to this post by Eric McCorkle-2
On Sun, Dec 30, 2018 at 10:38 PM Eric McCorkle <[hidden email]> wrote:

>
> Before I begin, I want to be clear that everything here is in the realm
> of speculative, long-term discussion.  My goal is to start a
> conversation, not to propose anything concrete right now.
>
>
>
> I've talked at several conferences about the possibility of bringing the
> Rust programming language into the base system, with the intent of
> making it a viable implementation language for parts of the base system.
>  I believe the potential security and stability benefits of doing this
> are *substantial*.  This would obviously be a serious undertaking, but I
> think the benefits are worth the cost.  For the rest of this mail, I'll
> outline my position and present the important facts that support it.
>
> I of course welcome any discussion, criticism, or input.
>
> = Why Rust? =
>
> Rust is a true systems programming language that incorporates results
> from programming language research that took place in the late 90's and
> 2000's, and can be viewed as a culmination of this work.  (Specifically,
> it draws on work on region types as well as substructural types, but
> that's its own survey article).
>
> This manifests most clearly in Rust's lifetime type system, which is
> able to perform quite sophisticated reasoning about memory allocation
> and ownership.  Rust programs do not require you to explicitly manage
> memory, but neither do they use garbage collection.  Memory management
> is automatically tracked by the lifetime types and allocations/frees are
> added during compilation.  Moreover, threaded programs are able to use
> the same type system features to track ownership.  This effectively
> prevents a large class of memory- and concurrency-related errors *at the
> type system level*, resulting in dramatically safer programs.  Rust
> provides an "unsafe" language facility for accomplishing low-level
> tasks, so you're not prevented from getting things done; however, almost
> all programs can be implemented without relying on unsafe, and doing so
> eliminates what has probably been the two worst classes of bugs in the
> history of computing.
>
> Because this is done at the type system level, it has no cost at
> runtime.  This theme of "zero-cost abstractions" runs throughout the
> language, which makes it suitable as a replacement for C in the long
> term.  It also incorporates features befitting a modern programming
> language such as parameterized types (aka generics), pattern-matching,
> and trait-based polymorphism.  Finally, it can interface with C/C++
> binaries and libraries relatively easily, making it a suitable systems
> implementation language.
>
> Because of these benefits, several projects (Mozilla and Tor) have begun
> to reimplement their systems using Rust.  Additionally, there is a
> master's thesis about writing FreeBSD device drivers using Rust out
> there somewhere (forgive me for not chasing it down).

I would be interested to read that, if you can find the link.

>
> A secondary benefit is that Rust's packaging system, cargo, is built
> around a more modern concept of modularity.  This is in line with
> FreeBSD's goals of modularizing the base system and reducing redundancy.
>
> = Downsides =
>
> There are a couple of downsides, which I'll outline briefly, and provide
> some counterargumentation for them.
>
> * It's a big component
>
> Adding the entire Rust platform would add more to the buildworld time,
> which is already quite long due to LLVM.  OTOH, LLVM itself is a big
> chunk of the rust build time, and we already build that as part of the
> base system now.  Moreover, it stands to reason that the compiler would
> end up being the majority of an open-source operating system's build
> time, with second place being the kernel itself.  Compilers are an
> active area of research with no signs of slowing down, and more hardware
> platforms means more backends, which means the compiler ends up growing
> over time, while most of the rest of the system doesn't.
>
> * The language is still evolving
>
> While this is true, the language has gotten *way* more mature in the
> past year and a half, and I think it's ready to hold up major codebases
> and production use, or at the very least it will be by the time we'd be
> seriously looking at bringing it in.
>
> * Rust binaries are large
>
> This issue can be avoided with the proper use of build flags.  The
> default settings use static linking for everything, and bring in the
> entire runtime library (which isn't necessary in most cases).  See this
> article:
> https://lifthrasiir.github.io/rustlog/why-is-a-rust-executable-large.html.
>  With LTO, the size can be cut down to that of a C++ binary, and with
> dynamic linking, the executable size is comparable to a C program.  If
> we were seriously using rust, we would probably change the default flags
> to use dynamic linking/LTO by default, in fact.

Another downside: Rust doesn't support all FreeBSD architectures.
Right now, on FreeBSD it only runs on i386, amd64, and aarch64.  On
other OSes it supports additional architectures, though.  The biggest
requirement for porting Rust to a new architecture is LLVM support.
So we can't use Rust for almost anything in the base system until all
architectures are on LLVM.

>
>
> = Alternatives =
>
> There have been several alternatives that come up when I mention this.
> I'll discuss the two big ones: go and C++
>
> == C++ ==
>
> C++ is already in the base system, of course.  It could be brought into
> the kernel with some effort and restrictions.  The biggest argument for
> Rust over C++ in my view is that Rust is a much simpler, cleaner
> language.  C++ at this point is bewilderingly complex as a language, and
> has so many bells and whistles that it almost requires an effort to
> decide which subset of the language will be used.  This also tends to
> slow down efforts to expand the language over time.
>
> Rust by comparison is much leaner.  Its lifetime type system can be
> intimidating, but the benefits are well worth getting used to it.  To
> sum it up, C++ certainly isn't going anywhere, but I think Rust has a
> great deal more vitality as a language.

Yeah.  C++ has none of Rust's safety features.  For OS work, it has
few advantages over C.

>
> == Go ==
>
> Go is not really a "systems language" in the sense that C or Rust are.
> It isn't a replacement for C so much as a replacement for python.
>
> What's more, it is almost universally disliked by people with a
> background in programming languages (I am one of those).  There's a
> couple of reasons for this.  First, it omits a number of features that
> modern languages *really* should include, parameterized types (generics)
> being one of the most glaring.  Beyond that, it completely ignores
> anything produced by modern PL research (unlike Rust, which benefits
> from that work).  Second, one of its main design parameters was "it
> should compile fast".  This as a design constraint basically guarantees
> that you can't do anything interesting in compilation, which rules out
> both advanced type systems as well as serious optimization.
>
> Unlike Rust, which offers substantial security and stability benefits
> through its type system, Go would do little more than just change the
> syntax of a bunch of code.  Moreover, it's likely that there would be
> performance consequences, given the design constraint of fast compilation.

I think there's a common misconception that Go and Rust are similar
just because they were invented at about the same time and they share
some superficial syntactial similarities.  But Go is much less
suitable for an OS.  It's a memory-managed language, like Java.
Remember when Java was the hot new thing, and people predicted that
OSes would be written entirely in Java, and microprocessors would be
redesigned to execute JVM bytecode directly?  It never happened,
because Java just isn't suitable for low-level work.  Neither is Go.
Rust is.

>
> = Summary =
>
> Rust is a modern systems language with substantial benefits, and is
> well-poised to become the heir apparent to C as the dominant systems
> language going into the future.  For this reason, I think we should
> seriously consider it as a language for the base system at some point in
> the future, when the time is right and after sufficient exploration and
> experimentation.

+100.  Rust is definitely the best systems programming language since
C.  It's better than C++.  It's already being used in experimental
operating systems like RedoxOS, and I think it's going to become the
standard for most innovative OS work soon.  FreeBSD should adopt it;
the only question is when.  And the answer, I think, is after all
architectures are built with LLVM.  There will be some problems to
work out, of course, but it will be worth it.

-Alan

P.S. I was writing some Rust when I got this email
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Speculative: Rust for base system components

Conrad Meyer-2
In reply to this post by Eric McCorkle-2
Rust has many of the exact same problems C++ has.  Sure, it avoids a
few classes of issue.  But it doesn't fix the classic C++ problems of
extremely slow build times (if you think C++ is bad, Rust is worse)
and having a kitchen sink of language features that make Rust and C++
code difficult to grok.

It may be a suitable replacement for C++; it is not a suitable
replacement for C.  And outside of a few components (Clang and devd),
most of the base system does not use C++.

Adding more LLVM build time to the base system is fairly objectionable
even if Rust was magical in all other aspects.  Too much of the build
is LLVM today.

I don't think this is a viable long term plan even if Rust was ported
to all of our platforms (and I don't see that happening, either).

Best,
Conrad


On Sun, Dec 30, 2018 at 9:39 PM Eric McCorkle <[hidden email]> wrote:

>
> Before I begin, I want to be clear that everything here is in the realm
> of speculative, long-term discussion.  My goal is to start a
> conversation, not to propose anything concrete right now.
>
>
>
> I've talked at several conferences about the possibility of bringing the
> Rust programming language into the base system, with the intent of
> making it a viable implementation language for parts of the base system.
>  I believe the potential security and stability benefits of doing this
> are *substantial*.  This would obviously be a serious undertaking, but I
> think the benefits are worth the cost.  For the rest of this mail, I'll
> outline my position and present the important facts that support it.
>
> I of course welcome any discussion, criticism, or input.
>
> = Why Rust? =
>
> Rust is a true systems programming language that incorporates results
> from programming language research that took place in the late 90's and
> 2000's, and can be viewed as a culmination of this work.  (Specifically,
> it draws on work on region types as well as substructural types, but
> that's its own survey article).
>
> This manifests most clearly in Rust's lifetime type system, which is
> able to perform quite sophisticated reasoning about memory allocation
> and ownership.  Rust programs do not require you to explicitly manage
> memory, but neither do they use garbage collection.  Memory management
> is automatically tracked by the lifetime types and allocations/frees are
> added during compilation.  Moreover, threaded programs are able to use
> the same type system features to track ownership.  This effectively
> prevents a large class of memory- and concurrency-related errors *at the
> type system level*, resulting in dramatically safer programs.  Rust
> provides an "unsafe" language facility for accomplishing low-level
> tasks, so you're not prevented from getting things done; however, almost
> all programs can be implemented without relying on unsafe, and doing so
> eliminates what has probably been the two worst classes of bugs in the
> history of computing.
>
> Because this is done at the type system level, it has no cost at
> runtime.  This theme of "zero-cost abstractions" runs throughout the
> language, which makes it suitable as a replacement for C in the long
> term.  It also incorporates features befitting a modern programming
> language such as parameterized types (aka generics), pattern-matching,
> and trait-based polymorphism.  Finally, it can interface with C/C++
> binaries and libraries relatively easily, making it a suitable systems
> implementation language.
>
> Because of these benefits, several projects (Mozilla and Tor) have begun
> to reimplement their systems using Rust.  Additionally, there is a
> master's thesis about writing FreeBSD device drivers using Rust out
> there somewhere (forgive me for not chasing it down).
>
> A secondary benefit is that Rust's packaging system, cargo, is built
> around a more modern concept of modularity.  This is in line with
> FreeBSD's goals of modularizing the base system and reducing redundancy.
>
> = Downsides =
>
> There are a couple of downsides, which I'll outline briefly, and provide
> some counterargumentation for them.
>
> * It's a big component
>
> Adding the entire Rust platform would add more to the buildworld time,
> which is already quite long due to LLVM.  OTOH, LLVM itself is a big
> chunk of the rust build time, and we already build that as part of the
> base system now.  Moreover, it stands to reason that the compiler would
> end up being the majority of an open-source operating system's build
> time, with second place being the kernel itself.  Compilers are an
> active area of research with no signs of slowing down, and more hardware
> platforms means more backends, which means the compiler ends up growing
> over time, while most of the rest of the system doesn't.
>
> * The language is still evolving
>
> While this is true, the language has gotten *way* more mature in the
> past year and a half, and I think it's ready to hold up major codebases
> and production use, or at the very least it will be by the time we'd be
> seriously looking at bringing it in.
>
> * Rust binaries are large
>
> This issue can be avoided with the proper use of build flags.  The
> default settings use static linking for everything, and bring in the
> entire runtime library (which isn't necessary in most cases).  See this
> article:
> https://lifthrasiir.github.io/rustlog/why-is-a-rust-executable-large.html.
>  With LTO, the size can be cut down to that of a C++ binary, and with
> dynamic linking, the executable size is comparable to a C program.  If
> we were seriously using rust, we would probably change the default flags
> to use dynamic linking/LTO by default, in fact.
>
>
> = Alternatives =
>
> There have been several alternatives that come up when I mention this.
> I'll discuss the two big ones: go and C++
>
> == C++ ==
>
> C++ is already in the base system, of course.  It could be brought into
> the kernel with some effort and restrictions.  The biggest argument for
> Rust over C++ in my view is that Rust is a much simpler, cleaner
> language.  C++ at this point is bewilderingly complex as a language, and
> has so many bells and whistles that it almost requires an effort to
> decide which subset of the language will be used.  This also tends to
> slow down efforts to expand the language over time.
>
> Rust by comparison is much leaner.  Its lifetime type system can be
> intimidating, but the benefits are well worth getting used to it.  To
> sum it up, C++ certainly isn't going anywhere, but I think Rust has a
> great deal more vitality as a language.
>
> == Go ==
>
> Go is not really a "systems language" in the sense that C or Rust are.
> It isn't a replacement for C so much as a replacement for python.
>
> What's more, it is almost universally disliked by people with a
> background in programming languages (I am one of those).  There's a
> couple of reasons for this.  First, it omits a number of features that
> modern languages *really* should include, parameterized types (generics)
> being one of the most glaring.  Beyond that, it completely ignores
> anything produced by modern PL research (unlike Rust, which benefits
> from that work).  Second, one of its main design parameters was "it
> should compile fast".  This as a design constraint basically guarantees
> that you can't do anything interesting in compilation, which rules out
> both advanced type systems as well as serious optimization.
>
> Unlike Rust, which offers substantial security and stability benefits
> through its type system, Go would do little more than just change the
> syntax of a bunch of code.  Moreover, it's likely that there would be
> performance consequences, given the design constraint of fast compilation.
>
> = Summary =
>
> Rust is a modern systems language with substantial benefits, and is
> well-poised to become the heir apparent to C as the dominant systems
> language going into the future.  For this reason, I think we should
> seriously consider it as a language for the base system at some point in
> the future, when the time is right and after sufficient exploration and
> experimentation.
>
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Speculative: Rust for base system components

Gleb Popov-2
In reply to this post by Eric McCorkle-2
On Mon, Dec 31, 2018 at 8:38 AM Eric McCorkle <[hidden email]> wrote:

> Before I begin, I want to be clear that everything here is in the realm
> of speculative, long-term discussion.  My goal is to start a
> conversation, not to propose anything concrete right now.
>
>
>
> I've talked at several conferences about the possibility of bringing the
> Rust programming language into the base system, with the intent of
> making it a viable implementation language for parts of the base system.
>  I believe the potential security and stability benefits of doing this
> are *substantial*.  This would obviously be a serious undertaking, but I
> think the benefits are worth the cost.  For the rest of this mail, I'll
> outline my position and present the important facts that support it.
>
> I of course welcome any discussion, criticism, or input.
>
> = Why Rust? =
>
> Rust is a true systems programming language that incorporates results
> from programming language research that took place in the late 90's and
> 2000's, and can be viewed as a culmination of this work.  (Specifically,
> it draws on work on region types as well as substructural types, but
> that's its own survey article).
>
> This manifests most clearly in Rust's lifetime type system, which is
> able to perform quite sophisticated reasoning about memory allocation
> and ownership.  Rust programs do not require you to explicitly manage
> memory, but neither do they use garbage collection.  Memory management
> is automatically tracked by the lifetime types and allocations/frees are
> added during compilation.  Moreover, threaded programs are able to use
> the same type system features to track ownership.  This effectively
> prevents a large class of memory- and concurrency-related errors *at the
> type system level*, resulting in dramatically safer programs.  Rust
> provides an "unsafe" language facility for accomplishing low-level
> tasks, so you're not prevented from getting things done; however, almost
> all programs can be implemented without relying on unsafe, and doing so
> eliminates what has probably been the two worst classes of bugs in the
> history of computing.
>

If we are striving for safety through type system, I'd jump straight to
Haskell. And let the bikeshedding begin.


> Because this is done at the type system level, it has no cost at
> runtime.  This theme of "zero-cost abstractions" runs throughout the
> language, which makes it suitable as a replacement for C in the long
> term.  It also incorporates features befitting a modern programming
> language such as parameterized types (aka generics), pattern-matching,
> and trait-based polymorphism.  Finally, it can interface with C/C++
> binaries and libraries relatively easily, making it a suitable systems
> implementation language.
>
> Because of these benefits, several projects (Mozilla and Tor) have begun
> to reimplement their systems using Rust.  Additionally, there is a
> master's thesis about writing FreeBSD device drivers using Rust out
> there somewhere (forgive me for not chasing it down).
>
> A secondary benefit is that Rust's packaging system, cargo, is built
> around a more modern concept of modularity.  This is in line with
> FreeBSD's goals of modularizing the base system and reducing redundancy.
>
> = Downsides =
>
> There are a couple of downsides, which I'll outline briefly, and provide
> some counterargumentation for them.
>
> * It's a big component
>
> Adding the entire Rust platform would add more to the buildworld time,
> which is already quite long due to LLVM.  OTOH, LLVM itself is a big
> chunk of the rust build time, and we already build that as part of the
> base system now.  Moreover, it stands to reason that the compiler would
> end up being the majority of an open-source operating system's build
> time, with second place being the kernel itself.  Compilers are an
> active area of research with no signs of slowing down, and more hardware
> platforms means more backends, which means the compiler ends up growing
> over time, while most of the rest of the system doesn't.
>
> * The language is still evolving
>
> While this is true, the language has gotten *way* more mature in the
> past year and a half, and I think it's ready to hold up major codebases
> and production use, or at the very least it will be by the time we'd be
> seriously looking at bringing it in.
>
> * Rust binaries are large
>
> This issue can be avoided with the proper use of build flags.  The
> default settings use static linking for everything, and bring in the
> entire runtime library (which isn't necessary in most cases).  See this
> article:
> https://lifthrasiir.github.io/rustlog/why-is-a-rust-executable-large.html.
>  With LTO, the size can be cut down to that of a C++ binary, and with
> dynamic linking, the executable size is comparable to a C program.  If
> we were seriously using rust, we would probably change the default flags
> to use dynamic linking/LTO by default, in fact.
>
>
> = Alternatives =
>
> There have been several alternatives that come up when I mention this.
> I'll discuss the two big ones: go and C++
>
> == C++ ==
>
> C++ is already in the base system, of course.  It could be brought into
> the kernel with some effort and restrictions.  The biggest argument for
> Rust over C++ in my view is that Rust is a much simpler, cleaner
> language.  C++ at this point is bewilderingly complex as a language, and
> has so many bells and whistles that it almost requires an effort to
> decide which subset of the language will be used.  This also tends to
> slow down efforts to expand the language over time.
>
> Rust by comparison is much leaner.  Its lifetime type system can be
> intimidating, but the benefits are well worth getting used to it.  To
> sum it up, C++ certainly isn't going anywhere, but I think Rust has a
> great deal more vitality as a language.
>
> == Go ==
>
> Go is not really a "systems language" in the sense that C or Rust are.
> It isn't a replacement for C so much as a replacement for python.
>
> What's more, it is almost universally disliked by people with a
> background in programming languages (I am one of those).  There's a
> couple of reasons for this.  First, it omits a number of features that
> modern languages *really* should include, parameterized types (generics)
> being one of the most glaring.  Beyond that, it completely ignores
> anything produced by modern PL research (unlike Rust, which benefits
> from that work).  Second, one of its main design parameters was "it
> should compile fast".  This as a design constraint basically guarantees
> that you can't do anything interesting in compilation, which rules out
> both advanced type systems as well as serious optimization.
>
> Unlike Rust, which offers substantial security and stability benefits
> through its type system, Go would do little more than just change the
> syntax of a bunch of code.  Moreover, it's likely that there would be
> performance consequences, given the design constraint of fast compilation.
>
> = Summary =
>
> Rust is a modern systems language with substantial benefits, and is
> well-poised to become the heir apparent to C as the dominant systems
> language going into the future.  For this reason, I think we should
> seriously consider it as a language for the base system at some point in
> the future, when the time is right and after sufficient exploration and
> experimentation.
>
>
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Speculative: Rust for base system components

Poul-Henning Kamp
In reply to this post by Eric McCorkle-2
--------
In message <[hidden email]>, Eric McCorkl
e writes:

>Before I begin, I want to be clear that everything here is in the realm
>of speculative, long-term discussion.  My goal is to start a
>conversation, not to propose anything concrete right now.

Tl;Dr:  Forget all about it.


The historical precedent is cvssup(1) which was written in Modula-3.

Cvsup was a very complex piece of machinery, and it was indisputably
incredibly important to the projects productivity, and we talked again and
again about including it in the tree.

The arguments were roughly (from memory:)

1. It would make porting to new platforms much harder. (M3 had its
   own threading implementation.)

2. Nobody but jdp@ knew or used M3, and he only used it for cvsup,
   because he wanted to try out the language in the first place.

3. It would increas buildworld time significantly.

4. Maintaining the M3 compiler in the tree would be a bother.

5. The only advantage over having it as a port were "It would be
   (more) convenient"

We never imported cvsup and it was not even a close call.


I think we can generalize and say that unless we are talking about
very big, complex and inescapable body of code, any potential benefits
will not outweigh the very concrete disadvantages.


So exactly which "base system components" are we talking about ?

The largest non-contrib program we maintain in the tree, is ppp(8)
and that is only 43KLOC.

That is not enough code to warrant a refactoring into a different
programming language, in particular not when usage is so low that
nobody has even bothered to merge the multi-link support from
net/mpd5 in the last 10 years.

So the only piece of code I can imagine which would ever come close
to qualifying, would be if somebody starts writing BSystemD(8)
from scratch.

And I'm 100% convinced that people will want that optional and firmly
segregated in a port for at least the first a decade.

And as far as I know, we *are* trying to make base more modular, and
migrate it to pkgbase to make the attachment of/to ports more
seamless, right?

Poul-Henning

--
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[hidden email]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Speculative: Rust for base system components

Igor Chudov

Hello!

That sounds nice. Is there any requirements or wiki page for BSystemD implementation?

---
With best regards, Igor Chudov.
Saratov Free Software Center
Work phone: +7 8452 98-78-18
Work (mobile) phone: +7 917 208-78-18
Personal (mobile) phone: +7 937 266-51-34


31.12.2018, 10:40, "Poul-Henning Kamp" <[hidden email]>:

> So the only piece of code I can imagine which would ever come close
> to qualifying, would be if somebody starts writing BSystemD(8)
> from scratch.
>
> And I'm 100% convinced that people will want that optional and firmly
> segregated in a port for at least the first a decade.
>
> And as far as I know, we *are* trying to make base more modular, and
> migrate it to pkgbase to make the attachment of/to ports more
> seamless, right?

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Speculative: Rust for base system components

Poul-Henning Kamp
--------
In message <[hidden email]>, =?utf-8?B?
0JjQs9C+0YDRjCDQp9GD0LTQvtCy?= writes:

>That sounds nice. Is there any requirements or wiki page for BSystemD implementation?

I think that subject is best tackled in an small working group of
interested people, so the "how" question does not get steam-rolled by the
"why" question.

Please don't hi-jack this mail-thread for it.

--
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[hidden email]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Speculative: Rust for base system components

Damjan Jovanovic
In reply to this post by Alan Somers-2
On Mon, Dec 31, 2018 at 8:37 AM Alan Somers <[hidden email]> wrote:

> On Sun, Dec 30, 2018 at 10:38 PM Eric McCorkle <[hidden email]>
> wrote:
> >
>
> >
> > == Go ==
> >
> > Go is not really a "systems language" in the sense that C or Rust are.
> > It isn't a replacement for C so much as a replacement for python.
> >
> > What's more, it is almost universally disliked by people with a
> > background in programming languages (I am one of those).  There's a
> > couple of reasons for this.  First, it omits a number of features that
> > modern languages *really* should include, parameterized types (generics)
> > being one of the most glaring.  Beyond that, it completely ignores
> > anything produced by modern PL research (unlike Rust, which benefits
> > from that work).  Second, one of its main design parameters was "it
> > should compile fast".  This as a design constraint basically guarantees
> > that you can't do anything interesting in compilation, which rules out
> > both advanced type systems as well as serious optimization.
> >
> > Unlike Rust, which offers substantial security and stability benefits
> > through its type system, Go would do little more than just change the
> > syntax of a bunch of code.  Moreover, it's likely that there would be
> > performance consequences, given the design constraint of fast
> compilation.
>
> I think there's a common misconception that Go and Rust are similar
> just because they were invented at about the same time and they share
> some superficial syntactial similarities.  But Go is much less
> suitable for an OS.  It's a memory-managed language, like Java.
> Remember when Java was the hot new thing, and people predicted that
> OSes would be written entirely in Java, and microprocessors would be
> redesigned to execute JVM bytecode directly?  It never happened,
> because Java just isn't suitable for low-level work.  Neither is Go.
> Rust is.
>
>
ARM chips could (and some did) have the Jazelle DBX extension to run Java
bytecode in hardware. There are Java editions used on SIM cards and Blu-ray
disks, so it certainly has low-level editions. It's Java SE that is heavy:
7 threads and 50 MB RAM for a hello world application, last time I checked.

In this regard, Go is much lighter: 2 threads and 1 MB RAM for a hello
world. Go has unsigned types, structure embedding, excellent
interoperability with C, and the ability to compile into shared libraries
which are loaded and called from C, so it is a lot better for low-level
programming than Java. Having said that, the number of architectures
supported by Go is fairly limited, and languages with automated garbage
collection are unusable for kernel development.

I would like to see more Rust (or a similar language) in systems
development.

Damjan
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Speculative: Rust for base system components

Gary Jennejohn-6
In reply to this post by Alan Somers-2
On Sun, 30 Dec 2018 23:03:08 -0700
Alan Somers <[hidden email]> wrote:

> On Sun, Dec 30, 2018 at 10:38 PM Eric McCorkle <[hidden email]> wrote:
[big snip]
> >
> > Because of these benefits, several projects (Mozilla and Tor) have begun
> > to reimplement their systems using Rust.  Additionally, there is a
> > master's thesis about writing FreeBSD device drivers using Rust out
> > there somewhere (forgive me for not chasing it down).  
>
> I would be interested to read that, if you can find the link.
>

I did a google search using "writing FreeBSD device drivers using Rust"
and the first hit was a link to the PDF.  Here it is:

www.diva-portal.org/smash/get/diva2:1238890/FULLTEXT01.pdf

[big snip]

--
Gary Jennejohn
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Speculative: Rust for base system components

Bjoern A. Zeeb
In reply to this post by Poul-Henning Kamp
On 31 Dec 2018, at 7:36, Poul-Henning Kamp wrote:

> Tl;Dr:  Forget all about it.
>
>
> The historical precedent is cvssup(1) which was written in Modula-3.

While we are on the topic, what was the history on how FreeBSD ended up
with perl in the base system?

/bz
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Speculative: Rust for base system components

Jan Knepper-2
In reply to this post by Eric McCorkle-2
What about D?

www.dlang.org



ManiaC++
Jan Knepper

> On Dec 30, 2018, at 21:18, Eric McCorkle <[hidden email]> wrote:
>
> Before I begin, I want to be clear that everything here is in the realm
> of speculative, long-term discussion.  My goal is to start a
> conversation, not to propose anything concrete right now.
>
>
>
> I've talked at several conferences about the possibility of bringing the
> Rust programming language into the base system, with the intent of
> making it a viable implementation language for parts of the base system.
> I believe the potential security and stability benefits of doing this
> are *substantial*.  This would obviously be a serious undertaking, but I
> think the benefits are worth the cost.  For the rest of this mail, I'll
> outline my position and present the important facts that support it.
>
> I of course welcome any discussion, criticism, or input.
>
> = Why Rust? =
>
> Rust is a true systems programming language that incorporates results
> from programming language research that took place in the late 90's and
> 2000's, and can be viewed as a culmination of this work.  (Specifically,
> it draws on work on region types as well as substructural types, but
> that's its own survey article).
>
> This manifests most clearly in Rust's lifetime type system, which is
> able to perform quite sophisticated reasoning about memory allocation
> and ownership.  Rust programs do not require you to explicitly manage
> memory, but neither do they use garbage collection.  Memory management
> is automatically tracked by the lifetime types and allocations/frees are
> added during compilation.  Moreover, threaded programs are able to use
> the same type system features to track ownership.  This effectively
> prevents a large class of memory- and concurrency-related errors *at the
> type system level*, resulting in dramatically safer programs.  Rust
> provides an "unsafe" language facility for accomplishing low-level
> tasks, so you're not prevented from getting things done; however, almost
> all programs can be implemented without relying on unsafe, and doing so
> eliminates what has probably been the two worst classes of bugs in the
> history of computing.
>
> Because this is done at the type system level, it has no cost at
> runtime.  This theme of "zero-cost abstractions" runs throughout the
> language, which makes it suitable as a replacement for C in the long
> term.  It also incorporates features befitting a modern programming
> language such as parameterized types (aka generics), pattern-matching,
> and trait-based polymorphism.  Finally, it can interface with C/C++
> binaries and libraries relatively easily, making it a suitable systems
> implementation language.
>
> Because of these benefits, several projects (Mozilla and Tor) have begun
> to reimplement their systems using Rust.  Additionally, there is a
> master's thesis about writing FreeBSD device drivers using Rust out
> there somewhere (forgive me for not chasing it down).
>
> A secondary benefit is that Rust's packaging system, cargo, is built
> around a more modern concept of modularity.  This is in line with
> FreeBSD's goals of modularizing the base system and reducing redundancy.
>
> = Downsides =
>
> There are a couple of downsides, which I'll outline briefly, and provide
> some counterargumentation for them.
>
> * It's a big component
>
> Adding the entire Rust platform would add more to the buildworld time,
> which is already quite long due to LLVM.  OTOH, LLVM itself is a big
> chunk of the rust build time, and we already build that as part of the
> base system now.  Moreover, it stands to reason that the compiler would
> end up being the majority of an open-source operating system's build
> time, with second place being the kernel itself.  Compilers are an
> active area of research with no signs of slowing down, and more hardware
> platforms means more backends, which means the compiler ends up growing
> over time, while most of the rest of the system doesn't.
>
> * The language is still evolving
>
> While this is true, the language has gotten *way* more mature in the
> past year and a half, and I think it's ready to hold up major codebases
> and production use, or at the very least it will be by the time we'd be
> seriously looking at bringing it in.
>
> * Rust binaries are large
>
> This issue can be avoided with the proper use of build flags.  The
> default settings use static linking for everything, and bring in the
> entire runtime library (which isn't necessary in most cases).  See this
> article:
> https://lifthrasiir.github.io/rustlog/why-is-a-rust-executable-large.html.
> With LTO, the size can be cut down to that of a C++ binary, and with
> dynamic linking, the executable size is comparable to a C program.  If
> we were seriously using rust, we would probably change the default flags
> to use dynamic linking/LTO by default, in fact.
>
>
> = Alternatives =
>
> There have been several alternatives that come up when I mention this.
> I'll discuss the two big ones: go and C++
>
> == C++ ==
>
> C++ is already in the base system, of course.  It could be brought into
> the kernel with some effort and restrictions.  The biggest argument for
> Rust over C++ in my view is that Rust is a much simpler, cleaner
> language.  C++ at this point is bewilderingly complex as a language, and
> has so many bells and whistles that it almost requires an effort to
> decide which subset of the language will be used.  This also tends to
> slow down efforts to expand the language over time.
>
> Rust by comparison is much leaner.  Its lifetime type system can be
> intimidating, but the benefits are well worth getting used to it.  To
> sum it up, C++ certainly isn't going anywhere, but I think Rust has a
> great deal more vitality as a language.
>
> == Go ==
>
> Go is not really a "systems language" in the sense that C or Rust are.
> It isn't a replacement for C so much as a replacement for python.
>
> What's more, it is almost universally disliked by people with a
> background in programming languages (I am one of those).  There's a
> couple of reasons for this.  First, it omits a number of features that
> modern languages *really* should include, parameterized types (generics)
> being one of the most glaring.  Beyond that, it completely ignores
> anything produced by modern PL research (unlike Rust, which benefits
> from that work).  Second, one of its main design parameters was "it
> should compile fast".  This as a design constraint basically guarantees
> that you can't do anything interesting in compilation, which rules out
> both advanced type systems as well as serious optimization.
>
> Unlike Rust, which offers substantial security and stability benefits
> through its type system, Go would do little more than just change the
> syntax of a bunch of code.  Moreover, it's likely that there would be
> performance consequences, given the design constraint of fast compilation.
>
> = Summary =
>
> Rust is a modern systems language with substantial benefits, and is
> well-poised to become the heir apparent to C as the dominant systems
> language going into the future.  For this reason, I think we should
> seriously consider it as a language for the base system at some point in
> the future, when the time is right and after sufficient exploration and
> experimentation.
>
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Speculative: Rust for base system components

Bakul Shah
In reply to this post by Eric McCorkle-2
On Sun, 30 Dec 2018 21:18:33 -0500 Eric McCorkle <[hidden email]> wrote:

>
> Go is not really a "systems language" in the sense that C or Rust are.
> It isn't a replacement for C so much as a replacement for python.
>
> What's more, it is almost universally disliked by people with a
> background in programming languages (I am one of those).  There's a
> couple of reasons for this.  First, it omits a number of features that
> modern languages *really* should include, parameterized types (generics)
> being one of the most glaring.  Beyond that, it completely ignores
> anything produced by modern PL research (unlike Rust, which benefits
> from that work).  Second, one of its main design parameters was "it
> should compile fast".  This as a design constraint basically guarantees
> that you can't do anything interesting in compilation, which rules out
> both advanced type systems as well as serious optimization.
>
> Unlike Rust, which offers substantial security and stability benefits
> through its type system, Go would do little more than just change the
> syntax of a bunch of code.  Moreover, it's likely that there would be
> performance consequences, given the design constraint of fast compilation

You haven't used Go at all, have you? No one is suggesting Go be
included in the base system but please stop spreading FUD.
While this is not the place to discuss Go, it is perfectly
usable for writing typical "systems" programs. Though I
wouldn't recommend it for writing kernel code.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Speculative: Rust for base system components

Poul-Henning Kamp
In reply to this post by Bjoern A. Zeeb
--------
In message <[hidden email]>, "Bjoern A
. Zeeb" writes:
>On 31 Dec 2018, at 7:36, Poul-Henning Kamp wrote:
>
>> Tl;Dr:  Forget all about it.
>>
>>
>> The historical precedent is cvssup(1) which was written in Modula-3.
>
>While we are on the topic, what was the history on how FreeBSD ended up
>with perl in the base system?

We wanted a scripting language stronger than /bin/sh, but preferably
without the kitchen-sink, guest-bathroom, sewer pipelines and wastewater
treatment plant,

One very desired use-case was to write the installer in a better
language than C, but other interesting ideas were floating
around too.  For instance inetd(8) or syslogd(8) could be given a
$script-program as "action" etc.

I can't remember if we had Forth in the boot code back then, but
it was certainly ruled out for this purpose by people who had grown
up on an inferior brand of pocket calculator :-)

We tried to import Tcl first, but that didn't work out, for N-1
values of "work out".

The one thing that worked was the one which made us pick Tcl:  It
was very small language designed for exactly that kind of embedding,
so the actual integration into the source tree wasn't too horrible.

Of course the version churn caused by John Ousterhout & Scriptics
trying to become DotCom millionaires gave some heartburn, but it
was mostly manageable.

However, everybody hated Tcl for all sorts of reasons, and since
this was the DotCom years, they hated it most of all because it was
not the "Endlösung for all programming requirements: Perl".

So we ripped out Tcl again and imported Perl, despite significant
misgivings, all if which came to fruition sooner or later.

Interesting, as soon as we had imported perl, all the much talked
about use-cases evaporated, and it never saw much actual use in
the tree.

Lua was brand new back then, and I dont remember if it was even
mentioned, it certainly was not in the running.

Rexx had no FOSS implementation, so that was not a candidate.

Importing bash(1) or ksh(1) was also proposed as a "half-way pregnant"
solution, but licenses were in the way.

--
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[hidden email]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Speculative: Rust for base system components

Eric McCorkle-2
In reply to this post by Poul-Henning Kamp
On 12/31/18 2:36 AM, Poul-Henning Kamp wrote:

> So exactly which "base system components" are we talking about ?
>
> The largest non-contrib program we maintain in the tree, is ppp(8)
> and that is only 43KLOC.
>
> That is not enough code to warrant a refactoring into a different
> programming language, in particular not when usage is so low that
> nobody has even bothered to merge the multi-link support from
> net/mpd5 in the last 10 years.
>
> So the only piece of code I can imagine which would ever come close
> to qualifying, would be if somebody starts writing BSystemD(8)
> from scratch.
>
> And I'm 100% convinced that people will want that optional and firmly
> segregated in a port for at least the first a decade.
>
> And as far as I know, we *are* trying to make base more modular, and
> migrate it to pkgbase to make the attachment of/to ports more
> seamless, right?
That's an interesting point.  If the modularization effort ends up
providing the ability to replace parts of contrib with certain ports,
then it would be rather straightforward to create rust-based alternatives.

As for what I'd be keen to rewrite, I'd put security-critical bits like,
say, pam, kerberos, su, and such on the list.  I've also kicked around
the idea of trying to get a simple Rust-based EFI boot loader up and
going.  All of these would benefit from

WRT kerberos, I did have the base heimdal replaced with MIT kerberos
from ports for a while.  This mostly works (and has some benefits), but
unfortunately doesn't seem to be able to support kerberized NFS.


signature.asc (235 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Speculative: Rust for base system components

Igor Mozolevsky-2
In reply to this post by Eric McCorkle-2
On Mon, 31 Dec 2018 at 05:42, Eric McCorkle <[hidden email]> wrote:
>
> Before I begin, I want to be clear that everything here is in the realm
> of speculative, long-term discussion.  My goal is to start a
> conversation, not to propose anything concrete right now.


Does the generated code size increase or decrease and does the
data-path get longer or shorter with switch from C to Rust with all
the "great features" of Rust? Are there any metrics for these? If a
CPU cache gets trashed repeatedly or data takes massive d-tours that
would be a significant downside, wouldn't it?


--
Igor M.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Speculative: Rust for base system components

Warner Losh
In reply to this post by Eric McCorkle-2
On Sun, Dec 30, 2018 at 10:41 PM Eric McCorkle <[hidden email]> wrote:

> Before I begin, I want to be clear that everything here is in the realm
> of speculative, long-term discussion.  My goal is to start a
> conversation, not to propose anything concrete right now.
>

Today, this is a losing bid. The cost for rust is high (both in terms of
people and added compile time), it's not well supported on all our
architectures (and its robustness on the ones it does support has only been
tested in limited scenarios), and there's 0 software it enables from day
one. Plus, since it's a fast evolving language, we'll still need the ports
to support those things that use it today since the likelihood of a version
mismatch is high (and supporting 1 version would be a big stretch, multiple
version is right out). So any sane cost / benefit analysis says: way more
cost than benefit, forget about it. We simply don't have the man-power to
maintain a high-cost, zero-benefit component in the tree. Lord knows we
have a lot of non-zero-cost-with-almost-zero-benefit things in the tree
today that we need to get rid of.

In the future, when there are actual replacement things written, or there
are new features written, that will shift the cost / benefit equation. And
the circumstances about what makes up base will also have shifted, if we're
lucky, and we'll be able to have a conversation. We imported perl and tcl
on the speculative notion that people would build great things. That never
really panned out, and they became a high-cost burden to keep modern for
only minor benefit. And version skew in Perl was terrible by the end. Forth
and Lua live in the tree because they have benefit (though Forth will be
departing, most likely by 13, and definitely by 14). They are also small
and easy to update to new versions.

And we can't say, with certainty, that if a bunch of rust things show up
we'll use them in base. We'll have to see what they provide to benefit the
system.

TBH, there's a stronger case for python than rust: there's actual python
scripts in the tree today that we have to install a port to use. And there
the benefit, while not zero, is small and the effort is large compared to
just dragging it in as a port, so it hasn't been done. It's another fast
evolving language that requires multiple versions as well...

So write something that everybody wants, that must be in base, and that
requires rust, and then we can have the conversation...

Warner
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Speculative: Rust for base system components

Warner Losh
On Mon, Dec 31, 2018 at 8:02 AM Warner Losh <[hidden email]> wrote:

>
>
> On Sun, Dec 30, 2018 at 10:41 PM Eric McCorkle <[hidden email]>
> wrote:
>
>> Before I begin, I want to be clear that everything here is in the realm
>> of speculative, long-term discussion.  My goal is to start a
>> conversation, not to propose anything concrete right now.
>>
>
> Today, this is a losing bid. The cost for rust is high (both in terms of
> people and added compile time), it's not well supported on all our
> architectures (and its robustness on the ones it does support has only been
> tested in limited scenarios), and there's 0 software it enables from day
> one. Plus, since it's a fast evolving language, we'll still need the ports
> to support those things that use it today since the likelihood of a version
> mismatch is high (and supporting 1 version would be a big stretch, multiple
> version is right out). So any sane cost / benefit analysis says: way more
> cost than benefit, forget about it. We simply don't have the man-power to
> maintain a high-cost, zero-benefit component in the tree. Lord knows we
> have a lot of non-zero-cost-with-almost-zero-benefit things in the tree
> today that we need to get rid of.
>
> In the future, when there are actual replacement things written, or there
> are new features written, that will shift the cost / benefit equation. And
> the circumstances about what makes up base will also have shifted, if we're
> lucky, and we'll be able to have a conversation. We imported perl and tcl
> on the speculative notion that people would build great things. That never
> really panned out, and they became a high-cost burden to keep modern for
> only minor benefit. And version skew in Perl was terrible by the end. Forth
> and Lua live in the tree because they have benefit (though Forth will be
> departing, most likely by 13, and definitely by 14). They are also small
> and easy to update to new versions.
>
> And we can't say, with certainty, that if a bunch of rust things show up
> we'll use them in base. We'll have to see what they provide to benefit the
> system.
>
> TBH, there's a stronger case for python than rust: there's actual python
> scripts in the tree today that we have to install a port to use. And there
> the benefit, while not zero, is small and the effort is large compared to
> just dragging it in as a port, so it hasn't been done. It's another fast
> evolving language that requires multiple versions as well...
>
> So write something that everybody wants, that must be in base, and that
> requires rust, and then we can have the conversation...
>

Just re-read this. It sounds a little more negative than I wanted to come
off. I'd only wanted to say today the needle is on 'nope' and I hope people
write enough cool stuff to justify moving the needle off 'nope' :) The last
part of that message seems more muted than I wanted.

Warner
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Speculative: Rust for base system components

Volker Lendecke
In reply to this post by Conrad Meyer-2
On Sun, Dec 30, 2018 at 10:56:11PM -0800, Conrad Meyer wrote:
> I don't think this is a viable long term plan even if Rust was ported
> to all of our platforms (and I don't see that happening, either).

Not being involved in FreeBSD but in Samba, I could see a migration of
parts of Samba to Rust if it had a Rust->C translator. I think for
FreeBSD base components similar arguments would apply. You could ship
the C output for the platforms that don't have native Rust.

Unfortunately the llvm C backend was cancelled a while ago.

Volker
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Speculative: Rust for base system components

Alan Somers-2
In reply to this post by Conrad Meyer-2
On Mon, Dec 31, 2018 at 12:17 AM Conrad Meyer <[hidden email]> wrote:
>
> Rust has many of the exact same problems C++ has.  Sure, it avoids a
> few classes of issue.  But it doesn't fix the classic C++ problems of
> extremely slow build times (if you think C++ is bad, Rust is worse)
> and having a kitchen sink of language features that make Rust and C++
> code difficult to grok.

Do you have any data to support the claim of slow build times?  I know
it's very hard to do an apples-to-apples comparison of build times
between different languages, but I would be interested to see somebody
try.

>
> It may be a suitable replacement for C++; it is not a suitable
> replacement for C.  And outside of a few components (Clang and devd),
> most of the base system does not use C++.
>
> Adding more LLVM build time to the base system is fairly objectionable
> even if Rust was magical in all other aspects.  Too much of the build
> is LLVM today.
>
> I don't think this is a viable long term plan even if Rust was ported
> to all of our platforms (and I don't see that happening, either).
>
> Best,
> Conrad
>
>
> On Sun, Dec 30, 2018 at 9:39 PM Eric McCorkle <[hidden email]> wrote:
> >
> > Before I begin, I want to be clear that everything here is in the realm
> > of speculative, long-term discussion.  My goal is to start a
> > conversation, not to propose anything concrete right now.
> >
> >
> >
> > I've talked at several conferences about the possibility of bringing the
> > Rust programming language into the base system, with the intent of
> > making it a viable implementation language for parts of the base system.
> >  I believe the potential security and stability benefits of doing this
> > are *substantial*.  This would obviously be a serious undertaking, but I
> > think the benefits are worth the cost.  For the rest of this mail, I'll
> > outline my position and present the important facts that support it.
> >
> > I of course welcome any discussion, criticism, or input.
> >
> > = Why Rust? =
> >
> > Rust is a true systems programming language that incorporates results
> > from programming language research that took place in the late 90's and
> > 2000's, and can be viewed as a culmination of this work.  (Specifically,
> > it draws on work on region types as well as substructural types, but
> > that's its own survey article).
> >
> > This manifests most clearly in Rust's lifetime type system, which is
> > able to perform quite sophisticated reasoning about memory allocation
> > and ownership.  Rust programs do not require you to explicitly manage
> > memory, but neither do they use garbage collection.  Memory management
> > is automatically tracked by the lifetime types and allocations/frees are
> > added during compilation.  Moreover, threaded programs are able to use
> > the same type system features to track ownership.  This effectively
> > prevents a large class of memory- and concurrency-related errors *at the
> > type system level*, resulting in dramatically safer programs.  Rust
> > provides an "unsafe" language facility for accomplishing low-level
> > tasks, so you're not prevented from getting things done; however, almost
> > all programs can be implemented without relying on unsafe, and doing so
> > eliminates what has probably been the two worst classes of bugs in the
> > history of computing.
> >
> > Because this is done at the type system level, it has no cost at
> > runtime.  This theme of "zero-cost abstractions" runs throughout the
> > language, which makes it suitable as a replacement for C in the long
> > term.  It also incorporates features befitting a modern programming
> > language such as parameterized types (aka generics), pattern-matching,
> > and trait-based polymorphism.  Finally, it can interface with C/C++
> > binaries and libraries relatively easily, making it a suitable systems
> > implementation language.
> >
> > Because of these benefits, several projects (Mozilla and Tor) have begun
> > to reimplement their systems using Rust.  Additionally, there is a
> > master's thesis about writing FreeBSD device drivers using Rust out
> > there somewhere (forgive me for not chasing it down).
> >
> > A secondary benefit is that Rust's packaging system, cargo, is built
> > around a more modern concept of modularity.  This is in line with
> > FreeBSD's goals of modularizing the base system and reducing redundancy.
> >
> > = Downsides =
> >
> > There are a couple of downsides, which I'll outline briefly, and provide
> > some counterargumentation for them.
> >
> > * It's a big component
> >
> > Adding the entire Rust platform would add more to the buildworld time,
> > which is already quite long due to LLVM.  OTOH, LLVM itself is a big
> > chunk of the rust build time, and we already build that as part of the
> > base system now.  Moreover, it stands to reason that the compiler would
> > end up being the majority of an open-source operating system's build
> > time, with second place being the kernel itself.  Compilers are an
> > active area of research with no signs of slowing down, and more hardware
> > platforms means more backends, which means the compiler ends up growing
> > over time, while most of the rest of the system doesn't.
> >
> > * The language is still evolving
> >
> > While this is true, the language has gotten *way* more mature in the
> > past year and a half, and I think it's ready to hold up major codebases
> > and production use, or at the very least it will be by the time we'd be
> > seriously looking at bringing it in.
> >
> > * Rust binaries are large
> >
> > This issue can be avoided with the proper use of build flags.  The
> > default settings use static linking for everything, and bring in the
> > entire runtime library (which isn't necessary in most cases).  See this
> > article:
> > https://lifthrasiir.github.io/rustlog/why-is-a-rust-executable-large.html.
> >  With LTO, the size can be cut down to that of a C++ binary, and with
> > dynamic linking, the executable size is comparable to a C program.  If
> > we were seriously using rust, we would probably change the default flags
> > to use dynamic linking/LTO by default, in fact.
> >
> >
> > = Alternatives =
> >
> > There have been several alternatives that come up when I mention this.
> > I'll discuss the two big ones: go and C++
> >
> > == C++ ==
> >
> > C++ is already in the base system, of course.  It could be brought into
> > the kernel with some effort and restrictions.  The biggest argument for
> > Rust over C++ in my view is that Rust is a much simpler, cleaner
> > language.  C++ at this point is bewilderingly complex as a language, and
> > has so many bells and whistles that it almost requires an effort to
> > decide which subset of the language will be used.  This also tends to
> > slow down efforts to expand the language over time.
> >
> > Rust by comparison is much leaner.  Its lifetime type system can be
> > intimidating, but the benefits are well worth getting used to it.  To
> > sum it up, C++ certainly isn't going anywhere, but I think Rust has a
> > great deal more vitality as a language.
> >
> > == Go ==
> >
> > Go is not really a "systems language" in the sense that C or Rust are.
> > It isn't a replacement for C so much as a replacement for python.
> >
> > What's more, it is almost universally disliked by people with a
> > background in programming languages (I am one of those).  There's a
> > couple of reasons for this.  First, it omits a number of features that
> > modern languages *really* should include, parameterized types (generics)
> > being one of the most glaring.  Beyond that, it completely ignores
> > anything produced by modern PL research (unlike Rust, which benefits
> > from that work).  Second, one of its main design parameters was "it
> > should compile fast".  This as a design constraint basically guarantees
> > that you can't do anything interesting in compilation, which rules out
> > both advanced type systems as well as serious optimization.
> >
> > Unlike Rust, which offers substantial security and stability benefits
> > through its type system, Go would do little more than just change the
> > syntax of a bunch of code.  Moreover, it's likely that there would be
> > performance consequences, given the design constraint of fast compilation.
> >
> > = Summary =
> >
> > Rust is a modern systems language with substantial benefits, and is
> > well-poised to become the heir apparent to C as the dominant systems
> > language going into the future.  For this reason, I think we should
> > seriously consider it as a language for the base system at some point in
> > the future, when the time is right and after sufficient exploration and
> > experimentation.
> >
> _______________________________________________
> [hidden email] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "[hidden email]"
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
1234 ... 7