[HEADSUP] Disallowing read() of a directory fd

classic Classic list List threaded Threaded
45 messages Options
123
Reply | Threaded
Open this post in threaded view
|

[HEADSUP] Disallowing read() of a directory fd

Kyle Evans-3
Hi,

This is a heads up, given that I'm completely flipping our historical
behavior- I intend to commit this review in a couple days' time
without substantial objection: https://reviews.freebsd.org/D24596

With this, FreeBSD 13 will not allow read() of a directory fd, which
could have previously returned some data from the underlying
filesystem in no particular standardized format.

This is a still-standards-compliant switch from one
implementation-defined behavior to another that's already been adopted
in various other popular kernels, to include OpenBSD, MacOS, and
Linux.

Worth noting is that there's not really one largely-compelling reasons
to switch this after so many years (unless you find yourself that
irate when you accidentally `cat` a directory), but there are some
benefits which are briefly discussed in the commentary around the
review along with the history of the current behavior.

This change also simplifies filesystem implementations to some extent.

Thanks,

Kyle Evans
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [HEADSUP] Disallowing read() of a directory fd

Poul-Henning Kamp
--------
In message <[hidden email]>

>This is a heads up, given that I'm completely flipping our historical
>behavior- I intend to commit this review in a couple days' time
>without substantial objection: https://reviews.freebsd.org/D24596

When we did UFS2, I tried to persuade Kirk and Robert that since
directories had the 'x' bit set anyway, they start with a magic
sequence of:

    0x23 0x21 0x2f 0x62 0x69 0x6e 0x2f 0x6c 0x73 0x20 0x2d 0x6c 0x0a

As you can probably guess, they nixed my idea, and now you're making
it impossible for ever :-(

--
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[hidden email]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [HEADSUP] Disallowing read() of a directory fd

Cy Schubert-4
In reply to this post by Kyle Evans-3
In message <[hidden email]
om>
, Kyle Evans writes:

> Hi,
>
> This is a heads up, given that I'm completely flipping our historical
> behavior- I intend to commit this review in a couple days' time
> without substantial objection: https://reviews.freebsd.org/D24596
>
> With this, FreeBSD 13 will not allow read() of a directory fd, which
> could have previously returned some data from the underlying
> filesystem in no particular standardized format.
>
> This is a still-standards-compliant switch from one
> implementation-defined behavior to another that's already been adopted
> in various other popular kernels, to include OpenBSD, MacOS, and
> Linux.
>
> Worth noting is that there's not really one largely-compelling reasons
> to switch this after so many years (unless you find yourself that
> irate when you accidentally `cat` a directory), but there are some
> benefits which are briefly discussed in the commentary around the
> review along with the history of the current behavior.
>
> This change also simplifies filesystem implementations to some extent.

OpenBSD has done this for a while and more importantly Linux.


--
Cheers,
Cy Schubert <[hidden email]>
FreeBSD UNIX:  <[hidden email]>   Web:  https://FreeBSD.org
NTP:           <[hidden email]>    Web:  https://nwtime.org

        The need of the many outweighs the greed of the few.


_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [HEADSUP] Disallowing read() of a directory fd

Julian H. Stacey-3
In reply to this post by Kyle Evans-3
Kyle Evans wrote:

> Hi,
>
> This is a heads up, given that I'm completely flipping our historical
> behavior- I intend to commit this review in a couple days' time
> without substantial objection: https://reviews.freebsd.org/D24596
>
> With this, FreeBSD 13 will not allow read() of a directory fd, which
> could have previously returned some data from the underlying
> filesystem in no particular standardized format.
>
> This is a still-standards-compliant switch from one
> implementation-defined behavior to another that's already been adopted
> in various other popular kernels, to include OpenBSD, MacOS, and
> Linux.
>
> Worth noting is that there's not really one largely-compelling reasons
> to switch this after so many years (unless you find yourself that
> irate when you accidentally `cat` a directory), but there are some
> benefits which are briefly discussed in the commentary around the
> review along with the history of the current behavior.
>
> This change also simplifies filesystem implementations to some extent.
>
> Thanks,
>
> Kyle Evans

There is ZERO need for a spurious change at 2 days notice after 42+ years !

"cat ." as been supported since Unix V6 1978 or earlier,
no problem, even occasionaly useful.

Most FreeBSD users wont have heard of https://reviews.freebsd.org/D24596
(& there's only 5 other people's opinions there, apart from proposer,
& skimming through the impression is far from un-qualified approval.

kevans@ issued arch@ just a couple days notice of intention to commit.
arch@ is also not widely read, ( I jhs@ added CC hackers@)

Many FreeBSD end users won't even be reading hackers@ (some not
even announce@,) & would notice just yet another pointless change
sometime after upgrade.

Do not force all FreeBSD users towards gratuitous change for personal
preference for Linux behaviour.  Switch to Linux, Or hack a
personalised shell on BSD that does what you want when you type
"cat ." If it's later widely popular, use it as proof to re-propose.  No Rush.

A bigger issue is due notice procedure, & respect to FreeBSD stability of code
& users expectations of predictability.
Unwarned playing about would detract from FreeBSD's business image.

Cheers
--
Julian Stacey, Consultant Systems Engineer, BSD Linux http://berklix.com/jhs/
http://www.berklix.org/corona/#masks  Tie 2 handkerchiefs or 1 pillow case.
Jobs & economy hit by Corona to be hit again by Crash Brexit 31st Dec. 2020
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [HEADSUP] Disallowing read() of a directory fd

Alan Somers-2
On Thu, May 14, 2020 at 2:17 PM Julian H. Stacey <[hidden email]> wrote:

> Kyle Evans wrote:
> > Hi,
> >
> > This is a heads up, given that I'm completely flipping our historical
> > behavior- I intend to commit this review in a couple days' time
> > without substantial objection: https://reviews.freebsd.org/D24596
> >
> > With this, FreeBSD 13 will not allow read() of a directory fd, which
> > could have previously returned some data from the underlying
> > filesystem in no particular standardized format.
> >
> > This is a still-standards-compliant switch from one
> > implementation-defined behavior to another that's already been adopted
> > in various other popular kernels, to include OpenBSD, MacOS, and
> > Linux.
> >
> > Worth noting is that there's not really one largely-compelling reasons
> > to switch this after so many years (unless you find yourself that
> > irate when you accidentally `cat` a directory), but there are some
> > benefits which are briefly discussed in the commentary around the
> > review along with the history of the current behavior.
> >
> > This change also simplifies filesystem implementations to some extent.
> >
> > Thanks,
> >
> > Kyle Evans
>
> There is ZERO need for a spurious change at 2 days notice after 42+ years !
>
> "cat ." as been supported since Unix V6 1978 or earlier,
> no problem, even occasionaly useful.
>

Really?  When is that occasionally useful?  I've never seen anything useful
come out of reading a directory.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [HEADSUP] Disallowing read() of a directory fd

Brooks Davis-2
In reply to this post by Kyle Evans-3
On Thu, May 14, 2020 at 01:26:46PM -0500, Kyle Evans wrote:

> Hi,
>
> This is a heads up, given that I'm completely flipping our historical
> behavior- I intend to commit this review in a couple days' time
> without substantial objection: https://reviews.freebsd.org/D24596
>
> With this, FreeBSD 13 will not allow read() of a directory fd, which
> could have previously returned some data from the underlying
> filesystem in no particular standardized format.
>
> This is a still-standards-compliant switch from one
> implementation-defined behavior to another that's already been adopted
> in various other popular kernels, to include OpenBSD, MacOS, and
> Linux.
>
> Worth noting is that there's not really one largely-compelling reasons
> to switch this after so many years (unless you find yourself that
> irate when you accidentally `cat` a directory), but there are some
> benefits which are briefly discussed in the commentary around the
> review along with the history of the current behavior.
>
> This change also simplifies filesystem implementations to some extent.
I'm sure this made sense in v7, but I've never opened a directory with vi
or cat and thought "that's what I wanted to do".

In theory can use it as a terrible ls I've you've totally foobar'd your
system, but we have /rescue these days.

-- Brooks

signature.asc (465 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [HEADSUP] Disallowing read() of a directory fd

Poul-Henning Kamp
In reply to this post by Alan Somers-2
--------
In message <CAOtMX2i2Z-KX=3rYR2nZ1g1Lb_tF==[hidden email]>
, Alan Somers writes:

>Really?  When is that occasionally useful?  I've never seen anything useful
>come out of reading a directory.

Two things I have done over the years:

Figure out which filenames prevent a enormous but sparse directory
from being compacted.

Figure out which control characters were in a filename.


It might be a good idea to add a override flag on this feature,
along the lines of:

        kern.geom.debugflags=0x10


--
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[hidden email]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [HEADSUP] Disallowing read() of a directory fd

Kyle Evans-3
In reply to this post by Julian H. Stacey-3
On Thu, May 14, 2020 at 3:17 PM Julian H. Stacey <[hidden email]> wrote:

>
> Kyle Evans wrote:
> > Hi,
> >
> > This is a heads up, given that I'm completely flipping our historical
> > behavior- I intend to commit this review in a couple days' time
> > without substantial objection: https://reviews.freebsd.org/D24596
> >
> > With this, FreeBSD 13 will not allow read() of a directory fd, which
> > could have previously returned some data from the underlying
> > filesystem in no particular standardized format.
> >
> > This is a still-standards-compliant switch from one
> > implementation-defined behavior to another that's already been adopted
> > in various other popular kernels, to include OpenBSD, MacOS, and
> > Linux.
> >
> > Worth noting is that there's not really one largely-compelling reasons
> > to switch this after so many years (unless you find yourself that
> > irate when you accidentally `cat` a directory), but there are some
> > benefits which are briefly discussed in the commentary around the
> > review along with the history of the current behavior.
> >
> > This change also simplifies filesystem implementations to some extent.
> >
> > Thanks,
> >
> > Kyle Evans
>
> There is ZERO need for a spurious change at 2 days notice after 42+ years !
>

Sorry, this was a sloppy wording/prediction. There's no way I would
have gotten to committing it before Tuesday, even, with how
overcommitted I am between FreeBSD and the physical world around me.

> "cat ." as been supported since Unix V6 1978 or earlier,
> no problem, even occasionaly useful.
>

Do you have examples? The entire point of this thread is to solicit
valid complaints from 'the other side.'

> Most FreeBSD users wont have heard of https://reviews.freebsd.org/D24596
> (& there's only 5 other people's opinions there, apart from proposer,
> & skimming through the impression is far from un-qualified approval.
>
> kevans@ issued arch@ just a couple days notice of intention to commit.
> arch@ is also not widely read, ( I jhs@ added CC hackers@)
>
> Many FreeBSD end users won't even be reading hackers@ (some not
> even announce@,) & would notice just yet another pointless change
> sometime after upgrade.
>

Sure- it's hard to know just how many lists to cross-post to. Thanks
for adding -hackers@.

> Do not force all FreeBSD users towards gratuitous change for personal
> preference for Linux behaviour.  Switch to Linux, Or hack a
> personalised shell on BSD that does what you want when you type
> "cat ." If it's later widely popular, use it as proof to re-propose.  No Rush.
>

Suggestions like this are wholly unwelcome. If I wanted Linux
semantics, I would instead hack on Linux and ditch FreeBSD entirely.
I can weave together a Linux + BSD userland if that's really what I
wanted, but alas- it's not. I want sane and consistent semantics here,
given that most of our filesystems either already return EISDIR or
just return some garbage that maybe resembles something useful to
someone (e.g. ZFS, where it's not even clear that it was intended to
allow vop_read of a dir).

To a degree I also want to no longer fix application bugs because they
assume here the implementation-defined semantics that almost the rest
of the world has already opted for (again, at least OpenBSD, MacOS,
Linux). Such bugs have wasted a lot of my already sparse time because
they're not always trivial to identify, and the reward for keeping
this around is almost nonexistent for most users of FreeBSD, a very
large chunk of which run filesystems where `cat .` doesn't produce any
meaningful data.

Thanks,

Kyle Evans
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [HEADSUP] Disallowing read() of a directory fd

Don Lewis-5
In reply to this post by Cy Schubert-4
On 14 May, Cy Schubert wrote:

> In message <[hidden email]
> om>
> , Kyle Evans writes:
>> Hi,
>>
>> This is a heads up, given that I'm completely flipping our historical
>> behavior- I intend to commit this review in a couple days' time
>> without substantial objection: https://reviews.freebsd.org/D24596
>>
>> With this, FreeBSD 13 will not allow read() of a directory fd, which
>> could have previously returned some data from the underlying
>> filesystem in no particular standardized format.
>>
>> This is a still-standards-compliant switch from one
>> implementation-defined behavior to another that's already been adopted
>> in various other popular kernels, to include OpenBSD, MacOS, and
>> Linux.
>>
>> Worth noting is that there's not really one largely-compelling reasons
>> to switch this after so many years (unless you find yourself that
>> irate when you accidentally `cat` a directory), but there are some
>> benefits which are briefly discussed in the commentary around the
>> review along with the history of the current behavior.
>>
>> This change also simplifies filesystem implementations to some extent.
>
> OpenBSD has done this for a while and more importantly Linux.

Which causes annoying noise to stderr if you 'grep something *' if there
are directories in the working directory.


_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [HEADSUP] Disallowing read() of a directory fd

Kyle Evans-3
On Thu, May 14, 2020 at 6:53 PM Don Lewis <[hidden email]> wrote:

>
> On 14 May, Cy Schubert wrote:
> > In message <[hidden email]
> > om>
> > , Kyle Evans writes:
> >> Hi,
> >>
> >> This is a heads up, given that I'm completely flipping our historical
> >> behavior- I intend to commit this review in a couple days' time
> >> without substantial objection: https://reviews.freebsd.org/D24596
> >>
> >> With this, FreeBSD 13 will not allow read() of a directory fd, which
> >> could have previously returned some data from the underlying
> >> filesystem in no particular standardized format.
> >>
> >> This is a still-standards-compliant switch from one
> >> implementation-defined behavior to another that's already been adopted
> >> in various other popular kernels, to include OpenBSD, MacOS, and
> >> Linux.
> >>
> >> Worth noting is that there's not really one largely-compelling reasons
> >> to switch this after so many years (unless you find yourself that
> >> irate when you accidentally `cat` a directory), but there are some
> >> benefits which are briefly discussed in the commentary around the
> >> review along with the history of the current behavior.
> >>
> >> This change also simplifies filesystem implementations to some extent.
> >
> > OpenBSD has done this for a while and more importantly Linux.
>
> Which causes annoying noise to stderr if you 'grep something *' if there
> are directories in the working directory.
>

This is one that I actually find particularly annoying when I'm on a
UFS system, as my partial inquiries will sometimes match the names of
directory entries, so I'll get:

Binary file ${dirname} matches

That's almost never what I wanted, though.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [HEADSUP] Disallowing read() of a directory fd

Arne Steinkamm-4
In reply to this post by Julian H. Stacey-3
On Thu, May 14, 2020 at 10:17:00PM +0200, Julian H. Stacey wrote:

>
> "cat ." as been supported since Unix V6 1978 or earlier,
> no problem, even occasionaly useful.
>
> Most FreeBSD users wont have heard of https://reviews.freebsd.org/D24596
> (& there's only 5 other people's opinions there, apart from proposer,
> & skimming through the impression is far from un-qualified approval.
>
> kevans@ issued arch@ just a couple days notice of intention to commit.
> arch@ is also not widely read, ( I jhs@ added CC hackers@)

+1 !!!

.//. Arne

--
Arne Steinkamm         | Home:     Mail: arne<at>steinkamm<dot>com
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [HEADSUP] Disallowing read() of a directory fd

Arne Steinkamm-4
In reply to this post by Alan Somers-2
On Thu, May 14, 2020 at 02:20:45PM -0600, Alan Somers wrote:

> On Thu, May 14, 2020 at 2:17 PM Julian H. Stacey <[hidden email]> wrote:
>
> > Kyle Evans wrote:
> > > Hi,
> > >
> > > This is a heads up, given that I'm completely flipping our historical
> > > behavior- I intend to commit this review in a couple days' time
> > > without substantial objection: https://reviews.freebsd.org/D24596
> > >
> > > With this, FreeBSD 13 will not allow read() of a directory fd, which
> > > could have previously returned some data from the underlying
> > > filesystem in no particular standardized format.
> > >
> > > This is a still-standards-compliant switch from one
> > > implementation-defined behavior to another that's already been adopted
> > > in various other popular kernels, to include OpenBSD, MacOS, and
> > > Linux.
> > >
> > > Worth noting is that there's not really one largely-compelling reasons
> > > to switch this after so many years (unless you find yourself that
> > > irate when you accidentally `cat` a directory), but there are some
> > > benefits which are briefly discussed in the commentary around the
> > > review along with the history of the current behavior.
> > >
> > > This change also simplifies filesystem implementations to some extent.
> > >
> > > Thanks,
> > >
> > > Kyle Evans
> >
> > There is ZERO need for a spurious change at 2 days notice after 42+ years !
> >
> > "cat ." as been supported since Unix V6 1978 or earlier,
> > no problem, even occasionaly useful.
> >
>
> Really?  When is that occasionally useful?  I've never seen anything useful
> come out of reading a directory.

It's all about files in Unix...

This is true since 1972.

And files can be read!

How many (bad programmed) shell scripts will break when directory files can't
be read anymore? I have no idea.

But I know for sure that there is no need to make this change.
Not 1976, not 2020 nor in the future.

.//. Arne

--
Arne Steinkamm         | Home:     Mail: arne<at>steinkamm<dot>com
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [HEADSUP] Disallowing read() of a directory fd

Warner Losh
On Thu, May 14, 2020, 7:14 PM Arne Steinkamm <[hidden email]>
wrote:

> On Thu, May 14, 2020 at 02:20:45PM -0600, Alan Somers wrote:
> > On Thu, May 14, 2020 at 2:17 PM Julian H. Stacey <[hidden email]>
> wrote:
> >
> > > Kyle Evans wrote:
> > > > Hi,
> > > >
> > > > This is a heads up, given that I'm completely flipping our historical
> > > > behavior- I intend to commit this review in a couple days' time
> > > > without substantial objection: https://reviews.freebsd.org/D24596
> > > >
> > > > With this, FreeBSD 13 will not allow read() of a directory fd, which
> > > > could have previously returned some data from the underlying
> > > > filesystem in no particular standardized format.
> > > >
> > > > This is a still-standards-compliant switch from one
> > > > implementation-defined behavior to another that's already been
> adopted
> > > > in various other popular kernels, to include OpenBSD, MacOS, and
> > > > Linux.
> > > >
> > > > Worth noting is that there's not really one largely-compelling
> reasons
> > > > to switch this after so many years (unless you find yourself that
> > > > irate when you accidentally `cat` a directory), but there are some
> > > > benefits which are briefly discussed in the commentary around the
> > > > review along with the history of the current behavior.
> > > >
> > > > This change also simplifies filesystem implementations to some
> extent.
> > > >
> > > > Thanks,
> > > >
> > > > Kyle Evans
> > >
> > > There is ZERO need for a spurious change at 2 days notice after 42+
> years !
> > >
> > > "cat ." as been supported since Unix V6 1978 or earlier,
> > > no problem, even occasionaly useful.
> > >
> >
> > Really?  When is that occasionally useful?  I've never seen anything
> useful
> > come out of reading a directory.
>
> It's all about files in Unix...
>
> This is true since 1972.
>
> And files can be read!
>
> How many (bad programmed) shell scripts will break when directory files
> can't
> be read anymore? I have no idea.
>
> But I know for sure that there is no need to make this change.
> Not 1976, not 2020 nor in the future


Why do you need this to work? It can disclose unwanted data...

Warner

.//. Arne
>
> --
> Arne Steinkamm         | Home:     Mail: arne<at>steinkamm<dot>com
> _______________________________________________
> [hidden email] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-arch
> To unsubscribe, send any mail to "[hidden email]"
>
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [HEADSUP] Disallowing read() of a directory fd

Grzegorz Junka-2
In reply to this post by Julian H. Stacey-3
On 14/05/2020 20:17, Julian H. Stacey wrote:

> Kyle Evans wrote:
>> Hi,
>>
>> This is a heads up, given that I'm completely flipping our historical
>> behavior- I intend to commit this review in a couple days' time
>> without substantial objection: https://reviews.freebsd.org/D24596
>>
>> With this, FreeBSD 13 will not allow read() of a directory fd, which
>> could have previously returned some data from the underlying
>> filesystem in no particular standardized format.
>>
>> This is a still-standards-compliant switch from one
>> implementation-defined behavior to another that's already been adopted
>> in various other popular kernels, to include OpenBSD, MacOS, and
>> Linux.
>>
>> Worth noting is that there's not really one largely-compelling reasons
>> to switch this after so many years (unless you find yourself that
>> irate when you accidentally `cat` a directory), but there are some
>> benefits which are briefly discussed in the commentary around the
>> review along with the history of the current behavior.
>>
>> This change also simplifies filesystem implementations to some extent.
>>
>> Thanks,
>>
>> Kyle Evans
> There is ZERO need for a spurious change at 2 days notice after 42+ years !
>
> "cat ." as been supported since Unix V6 1978 or earlier,
> no problem, even occasionaly useful.


I see it as an attempt at unifying the behavior across filesystems. 42+
years ago we didn't have so many of them. And I don't see the age
argument as valid when fixing inconsistent behavior, rather red herring
that it hasn't been addressed sooner!


> Most FreeBSD users wont have heard of https://reviews.freebsd.org/D24596
> (& there's only 5 other people's opinions there, apart from proposer,
> & skimming through the impression is far from un-qualified approval.
>
> kevans@ issued arch@ just a couple days notice of intention to commit.
> arch@ is also not widely read, ( I jhs@ added CC hackers@)
>
> Many FreeBSD end users won't even be reading hackers@ (some not
> even announce@,) & would notice just yet another pointless change
> sometime after upgrade.


Are you opposing for the sake of being different, or because you were
not involved in the decision process, or are you just trying to postpone
the process a little bit for it to have more scrutiny, or are you
genuinely against the change and will actively seek for ways to abandon it?


> Do not force all FreeBSD users towards gratuitous change for personal
> preference for Linux behaviour.  Switch to Linux, Or hack a
> personalised shell on BSD that does what you want when you type
> "cat ." If it's later widely popular, use it as proof to re-propose.  No Rush.


The committer explained in the commit request that it's to avoid bugs
and make the behavior more consistent. The fact that you don't believe
him and think that he has a hidden agenda (i.e. personal preference
towards Linux) should not be enough reason to abandon it unless proven.

As a FreeBSD user I am all about less bugs in the software I am using
and frankly I don't care about "cat ."

GrzegorzJ


_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [HEADSUP] Disallowing read() of a directory fd

Poul-Henning Kamp
In reply to this post by Poul-Henning Kamp
--------
In message <CACNAnaFDHMkConkBLY-2BMAudueDA8-HTJ5_FNpt4WrB=[hidden email]>
, Kyle Evans writes:
>On Thu, May 14, 2020 at 3:30 PM Poul-Henning Kamp <[hidden email]> wrote:

>Can we explore the possibility of using fsdb(8) to fulfill these needs
>in a way that you'd be comfortable with?

First, I am perfectly comfortable with fsdb(8), but in both the two
scenarios it was much less timeconsuming to do:

        strings < /some/directory | head

Which immediately gives you the first filenames in the directory,
without waiting for ls(1) to read the entire directory, which in
this case was well north of 100MB.

In the other case

        hexdump -C < /some/directory | grep part_of_suspect_filename

gave me the answer faster than I could have started fsdb, it gave
me the answer in convenient hexadecimal, and besides it was not
a UFS filesystem.

Second, one of the major reasons, probably about 3/4 of the total
reason I ended up in FreeBSD, was because of some utterly shit
experiences with commercial operating systems, where I had been in
a tight corner at 00-dark o'clock, and run straight into something
some idiot of at the vendor thought root should not be able to do
"for their own safety".

This change falls right into that category:  If root wants to hexdump
a directory, an entire bloody disk, or even if root wants to go
and do binary surgery on a mounted file system with a hex-editor,
root should be able to do that.

She may have to `sysctl kern.warranty_voided=999` first, to disable
*under normal circumstances* reasonable and sensible protections.

I'm perfectly fine with that:  We do not want to make being root a
minefield, and I myself put the ability to munge mounted filesystems
under such a interlock in GEOM.

But we should not make it *impossible* to do these things, and in
particular, we should not make them require a reboot, because ten
times out of nine, when you find yourself doing this kind of shit,
it is usually because you're pretty sure everything is lost if you
reboot.

Summary:  I'm perfectly fine with read(2) returning error on a
directory *under normal circumstances*, and I think it makes good
sense by protecting a lot of terminals from a lot of binary
garbage.

But there is absolutely no reason to make it *impossible* for
a competent root to do what competent roots do.


--
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[hidden email]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [HEADSUP] Disallowing read() of a directory fd

Stefan Esser-3
In reply to this post by Arne Steinkamm-4
Am 15.05.20 um 03:12 schrieb Arne Steinkamm:

> On Thu, May 14, 2020 at 02:20:45PM -0600, Alan Somers wrote:
>> On Thu, May 14, 2020 at 2:17 PM Julian H. Stacey <[hidden email]> wrote:
>>
>>> Kyle Evans wrote:
>>>> Hi,
>>>>
>>>> This is a heads up, given that I'm completely flipping our historical
>>>> behavior- I intend to commit this review in a couple days' time
>>>> without substantial objection: https://reviews.freebsd.org/D24596
>>>>
>>>> With this, FreeBSD 13 will not allow read() of a directory fd, which
>>>> could have previously returned some data from the underlying
>>>> filesystem in no particular standardized format.
>>>>
>>>> This is a still-standards-compliant switch from one
>>>> implementation-defined behavior to another that's already been adopted
>>>> in various other popular kernels, to include OpenBSD, MacOS, and
>>>> Linux.
>>>>
>>>> Worth noting is that there's not really one largely-compelling reasons
>>>> to switch this after so many years (unless you find yourself that
>>>> irate when you accidentally `cat` a directory), but there are some
>>>> benefits which are briefly discussed in the commentary around the
>>>> review along with the history of the current behavior.
>>>>
>>>> This change also simplifies filesystem implementations to some extent.
>>>>
>>>> Thanks,
>>>>
>>>> Kyle Evans
>>>
>>> There is ZERO need for a spurious change at 2 days notice after 42+ years !
>>>
>>> "cat ." as been supported since Unix V6 1978 or earlier,
>>> no problem, even occasionaly useful.
>>>
>>
>> Really?  When is that occasionally useful?  I've never seen anything useful
>> come out of reading a directory.
>
> It's all about files in Unix...
>
> This is true since 1972.
>
> And files can be read!
>
> How many (bad programmed) shell scripts will break when directory files can't
> be read anymore? I have no idea.

And how many shell scripts will break if you migrate from UFS to ZFS
which does not return the same data?

Reading a directory entry as a byte stream for low level debugging has
been the only valid use case I've seen in this thread.

And I'm convinced that any developer going to work on that part of UFS
would consider adding debug code to provide or display that information
(or remove the error return just for local testing) a minor annoyance.

> But I know for sure that there is no need to make this change.
> Not 1976, not 2020 nor in the future.

There might be no need in the strict sense, but some reasons have been
provided:

- Linux, macOS, OpenBSD do it (not a good reason in itself)

- applications developed with Linux or macOS in mind might expect it
  (and that is a valid reason, IMHO)

- currently we make no guarantee with regard to success or failure of
  reading a directory - it depends on the choice made by the file system
  (and technical limitations, e.g. in case of NFS)

- information could be leaked that is of use to an attacker (and that
  might have been the main reason it has been prohibited in OpenBSD)

Every script run by an unprivileged user ought to be able to deal with a
directory that cannot be read e.g. because of insufficient privilege.

A script run by root should never depend on unspecified behavior (even
if in case that it has been defined behavior in BSD for a long time).

I'm convinced that there is more code today that has been developed on
Linux and breaks on FreeBSD, unless specifically patched, then there are
shell scripts that break when directory access is limited to the access
functions provided for this purpose for decades.

I've always been a strong supporter of POLA, but do not see how this
suggested change is going to be a violation of that principle.


We could make this change in -CURRENT, not MFC at all or after a long
period of testing whether there is any fall-out, and then decide whether
we want it backed out or controller by a tunable or sysctl.

Such an experiment in -CURRENT (announced on the appropriate lists)
should not cause any trouble or churn for developers and let us find
out, if there really is some negative impact.

Regards, STefan
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [HEADSUP] Disallowing read() of a directory fd

Daniel Ebdrup Jensen
On Fri, May 15, 2020 at 11:47:44AM +0200, Stefan E├čer wrote:

>And I'm convinced that any developer going to work on that part of UFS
>would consider adding debug code to provide or display that information
>(or remove the error return just for local testing) a minor annoyance.
>[SNIP]
>
>Regards, STefan
>_______________________________________________
>[hidden email] mailing list
>https://lists.freebsd.org/mailman/listinfo/freebsd-arch
>To unsubscribe, send any mail to "[hidden email]"
They could also add tracepoints for dtrace, and commit them to the tree so
everyone can use them. :)

Afterall, there's only +82k tracepoints (on FreeBSD 12.1-RELEASE, probably more
in -CURRENT), so there's clearly room for improvement, especially when you look
at the distribution with:
dtrace -l | awk '{ print $2 }' | uniq -c | sort -n

Yours,
Daniel Ebdrup Jensen

signature.asc (631 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [HEADSUP] Disallowing read() of a directory fd

Rodney W. Grimes-6
In reply to this post by Stefan Esser-3
> Am 15.05.20 um 03:12 schrieb Arne Steinkamm:
> > On Thu, May 14, 2020 at 02:20:45PM -0600, Alan Somers wrote:
> >> On Thu, May 14, 2020 at 2:17 PM Julian H. Stacey <[hidden email]> wrote:
> >>
> >>> Kyle Evans wrote:
> >>>> Hi,
> >>>>
> >>>> This is a heads up, given that I'm completely flipping our historical
> >>>> behavior- I intend to commit this review in a couple days' time
> >>>> without substantial objection: https://reviews.freebsd.org/D24596
> >>>>
> >>>> With this, FreeBSD 13 will not allow read() of a directory fd, which
> >>>> could have previously returned some data from the underlying
> >>>> filesystem in no particular standardized format.
> >>>>
> >>>> This is a still-standards-compliant switch from one
> >>>> implementation-defined behavior to another that's already been adopted
> >>>> in various other popular kernels, to include OpenBSD, MacOS, and
> >>>> Linux.
> >>>>
> >>>> Worth noting is that there's not really one largely-compelling reasons
> >>>> to switch this after so many years (unless you find yourself that
> >>>> irate when you accidentally `cat` a directory), but there are some
> >>>> benefits which are briefly discussed in the commentary around the
> >>>> review along with the history of the current behavior.
> >>>>
> >>>> This change also simplifies filesystem implementations to some extent.
> >>>>
> >>>> Thanks,
> >>>>
> >>>> Kyle Evans
> >>>
> >>> There is ZERO need for a spurious change at 2 days notice after 42+ years !
> >>>
> >>> "cat ." as been supported since Unix V6 1978 or earlier,
> >>> no problem, even occasionaly useful.
> >>>
> >>
> >> Really?  When is that occasionally useful?  I've never seen anything useful
> >> come out of reading a directory.
> >
> > It's all about files in Unix...
> >
> > This is true since 1972.
> >
> > And files can be read!
> >
> > How many (bad programmed) shell scripts will break when directory files can't
> > be read anymore? I have no idea.
>
> And how many shell scripts will break if you migrate from UFS to ZFS
> which does not return the same data?

I believe there is some confusion over what the symantics of read
means on a directory and what is being changed, and file protections.
This change does not change the actual protection on a directory,
so the effect on if [ -r ${pathname} ]; is none.

>
> Reading a directory entry as a byte stream for low level debugging has
> been the only valid use case I've seen in this thread.
>
> And I'm convinced that any developer going to work on that part of UFS
> would consider adding debug code to provide or display that information
> (or remove the error return just for local testing) a minor annoyance.

The use case is usually an emergency and these options are generally
not very viable solutions.

>
> > But I know for sure that there is no need to make this change.
> > Not 1976, not 2020 nor in the future.
>
> There might be no need in the strict sense, but some reasons have been
> provided:
>
> - Linux, macOS, OpenBSD do it (not a good reason in itself)
>
> - applications developed with Linux or macOS in mind might expect it
>   (and that is a valid reason, IMHO)

Thats actually a bad reason, as this only allows non portable code
to remain non portable without consequence.

>
> - currently we make no guarantee with regard to success or failure of
>   reading a directory - it depends on the choice made by the file system
>   (and technical limitations, e.g. in case of NFS)
>
> - information could be leaked that is of use to an attacker (and that
>   might have been the main reason it has been prohibited in OpenBSD)
>
> Every script run by an unprivileged user ought to be able to deal with a
> directory that cannot be read e.g. because of insufficient privilege.
>
> A script run by root should never depend on unspecified behavior (even
> if in case that it has been defined behavior in BSD for a long time).
>
> I'm convinced that there is more code today that has been developed on
> Linux and breaks on FreeBSD, unless specifically patched, then there are
> shell scripts that break when directory access is limited to the access
> functions provided for this purpose for decades.
>
> I've always been a strong supporter of POLA, but do not see how this
> suggested change is going to be a violation of that principle.
>
>
> We could make this change in -CURRENT, not MFC at all or after a long
> period of testing whether there is any fall-out, and then decide whether
> we want it backed out or controller by a tunable or sysctl.
>
> Such an experiment in -CURRENT (announced on the appropriate lists)
> should not cause any trouble or churn for developers and let us find
> out, if there really is some negative impact.

I can support this change so long as there is a *RUN* time way
to disable it for root which does not require a reboot of any
kind.

Much like Kirk, and Poul who have pointed out a use cases when doing
a UFS data recovery this is a usefull feature.  Often the directory
tree is valid, but the file names are mangled.

--
Rod Grimes                                                 [hidden email]
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [HEADSUP] Disallowing read() of a directory fd

Kyle Evans-3
In reply to this post by Poul-Henning Kamp
On Fri, May 15, 2020 at 2:51 AM Poul-Henning Kamp <[hidden email]> wrote:

>
> --------
> In message <CACNAnaFDHMkConkBLY-2BMAudueDA8-HTJ5_FNpt4WrB=[hidden email]>
> , Kyle Evans writes:
> >On Thu, May 14, 2020 at 3:30 PM Poul-Henning Kamp <[hidden email]> wrote:
>
> >Can we explore the possibility of using fsdb(8) to fulfill these needs
> >in a way that you'd be comfortable with?
>>
> Summary:  I'm perfectly fine with read(2) returning error on a
> directory *under normal circumstances*, and I think it makes good
> sense by protecting a lot of terminals from a lot of binary
> garbage.
>
> But there is absolutely no reason to make it *impossible* for
> a competent root to do what competent roots do.
>

First, apologies if my previous message had offended you -- I didn't
mean for this, but as you can tell I was not well-equipped to discuss
the possibilities with a seasoned veteran such as yourself.

I've prepared a patch locally to update the review that both hides it
off behind security.bsd.allow_read_dir (default off) and restricts it
to a new PRIV_VFS_READ_DIR that *is not* granted to jailed root. I
know we've already discussed this to some extent, but can you confirm
that these restrictions are reasonable and acceptable for you? I've
tentatively placed it in the security.bsd.* namespace because it can
and has had security implications, but I'm certainly not dead-set on
it staying there.

Thanks,

Kyle Evans
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [HEADSUP] Disallowing read() of a directory fd

Diane Bruce
In reply to this post by Rodney W. Grimes-6
On Fri, May 15, 2020 at 05:47:43AM -0700, Rodney W. Grimes wrote:
> > Am 15.05.20 um 03:12 schrieb Arne Steinkamm:
...
> > >>>> Worth noting is that there's not really one largely-compelling reasons
> > >>>> to switch this after so many years (unless you find yourself that
> > >>>> irate when you accidentally `cat` a directory), but there are some
> > >>>> benefits which are briefly discussed in the commentary around the
> > >>>> review along with the history of the current behavior.


All I have to say on this noisy bikeshed is, let's resurrect the mkdir
bug of V7 because it's tradition and the BSD way and history and stuff.
(I only expect a few of you to remember this one.)

Diane
--
- [hidden email] [hidden email] http://www.db.net/~db
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
123