[Bug 239293] lang/rust: `make fetch-list` fails

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

[Bug 239293] lang/rust: `make fetch-list` fails

bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239293

            Bug ID: 239293
           Summary: lang/rust: `make fetch-list` fails
           Product: Ports & Packages
           Version: Latest
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: Individual Port(s)
          Assignee: [hidden email]
          Reporter: [hidden email]
             Flags: maintainer-feedback?([hidden email])
          Assignee: [hidden email]

I can't get `fetch-list` for the port:

```
$ cd /usr/ports/lang/rust
$ make fetch-list
mkdir -p /usr/ports/distfiles/rust && cd /usr/ports/distfiles/rust && { env
/usr/bin/fetch -Fpr  -S 98707920
https://static.rust-lang.org/dist/rustc-1.36.0-src.tar.xz  || env
/usr/bin/fetch -Fpr  -S 98707920
http://distcache.FreeBSD.org/ports-distfiles/rust/rustc-1.36.0-src.tar.xz  ||
echo "rustc-1.36.0-src.tar.xz" not fetched; }
mkdir -p /usr/ports/distfiles/rust && cd /usr/ports/distfiles/rust && { mkdir:
2019-05-23: Permission denied
*** Error code 1

Stop.
make: stopped in /usr/ports/lang/rust
```

On other ports I need `make fetch-list` generate the lists without errors.  And
I never `fetch-list` for lang/rust before (therefore, I'm not sure that this
error appears near the r506903).

The ports tree is at r506903.

FreeBSD 13.0-CURRENT and 12.0-RELEASE-p4.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports-bugs
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 239293] fetch-list: broken handling of DISTFILES when it has entries with subdirectories

bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239293

Tobias Kortkamp <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
              Flags|maintainer-feedback?(rust@F |maintainer-feedback+
                   |reeBSD.org)                 |
                 CC|                            |[hidden email]
           Assignee|[hidden email]            |[hidden email]
          Component|Individual Port(s)          |Ports Framework
            Summary|lang/rust: `make            |fetch-list: broken handling
                   |fetch-list` fails           |of DISTFILES when it has
                   |                            |entries with subdirectories
             Status|New                         |Open

--- Comment #3 from Tobias Kortkamp <[hidden email]> ---
Ok, I am able to reproduce it if I do not have write permissions to
/usr/ports/lang/rust.  For some reason fetch-list or fetch-url-list
attempt to mkdir /usr/ports/lang/rust/2019-05-23 on fetch-list or
fetch-url-list, which does not make a whole lot of sense since it
is not even under /usr/ports/distfiles/.  But then again fetch-list
should probably not even attempt creating any directories at all.

This also affects other ports that have a DISTFILES with a / in
them like devel/rust-cbindgen or devel/rust-bindgen.  This seems
like bad behavior of or an unhandled edge case in do-fetch.sh (or
somewhere else?) to me.  Assigning to portmgr.

--
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports-bugs
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 239293] fetch-list: broken handling of DISTFILES when it has entries with subdirectories

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239293

--- Comment #4 from Tobias Kortkamp <[hidden email]> ---
Created attachment 205879
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=205879&action=edit
v0

Here's a patch that seems to fix it.

--
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports-bugs
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 239293] fetch-list: broken handling of DISTFILES when it has entries with subdirectories

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239293

--- Comment #5 from Ruslan Garipov <[hidden email]> ---
(In reply to Tobias Kortkamp from comment #4)
I'm sorry for delay.  Your patch works fine for me, thanks.

--
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports-bugs
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 239293] fetch-list: broken handling of DISTFILES when it has entries with subdirectories

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239293

--- Comment #6 from Ruslan Garipov <[hidden email]> ---
Created attachment 206151
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=206151&action=edit
Add `mkdir` to Tobias Kortkamp's v0 patch

(In reply to Tobias Kortkamp from comment #4)
Oh, sorry, I hastened to say ``the patch looks good''.

Unfortunately, it doesn't fix the problem completely: it prevents
mkdir(1) on fetch-list, but the result script itself ends up with
issues:

$ cd /usr/ports/lang/rush
$ make fetch-list
{elided}
mkdir -p /usr/ports/distfiles/rust && cd /usr/ports/distfiles/rust && { env
/usr/bin/fetch -Fpr -S 73812596 -o
2019-05-23/rustc-1.35.0-x86_64-unknown-freebsd.tar.gz
https://static.rust-lang.org/dist/2019-05-23/rustc-1.35.0-x86_64-unknown-freebsd.tar.gz
...{elided}
mkdir -p /usr/ports/distfiles/rust && cd /usr/ports/distfiles/rust && { env
/usr/bin/fetch -Fpr -S 83588135 -o
2019-05-23/rust-std-1.35.0-x86_64-unknown-freebsd.tar.gz
https://static.rust-lang.org/dist/2019-05-23/rust-std-1.35.0-x86_64-unknown-freebsd.tar.gz
... {elided}
{elided}

As one can see, the script generated by fetch-list creates
/usr/ports/distfiles/rust directory, but fetch(1) writes its output to
/usr/ports/distfiles/rust/2019-05-23 directory, which doesn't exist.
Therefore, fetch(1) fails with errors like the following ones:

fetch: 2019-05-23/rustc-1.35.0-x86_64-unknown-freebsd.tar.gz: open(): No such
file or directory
fetch: 2019-05-23/rust-std-1.35.0-x86_64-unknown-freebsd.tar.gz: open(): No
such file or directory

Tobias, I've modified your patch slightly, and tested on lang/rust,
shells/bash, www/firefox and x11/xorg (meta)port.  For the former the
new patch gives the following script:

$ cd /usr/ports/lang/rust
$ make fetch-list
{the first row is elided}
mkdir -p "/usr/ports/distfiles/rust/2019-05-23" && cd /usr/ports/distfiles/rust
&& { env /usr/bin/fetch -Fpr -S 73812596 -o
2019-05-23/rustc-1.35.0-x86_64-unknown-freebsd.tar.gz
https://static.rust-lang.org... {elided}
{elided}

which works fine on a machine having the Internet access.

Are there any mandatory tests I should run against the updated
do-fetch.sh script?

--
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports-bugs
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 239293] fetch-list: broken handling of DISTFILES when it has entries with subdirectories

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239293

--- Comment #7 from Mathieu Arnold <[hidden email]> ---
Sorry for being silent, I was looking into it. (but slowly because you don't
want to rush things).
You found out what was missing from Tobias's patch, but I don't like yours
either, it could be made in a much simpler way.

--
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports-bugs
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 239293] fetch-list: broken handling of DISTFILES when it has entries with subdirectories

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239293

--- Comment #8 from Mathieu Arnold <[hidden email]> ---
(As a side note, your patch does not work, file is not defined when it runs.)

--
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports-bugs
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 239293] fetch-list: broken handling of DISTFILES when it has entries with subdirectories

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239293

--- Comment #9 from Mathieu Arnold <[hidden email]> ---
https://reviews.freebsd.org/D21112

--
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports-bugs
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 239293] fetch-list: broken handling of DISTFILES when it has entries with subdirectories

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239293

--- Comment #10 from Ruslan Garipov <[hidden email]> ---
(In reply to Mathieu Arnold from comment #9)
Thanks for your answers and the revision.

> it could be made in a much simpler way
I just wanted to concentrate all the echos in one place.

> your patch does not work
For fetch/fetch-recursive may be?..  Yes, it's my fault -- I didn't test those
actions; I just checked fetch-list/fetch-recursive-list.  Sorry.

--
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports-bugs
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 239293] fetch-list: broken handling of DISTFILES when it has entries with subdirectories

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239293

--- Comment #11 from Mathieu Arnold <[hidden email]> ---
(In reply to Ruslan Garipov from comment #10)
> (In reply to Mathieu Arnold from comment #9)
> Thanks for your answers and the revision.
>
> > it could be made in a much simpler way
> I just wanted to concentrate all the echos in one place.

It was a noble thought, but I (re)wrote this code a few years ago, if it could
be made simpler, it would :-)

> > your patch does not work
> For fetch/fetch-recursive may be?..  Yes, it's my fault -- I didn't test
> those actions; I just checked fetch-list/fetch-recursive-list.  Sorry.

Well, fetch-list, yes.  You were using ${file} before it was defined, so it did
not do what you wanted, at all.

--
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports-bugs
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 239293] fetch-list: broken handling of DISTFILES when it has entries with subdirectories

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239293

--- Comment #12 from Ruslan Garipov <[hidden email]> ---
(In reply to Mathieu Arnold from comment #11)
Sorry for off topic, but I just want to understand where I've failed
and I don't see why I was

> ... using ${file} before it was defined
?  I believe the `file` is defined on the 30th line[1].
Unconditionally, just as the first statement of the first `for` loop.
And my patch modified the 110th line in the file.  The variable is used
above and beyond the 110th line -- but why it's undefined there?  I
also see no `unset` in the file...  Am I missed something?

> so it did not do what you wanted, at all.
I'm sorry, but it did -- as I already said in the comment #6 I
successfully fetch-list for lang/rust, shells/bash, www/firefox and
x11/xorg ports.  Was it a lucky break?

[1]
https://svnweb.freebsd.org/ports/head/Mk/Scripts/do-fetch.sh?revision=462544&view=markup#l30

--
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports-bugs
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 239293] fetch-list: broken handling of DISTFILES when it has entries with subdirectories

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239293

Mathieu Arnold <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|Open                        |Closed

--
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports-bugs
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 239293] fetch-list: broken handling of DISTFILES when it has entries with subdirectories

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239293

--- Comment #13 from [hidden email] ---
A commit references this bug:

Author: mat
Date: Wed Jul 31 10:10:36 UTC 2019
New revision: 507705
URL: https://svnweb.freebsd.org/changeset/ports/507705

Log:
  Fix fetch-list when running as a user.

  The fetch-list target is used to generate a shell script that will more
  or less replicates what do-fetch does. It allows one to do most things
  as a regular user, and generate that script to run, say, on another
  machine, if the one where you build things does not have access to the
  internet, or has much slower access.

  It was failing when DISTDIR was not writable by the current user, and
  the port had a distribution file with a path in it. (Not using
  DIST_SUBDIR, something else, like lang/rust does.) It was failing
  because it was trying to create that subdirectory unconditionally,
  instead of only creating the subdirectory if actually had to.  This also
  fixes the bug that the generated script did not have the appropriate
  mkdirs for those directories.

  PR:           239293
  Submitted by: tobik (earlier version)
  Reported by:  Ruslan Garipov
  Differential Revision:        https://reviews.freebsd.org/D21112

Changes:
  head/Mk/Scripts/do-fetch.sh

--
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports-bugs
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 239293] fetch-list: broken handling of DISTFILES when it has entries with subdirectories

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239293

--- Comment #14 from Ruslan Garipov <[hidden email]> ---
(In reply to Mathieu Arnold from comment #9)
(In reply to commit-hook from comment #13)
Please add -n to your `echo`:

Index: do-fetch.sh
===================================================================
--- do-fetch.sh (revision 508165)
+++ do-fetch.sh (working copy)
@@ -128,7 +128,7 @@
                        */*)
                                case ${dp_TARGET} in
                                fetch-list|fetch-url-list-int)
-                                       echo "mkdir -p \"${file%/*}\" && "
+                                       echo -n "mkdir -p \"${file%/*}\" && "
                                        ;;
                                *)
                                        mkdir -p "${file%/*}"

like in all `echo`s in the file.

I'm usually fetch-list several ports into one file, and then remove
duplicated lines.  Without -n in the `echo` above, I'm unable to do
that.  Thanks.

--
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports-bugs
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 239293] fetch-list: broken handling of DISTFILES when it has entries with subdirectories

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239293

Kubilay Kocak <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           See Also|                            |https://bugs.freebsd.org/bu
                   |                            |gzilla/show_bug.cgi?id=2397
                   |                            |61

--
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports-bugs
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 239293] fetch-list: broken handling of DISTFILES when it has entries with subdirectories

bugzilla-noreply
In reply to this post by bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239293

Ruslan Garipov <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           See Also|                            |https://bugs.freebsd.org/bu
                   |                            |gzilla/show_bug.cgi?id=2411
                   |                            |48

--
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports-bugs
To unsubscribe, send any mail to "[hidden email]"