setting distinct core file names

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

setting distinct core file names

Willem Jan Withagen-2
Hi,

Looking at core(5) and sysctl it looks like these are system wide
settings....

Is there a possibility that a program can set its own corefile name (and
path?)

During parallel testing I'm running into these scripts that generate
cores, but they end up all in the same location. But it would be nice if
I could one way or another determine which file came from what script.

But for that I would need to be able to set something like
        %N."script".core
as the core name. I could then put that in then ENV of the script and
the program would pick it up and set its own corefile name.

Possible??
--WjW
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: setting distinct core file names

Konstantin Belousov
On Tue, Nov 27, 2018 at 06:27:59PM +0100, Willem Jan Withagen wrote:

> Hi,
>
> Looking at core(5) and sysctl it looks like these are system wide
> settings....
>
> Is there a possibility that a program can set its own corefile name (and
> path?)
>
> During parallel testing I'm running into these scripts that generate
> cores, but they end up all in the same location. But it would be nice if
> I could one way or another determine which file came from what script.
>
> But for that I would need to be able to set something like
> %N."script".core
> as the core name. I could then put that in then ENV of the script and
> the program would pick it up and set its own corefile name.
>
> Possible??
No.

Do not expect any proposal that requires kernel to read user mode
environment variable to work.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: setting distinct core file names

Lowell Gilbert-14
In reply to this post by Willem Jan Withagen-2
Willem Jan Withagen <[hidden email]> writes:

> Looking at core(5) and sysctl it looks like these are system wide
> settings....
>
> Is there a possibility that a program can set its own corefile name
> (and path?)
>
> During parallel testing I'm running into these scripts that generate
> cores, but they end up all in the same location. But it would be nice
> if I could one way or another determine which file came from what
> script.
>
> But for that I would need to be able to set something like
> %N."script".core
> as the core name. I could then put that in then ENV of the script and
> the program would pick it up and set its own corefile name.
>
> Possible??

If you can run the scripts in arbitrary paths, you can encode any extra
information you need in a directory name. [I'd recommend just changing
the process name, but I'm guessing that the cores themselves are being
generated by something running in a subshell.]
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: setting distinct core file names

Conrad Meyer-2
In reply to this post by Willem Jan Withagen-2
One (ugly) trick is to use multiple filesystem links to the script
interpreter, where the link names distinguish the scripts.  E.g.,

$ ln /bin/sh /libexec/my_script_one_sh
$ ln /bin/sh /libexec/my_script_two_sh
$ cat myscript1.sh
#!/libexec/my_script_one_sh
...

Cores will be dumped with %N of "my_script_one_sh."

Best,
Conrad
On Tue, Nov 27, 2018 at 9:29 AM Willem Jan Withagen <[hidden email]> wrote:

>
> Hi,
>
> Looking at core(5) and sysctl it looks like these are system wide
> settings....
>
> Is there a possibility that a program can set its own corefile name (and
> path?)
>
> During parallel testing I'm running into these scripts that generate
> cores, but they end up all in the same location. But it would be nice if
> I could one way or another determine which file came from what script.
>
> But for that I would need to be able to set something like
>         %N."script".core
> as the core name. I could then put that in then ENV of the script and
> the program would pick it up and set its own corefile name.
>
> Possible??
> --WjW
> _______________________________________________
> [hidden email] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "[hidden email]"
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: setting distinct core file names

Willem Jan Withagen-2
On 27-11-2018 21:46, Conrad Meyer wrote:

> One (ugly) trick is to use multiple filesystem links to the script
> interpreter, where the link names distinguish the scripts.  E.g.,
>
> $ ln /bin/sh /libexec/my_script_one_sh
> $ ln /bin/sh /libexec/my_script_two_sh
> $ cat myscript1.sh
> #!/libexec/my_script_one_sh
> ...
>
> Cores will be dumped with %N of "my_script_one_sh."

Neat trick... got to try and remember this.
But it is not the shell scripts that are crashing...

When running Ceph tests during Jenkins building some
programs/executables intentionally crash leaving cores.
Others (scripts) use some of these programs with correct input and
should NOT crash. And test during startup and termination that there are
no cores left.

One jenkins test run takes about 4 hours when not executed in parallel.
I'm testing 4 version multiple times a day to not have this huge list of
PRs the go thru when testing fails.

But the intentional cores and the failure cores here collide.
And when I have a core program_x.core I can't tell if they are from a
failure or from an intentional crash.

Now if could tell per program  how to name its core that would allow me
to fix the problem, without overturning the complete Ceph testing
infrastructure and still keep parallel tests.

It would also help in that "regular" cores just keep going the way the
are. So other application still have the same behaviour. And are still
picked up by periodic processing.

--WjW

> Best,
> Conrad
> On Tue, Nov 27, 2018 at 9:29 AM Willem Jan Withagen <[hidden email]> wrote:
>>
>> Hi,
>>
>> Looking at core(5) and sysctl it looks like these are system wide
>> settings....
>>
>> Is there a possibility that a program can set its own corefile name (and
>> path?)
>>
>> During parallel testing I'm running into these scripts that generate
>> cores, but they end up all in the same location. But it would be nice if
>> I could one way or another determine which file came from what script.
>>
>> But for that I would need to be able to set something like
>>          %N."script".core
>> as the core name. I could then put that in then ENV of the script and
>> the program would pick it up and set its own corefile name.
>>
>> Possible??
>> --WjW
>> _______________________________________________
>> [hidden email] mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
>> To unsubscribe, send any mail to "[hidden email]"

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: setting distinct core file names

Willem Jan Withagen-2
On 28-11-2018 11:43, Willem Jan Withagen wrote:

> On 27-11-2018 21:46, Conrad Meyer wrote:
>> One (ugly) trick is to use multiple filesystem links to the script
>> interpreter, where the link names distinguish the scripts.  E.g.,
>>
>> $ ln /bin/sh /libexec/my_script_one_sh
>> $ ln /bin/sh /libexec/my_script_two_sh
>> $ cat myscript1.sh
>> #!/libexec/my_script_one_sh
>> ...
>>
>> Cores will be dumped with %N of "my_script_one_sh."
>
> Neat trick... got to try and remember this.
> But it is not the shell scripts that are crashing...
>
> When running Ceph tests during Jenkins building some
> programs/executables intentionally crash leaving cores.
> Others (scripts) use some of these programs with correct input and
> should NOT crash. And test during startup and termination that there are
> no cores left.
>
> One jenkins test run takes about 4 hours when not executed in parallel.
> I'm testing 4 version multiple times a day to not have this huge list of
> PRs the go thru when testing fails.
>
> But the intentional cores and the failure cores here collide.
> And when I have a core program_x.core I can't tell if they are from a
> failure or from an intentional crash.
>
> Now if could tell per program  how to name its core that would allow me
> to fix the problem, without overturning the complete Ceph testing
> infrastructure and still keep parallel tests.
>
> It would also help in that "regular" cores just keep going the way the
> are. So other application still have the same behaviour. And are still
> picked up by periodic processing.

So I read a bit more about the prcctl and prctl(the Linux variant) and
turns out that Linux can set PR_SET_DUMPABLE. And that is actually used
in some of the Ceph applications...

Being able to set this to 0 or 1  would perhaps be a nice start as well.

--WjW

> --WjW
>
>> Best,
>> Conrad
>> On Tue, Nov 27, 2018 at 9:29 AM Willem Jan Withagen <[hidden email]>
>> wrote:
>>>
>>> Hi,
>>>
>>> Looking at core(5) and sysctl it looks like these are system wide
>>> settings....
>>>
>>> Is there a possibility that a program can set its own corefile name (and
>>> path?)
>>>
>>> During parallel testing I'm running into these scripts that generate
>>> cores, but they end up all in the same location. But it would be nice if
>>> I could one way or another determine which file came from what script.
>>>
>>> But for that I would need to be able to set something like
>>>          %N."script".core
>>> as the core name. I could then put that in then ENV of the script and
>>> the program would pick it up and set its own corefile name.

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: setting distinct core file names

Konstantin Belousov
On Wed, Nov 28, 2018 at 12:21:33PM +0100, Willem Jan Withagen wrote:

> On 28-11-2018 11:43, Willem Jan Withagen wrote:
> > On 27-11-2018 21:46, Conrad Meyer wrote:
> >> One (ugly) trick is to use multiple filesystem links to the script
> >> interpreter, where the link names distinguish the scripts.  E.g.,
> >>
> >> $ ln /bin/sh /libexec/my_script_one_sh
> >> $ ln /bin/sh /libexec/my_script_two_sh
> >> $ cat myscript1.sh
> >> #!/libexec/my_script_one_sh
> >> ...
> >>
> >> Cores will be dumped with %N of "my_script_one_sh."
> >
> > Neat trick... got to try and remember this.
> > But it is not the shell scripts that are crashing...
> >
> > When running Ceph tests during Jenkins building some
> > programs/executables intentionally crash leaving cores.
> > Others (scripts) use some of these programs with correct input and
> > should NOT crash. And test during startup and termination that there are
> > no cores left.
> >
> > One jenkins test run takes about 4 hours when not executed in parallel.
> > I'm testing 4 version multiple times a day to not have this huge list of
> > PRs the go thru when testing fails.
> >
> > But the intentional cores and the failure cores here collide.
> > And when I have a core program_x.core I can't tell if they are from a
> > failure or from an intentional crash.
> >
> > Now if could tell per program  how to name its core that would allow me
> > to fix the problem, without overturning the complete Ceph testing
> > infrastructure and still keep parallel tests.
> >
> > It would also help in that "regular" cores just keep going the way the
> > are. So other application still have the same behaviour. And are still
> > picked up by periodic processing.
>
> So I read a bit more about the prcctl and prctl(the Linux variant) and
> turns out that Linux can set PR_SET_DUMPABLE. And that is actually used
> in some of the Ceph applications...
>
> Being able to set this to 0 or 1  would perhaps be a nice start as well.
Isn't setrlimit(RLIMIT_CORE, 0) enough ?  It is slightly different syntax,
but the idea is that you set RLIMIT_CORE to zero, then we do not even
start dumping.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: setting distinct core file names

Willem Jan Withagen-2
On 28/11/2018 15:43, Konstantin Belousov wrote:

> On Wed, Nov 28, 2018 at 12:21:33PM +0100, Willem Jan Withagen wrote:
>> On 28-11-2018 11:43, Willem Jan Withagen wrote:
>>> On 27-11-2018 21:46, Conrad Meyer wrote:
>>>> One (ugly) trick is to use multiple filesystem links to the script
>>>> interpreter, where the link names distinguish the scripts.  E.g.,
>>>>
>>>> $ ln /bin/sh /libexec/my_script_one_sh
>>>> $ ln /bin/sh /libexec/my_script_two_sh
>>>> $ cat myscript1.sh
>>>> #!/libexec/my_script_one_sh
>>>> ...
>>>>
>>>> Cores will be dumped with %N of "my_script_one_sh."
>>> Neat trick... got to try and remember this.
>>> But it is not the shell scripts that are crashing...
>>>
>>> When running Ceph tests during Jenkins building some
>>> programs/executables intentionally crash leaving cores.
>>> Others (scripts) use some of these programs with correct input and
>>> should NOT crash. And test during startup and termination that there are
>>> no cores left.
>>>
>>> One jenkins test run takes about 4 hours when not executed in parallel.
>>> I'm testing 4 version multiple times a day to not have this huge list of
>>> PRs the go thru when testing fails.
>>>
>>> But the intentional cores and the failure cores here collide.
>>> And when I have a core program_x.core I can't tell if they are from a
>>> failure or from an intentional crash.
>>>
>>> Now if could tell per program  how to name its core that would allow me
>>> to fix the problem, without overturning the complete Ceph testing
>>> infrastructure and still keep parallel tests.
>>>
>>> It would also help in that "regular" cores just keep going the way the
>>> are. So other application still have the same behaviour. And are still
>>> picked up by periodic processing.
>> So I read a bit more about the prcctl and prctl(the Linux variant) and
>> turns out that Linux can set PR_SET_DUMPABLE. And that is actually used
>> in some of the Ceph applications...
>>
>> Being able to set this to 0 or 1  would perhaps be a nice start as well.
> Isn't setrlimit(RLIMIT_CORE, 0) enough ?  It is slightly different syntax,
> but the idea is that you set RLIMIT_CORE to zero, then we do not even
> start dumping.
Right,

At one point I think I had this code in some tests code....
I also think this is the default on the CentOS when I tested it there.

So I set it from the top-shell to propagate.
But then I could have run into:
      [EPERM]            The limit specified to setrlimit() would have
raised
                         the maximum limit value, and the caller is not the
                         super-user.
When do wanting dumps.

I'm not sure, it was quite some time ago.
But that might be a nice suggestion to look into.

--WjW

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: setting distinct core file names

Matthew D. Fuller
In reply to this post by Willem Jan Withagen-2
On Wed, Nov 28, 2018 at 11:43:50AM +0100 I heard the voice of
Willem Jan Withagen, and lo! it spake thus:
>
> Neat trick... got to try and remember this.
> But it is not the shell scripts that are crashing...

You could still make per-test hardlinks to the binary and run from
them instead.


--
Matthew Fuller     (MF4839)   |  [hidden email]
Systems/Network Administrator |  http://www.over-yonder.net/~fullermd/
           On the Internet, nobody can hear you scream.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"