Extra memory mapping seen on freebsd-12 which was not seen in freebsd-11

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Extra memory mapping seen on freebsd-12 which was not seen in freebsd-11

karnajit wangkhem
Hi freebsd team,



While checking on a valgrind issue, I encountered the following mapping of
a simple test program on a freebsd-12.



Below is the procstat -v output I took by breaking on _rtld.



[Stable 12]

# procstat -v 76573

  PID              START                END PRT  RES PRES REF SHD FLAG  TP
PATH

<SNIP>

76573        0x800227000        0x800229000 rw-    1    1   1   0 ----- df

76573     0x7fffdffff000     0x7ffffffdf000 ---    0    0   0   0 -----
--                  <<< This ~511MB memory segment

76573     0x7ffffffdf000     0x7ffffffff000 rw-    1    1   1   0 ---D- df

76573     0x7ffffffff000     0x800000000000 r-x    1    1  99   0 ----- ph



[Stable 11]

# procstat -v 85507

  PID              START                END PRT  RES PRES REF SHD FLAG TP
PATH

<SNIP>

85507        0x800820000        0x800822000 rw-    1    1   1   0 ---- df

85507     0x7ffffffdf000     0x7ffffffff000 rw-    1    1   1   0 ---D df

85507     0x7ffffffff000     0x800000000000 r-x    1    1 104   0 ---- ph



There is an extra ~511MB reserved mmap area starting at 0x7fffdffff000 in
stable 12. Could you please give me an insight of what this is for and is
it ok for a userspace program to modify this mapping?



The reason is that our applications reserve some fixed memory that
crosses/modify the above region. As this mapping was not called by the
client program, valgrind had taken control of it. So, I have to decide
whether to give the control to the client and allow modifications(mprotect,
unmap, mmap, etc) on this memory segment or logically(not mandatorily) a
client program should be allowed to modify this area?


Regards,

Karan
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Extra memory mapping seen on freebsd-12 which was not seen in freebsd-11

Paul Floyd-2

> On 15 Oct 2020, at 16:54, karnajit wangkhem <[hidden email]> wrote:
>
> Hi freebsd team,
>
> While checking on a valgrind issue, I encountered the following mapping of
> a simple test program on a freebsd-12.

;-)


> [Stable 11]
>
> # procstat -v 85507
>
>  PID              START                END PRT  RES PRES REF SHD FLAG TP
> PATH
>
> <SNIP>
>
> 85507        0x800820000        0x800822000 rw-    1    1   1   0 ---- df
> 85507     0x7ffffffdf000     0x7ffffffff000 rw-    1    1   1   0 ---D df
> 85507     0x7ffffffff000     0x800000000000 r-x    1    1 104   0 ---- ph
>
> There is an extra ~511MB reserved mmap area starting at 0x7fffdffff000 in
> stable 12. Could you please give me an insight of what this is for and is
> it ok for a userspace program to modify this mapping?
>
> The reason is that our applications reserve some fixed memory that
> crosses/modify the above region. As this mapping was not called by the
> client program, valgrind had taken control of it. So, I have to decide
> whether to give the control to the client and allow modifications(mprotect,
> unmap, mmap, etc) on this memory segment or logically(not mandatorily) a
> client program should be allowed to modify this area?


This extra memory is the MAP_GUARD, which was introduced in FreeBSD 10.4
and changed to be a large zone in FreeBSD 11.1.

If I understand correctly, it’s a kind of super-sized guard page for the stack.
There are more details in the mmap man page.

If you run Valgrind with the -d option it will print a table of the memory mapping
(Prefixed with ‘aspacem’ for Address Space Manager). If you want to see some
more Valgrind details, see aspacemgr-linux.c from line 1646 (despite the name
it's used by all supported platforms).

A+
Paul

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Extra memory mapping seen on freebsd-12 which was not seen in freebsd-11

karnajit wangkhem
Thanks for the reply. It helped in my understanding.

Below is a sample code

#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <sys/mman.h>

int main()
{
  char *str = NULL;
  str = (char *)mmap((void *)0x7fffdfffe000UL, 0x2000, PROT_READ |
PROT_WRITE, MAP_FIXED | MAP_ANON, -1, 0);
  if ((void *)str == (void *)MAP_FAILED) {
    int err = errno;
    printf("mmap failed. err (%s)\n", strerror(err));
  } else {
    memcpy(str, "Hello World", 12);
    printf("str = %s\n", str);
  }

  return 0;
}

Now, the below code under valgrind will give
- mmap failed. err (Invalid argument)

But, if we give control of this segment to the client program
with VG_(am_change_ownership_v_to_c), then valgrind allows the client to do
the following mmap.
- str = Hello World

And, the resultant procstat result looks like this:
 2382        0x7fbfff000        0x7fc001000 rwx    2    2   1   0 ----- df
 2382     0x7fffdfffe000     0x7fffe0000000 rw-    0    0   0   0 ----- --
   <<< Client mmap call
 2382     0x7fffe0000000     0x7ffffffdf000 ---    0    0   0   0 ----- --
     <<< 0x1000 bytes is taken away from the MAP_GUARD area
 2382     0x7ffffffdf000     0x7ffffffff000 rw-    1    1   1   0 ---D- df
 2382     0x7ffffffff000     0x800000000000 r-x    1    1 104   0 ----- ph

So, is it right for the application with or without valgrind to cross the
above boundary, If that memory which the application reserved is just for
normal application specific use?

Regards,
Karan

On Thu, Oct 15, 2020 at 11:56 PM Paul Floyd <[hidden email]> wrote:

>
> > On 15 Oct 2020, at 16:54, karnajit wangkhem <[hidden email]> wrote:
> >
> > Hi freebsd team,
> >
> > While checking on a valgrind issue, I encountered the following mapping
> of
> > a simple test program on a freebsd-12.
>
> ;-)
>
>
> > [Stable 11]
> >
> > # procstat -v 85507
> >
> >  PID              START                END PRT  RES PRES REF SHD FLAG TP
> > PATH
> >
> > <SNIP>
> >
> > 85507        0x800820000        0x800822000 rw-    1    1   1   0 ---- df
> > 85507     0x7ffffffdf000     0x7ffffffff000 rw-    1    1   1   0 ---D df
> > 85507     0x7ffffffff000     0x800000000000 r-x    1    1 104   0 ---- ph
> >
> > There is an extra ~511MB reserved mmap area starting at 0x7fffdffff000 in
> > stable 12. Could you please give me an insight of what this is for and is
> > it ok for a userspace program to modify this mapping?
> >
> > The reason is that our applications reserve some fixed memory that
> > crosses/modify the above region. As this mapping was not called by the
> > client program, valgrind had taken control of it. So, I have to decide
> > whether to give the control to the client and allow
> modifications(mprotect,
> > unmap, mmap, etc) on this memory segment or logically(not mandatorily) a
> > client program should be allowed to modify this area?
>
>
> This extra memory is the MAP_GUARD, which was introduced in FreeBSD 10.4
> and changed to be a large zone in FreeBSD 11.1.
>
> If I understand correctly, it’s a kind of super-sized guard page for the
> stack.
> There are more details in the mmap man page.
>
> If you run Valgrind with the -d option it will print a table of the memory
> mapping
> (Prefixed with ‘aspacem’ for Address Space Manager). If you want to see
> some
> more Valgrind details, see aspacemgr-linux.c from line 1646 (despite the
> name
> it's used by all supported platforms).
>
> A+
> Paul
>
> _______________________________________________
> [hidden email] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "[hidden email]"
>
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Extra memory mapping seen on freebsd-12 which was not seen in freebsd-11

Konstantin Belousov
On Fri, Oct 16, 2020 at 10:43:32AM +0530, karnajit wangkhem wrote:

> Thanks for the reply. It helped in my understanding.
>
> Below is a sample code
>
> #include <stdio.h>
> #include <string.h>
> #include <errno.h>
> #include <sys/mman.h>
>
> int main()
> {
>   char *str = NULL;
>   str = (char *)mmap((void *)0x7fffdfffe000UL, 0x2000, PROT_READ |
> PROT_WRITE, MAP_FIXED | MAP_ANON, -1, 0);
>   if ((void *)str == (void *)MAP_FAILED) {
>     int err = errno;
>     printf("mmap failed. err (%s)\n", strerror(err));
>   } else {
>     memcpy(str, "Hello World", 12);
>     printf("str = %s\n", str);
>   }
>
>   return 0;
> }
>
> Now, the below code under valgrind will give
> - mmap failed. err (Invalid argument)
>
> But, if we give control of this segment to the client program
> with VG_(am_change_ownership_v_to_c), then valgrind allows the client to do
> the following mmap.
> - str = Hello World
>
> And, the resultant procstat result looks like this:
>  2382        0x7fbfff000        0x7fc001000 rwx    2    2   1   0 ----- df
>  2382     0x7fffdfffe000     0x7fffe0000000 rw-    0    0   0   0 ----- --
>    <<< Client mmap call
>  2382     0x7fffe0000000     0x7ffffffdf000 ---    0    0   0   0 ----- --
>      <<< 0x1000 bytes is taken away from the MAP_GUARD area
>  2382     0x7ffffffdf000     0x7ffffffff000 rw-    1    1   1   0 ---D- df
>  2382     0x7ffffffff000     0x800000000000 r-x    1    1 104   0 ----- ph
>
> So, is it right for the application with or without valgrind to cross the
> above boundary, If that memory which the application reserved is just for
> normal application specific use?

You called mmap(2) with MAP_FIXED flag, which means that mmap must destroy
any mapping existing at the specified address, and create the requested
mapping instead.  This should work as far as the requested range fits into
the userspace virtual address space, and mapping object provides requested
permissions.

If valgrind does not emulate that behavior of MAP_FIXED correctly, it is
valgrind bug.

That said, application trying to mmap something at the guard holding the
stack grow area is most likely buggy.  Old libthr intentionally split main
thread' stack into stacks of the new threads, but this was changed.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Extra memory mapping seen on freebsd-12 which was not seen in freebsd-11

Paul Floyd-2
In reply to this post by karnajit wangkhem


> On 16 Oct 2020, at 07:13, karnajit wangkhem <[hidden email]> wrote:
>
> Thanks for the reply. It helped in my understanding.
>
> Below is a sample code
>
> #include <stdio.h>
> #include <string.h>
> #include <errno.h>
> #include <sys/mman.h>
>
> int main()
> {
>  char *str = NULL;
>  str = (char *)mmap((void *)0x7fffdfffe000UL, 0x2000, PROT_READ |
> PROT_WRITE, MAP_FIXED | MAP_ANON, -1, 0);
>  if ((void *)str == (void *)MAP_FAILED) {
>    int err = errno;
>    printf("mmap failed. err (%s)\n", strerror(err));
>  } else {
>    memcpy(str, "Hello World", 12);
>    printf("str = %s\n", str);
>  }
>
>  return 0;
> }
>
> Now, the below code under valgrind will give
> - mmap failed. err (Invalid argument)
>
> But, if we give control of this segment to the client program
> with VG_(am_change_ownership_v_to_c), then valgrind allows the client to do
> the following mmap.
> - str = Hello World
>
> And, the resultant procstat result looks like this:
> 2382        0x7fbfff000        0x7fc001000 rwx    2    2   1   0 ----- df
> 2382     0x7fffdfffe000     0x7fffe0000000 rw-    0    0   0   0 ----- --
>   <<< Client mmap call
> 2382     0x7fffe0000000     0x7ffffffdf000 ---    0    0   0   0 ----- --
>     <<< 0x1000 bytes is taken away from the MAP_GUARD area
> 2382     0x7ffffffdf000     0x7ffffffff000 rw-    1    1   1   0 ---D- df
> 2382     0x7ffffffff000     0x800000000000 r-x    1    1 104   0 ----- ph
>
> So, is it right for the application with or without valgrind to cross the
> above boundary, If that memory which the application reserved is just for
> normal application specific use?

Hi

Obviously threre are some restrictions for the guest application running under Valgrind.
Valgrind needs its own stack and heap, so the guest can’t mmap these regions.

Why do you need to mmap into this region?

A+
Paul

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Extra memory mapping seen on freebsd-12 which was not seen in freebsd-11

karnajit wangkhem
In reply to this post by Konstantin Belousov
Thanks for the info. This was exactly my worry.

On one side valgrind should have allowed this mapping and at the same time
the application would be doing wrong. I will keep this in mind.

On Fri, Oct 16, 2020 at 3:41 PM Konstantin Belousov <[hidden email]>
wrote:

> On Fri, Oct 16, 2020 at 10:43:32AM +0530, karnajit wangkhem wrote:
> > Thanks for the reply. It helped in my understanding.
> >
> > Below is a sample code
> >
> > #include <stdio.h>
> > #include <string.h>
> > #include <errno.h>
> > #include <sys/mman.h>
> >
> > int main()
> > {
> >   char *str = NULL;
> >   str = (char *)mmap((void *)0x7fffdfffe000UL, 0x2000, PROT_READ |
> > PROT_WRITE, MAP_FIXED | MAP_ANON, -1, 0);
> >   if ((void *)str == (void *)MAP_FAILED) {
> >     int err = errno;
> >     printf("mmap failed. err (%s)\n", strerror(err));
> >   } else {
> >     memcpy(str, "Hello World", 12);
> >     printf("str = %s\n", str);
> >   }
> >
> >   return 0;
> > }
> >
> > Now, the below code under valgrind will give
> > - mmap failed. err (Invalid argument)
> >
> > But, if we give control of this segment to the client program
> > with VG_(am_change_ownership_v_to_c), then valgrind allows the client to
> do
> > the following mmap.
> > - str = Hello World
> >
> > And, the resultant procstat result looks like this:
> >  2382        0x7fbfff000        0x7fc001000 rwx    2    2   1   0 -----
> df
> >  2382     0x7fffdfffe000     0x7fffe0000000 rw-    0    0   0   0 -----
> --
> >    <<< Client mmap call
> >  2382     0x7fffe0000000     0x7ffffffdf000 ---    0    0   0   0 -----
> --
> >      <<< 0x1000 bytes is taken away from the MAP_GUARD area
> >  2382     0x7ffffffdf000     0x7ffffffff000 rw-    1    1   1   0 ---D-
> df
> >  2382     0x7ffffffff000     0x800000000000 r-x    1    1 104   0 -----
> ph
> >
> > So, is it right for the application with or without valgrind to cross the
> > above boundary, If that memory which the application reserved is just for
> > normal application specific use?
>
> You called mmap(2) with MAP_FIXED flag, which means that mmap must destroy
> any mapping existing at the specified address, and create the requested
> mapping instead.  This should work as far as the requested range fits into
> the userspace virtual address space, and mapping object provides requested
> permissions.
>
> If valgrind does not emulate that behavior of MAP_FIXED correctly, it is
> valgrind bug.
>
> That said, application trying to mmap something at the guard holding the
> stack grow area is most likely buggy.  Old libthr intentionally split main
> thread' stack into stacks of the new threads, but this was changed.
>
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Extra memory mapping seen on freebsd-12 which was not seen in freebsd-11

karnajit wangkhem
In reply to this post by Paul Floyd-2
Hi Paul,

The mappings of these applications existed prior to the guard change, which
was fine as no mapping existed on the memory range. With migration to
stable 12, I was doubting that these mappings are no longer correct. But at
the same time, does valgrind have to own this segment, which only came post
certain freebsd releases?

Regards,
Karan

On Sat, Oct 17, 2020 at 12:39 AM Paul Floyd <[hidden email]> wrote:

>
>
> > On 16 Oct 2020, at 07:13, karnajit wangkhem <[hidden email]> wrote:
> >
> > Thanks for the reply. It helped in my understanding.
> >
> > Below is a sample code
> >
> > #include <stdio.h>
> > #include <string.h>
> > #include <errno.h>
> > #include <sys/mman.h>
> >
> > int main()
> > {
> >  char *str = NULL;
> >  str = (char *)mmap((void *)0x7fffdfffe000UL, 0x2000, PROT_READ |
> > PROT_WRITE, MAP_FIXED | MAP_ANON, -1, 0);
> >  if ((void *)str == (void *)MAP_FAILED) {
> >    int err = errno;
> >    printf("mmap failed. err (%s)\n", strerror(err));
> >  } else {
> >    memcpy(str, "Hello World", 12);
> >    printf("str = %s\n", str);
> >  }
> >
> >  return 0;
> > }
> >
> > Now, the below code under valgrind will give
> > - mmap failed. err (Invalid argument)
> >
> > But, if we give control of this segment to the client program
> > with VG_(am_change_ownership_v_to_c), then valgrind allows the client to
> do
> > the following mmap.
> > - str = Hello World
> >
> > And, the resultant procstat result looks like this:
> > 2382        0x7fbfff000        0x7fc001000 rwx    2    2   1   0 ----- df
> > 2382     0x7fffdfffe000     0x7fffe0000000 rw-    0    0   0   0 ----- --
> >   <<< Client mmap call
> > 2382     0x7fffe0000000     0x7ffffffdf000 ---    0    0   0   0 ----- --
> >     <<< 0x1000 bytes is taken away from the MAP_GUARD area
> > 2382     0x7ffffffdf000     0x7ffffffff000 rw-    1    1   1   0 ---D- df
> > 2382     0x7ffffffff000     0x800000000000 r-x    1    1 104   0 ----- ph
> >
> > So, is it right for the application with or without valgrind to cross the
> > above boundary, If that memory which the application reserved is just for
> > normal application specific use?
>
> Hi
>
> Obviously threre are some restrictions for the guest application running
> under Valgrind.
> Valgrind needs its own stack and heap, so the guest can’t mmap these
> regions.
>
> Why do you need to mmap into this region?
>
> A+
> Paul
>
> _______________________________________________
> [hidden email] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "[hidden email]"
>
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"