jmalloc in shared memory

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

jmalloc in shared memory

Slawa Olhovchenkov
How I can use jmalloc in shared memory?

I.e. parent process do mmap w/ MAP_ANON|MAP_SHARED, create jmalloc
"instance" in this memory and use jmalloc routines for memory management
in this region.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: jmalloc in shared memory

Alexander Tarasikov
Hi Slawa,
I'm not sure jemalloc is supposed to be used this way, but you should
be able to achieve this functionality in two ways:
1. Editing jemalloc's allocator ("src/pages.c") to allocate in your
region instead of calling mmap()
2. Hook mmap() and brk(), sbrk() with LD_PRELOAD and provide
implementations that will allocate in the shared memory

Hope this helps.

On Mon, Mar 20, 2017 at 2:12 PM, Slawa Olhovchenkov <[hidden email]> wrote:
> How I can use jmalloc in shared memory?
>
> I.e. parent process do mmap w/ MAP_ANON|MAP_SHARED, create jmalloc
> "instance" in this memory and use jmalloc routines for memory management
> in this region.
> _______________________________________________
> [hidden email] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "[hidden email]"



--
Regards, Alexander
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: jmalloc in shared memory

Slawa Olhovchenkov
On Mon, Mar 27, 2017 at 03:27:31PM +0200, Alexander Tarasikov wrote:

> Hi Slawa,
> I'm not sure jemalloc is supposed to be used this way, but you should
> be able to achieve this functionality in two ways:
> 1. Editing jemalloc's allocator ("src/pages.c") to allocate in your
> region instead of calling mmap()
> 2. Hook mmap() and brk(), sbrk() with LD_PRELOAD and provide
> implementations that will allocate in the shared memory
>
> Hope this helps.

I am don't need to redirect ALL allocations in the shared memory.
I am need only do it for selected structures.
For example, I am need create red-black tree and update it (in shared
memory). For this, I am need create own memory management in this
region or use existing memory management tool (for allocate,
dealloacate and tracks chunks) worked for dedicated segment only.

> On Mon, Mar 20, 2017 at 2:12 PM, Slawa Olhovchenkov <[hidden email]> wrote:
> > How I can use jmalloc in shared memory?
> >
> > I.e. parent process do mmap w/ MAP_ANON|MAP_SHARED, create jmalloc
> > "instance" in this memory and use jmalloc routines for memory management
> > in this region.
> > _______________________________________________
> > [hidden email] mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> > To unsubscribe, send any mail to "[hidden email]"
>
>
>
> --
> Regards, Alexander
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: jmalloc in shared memory

Brooks Davis-2
On Mon, Mar 27, 2017 at 04:53:33PM +0300, Slawa Olhovchenkov wrote:

> On Mon, Mar 27, 2017 at 03:27:31PM +0200, Alexander Tarasikov wrote:
>
> > Hi Slawa,
> > I'm not sure jemalloc is supposed to be used this way, but you should
> > be able to achieve this functionality in two ways:
> > 1. Editing jemalloc's allocator ("src/pages.c") to allocate in your
> > region instead of calling mmap()
> > 2. Hook mmap() and brk(), sbrk() with LD_PRELOAD and provide
> > implementations that will allocate in the shared memory
> >
> > Hope this helps.
>
> I am don't need to redirect ALL allocations in the shared memory.
> I am need only do it for selected structures.
> For example, I am need create red-black tree and update it (in shared
> memory). For this, I am need create own memory management in this
> region or use existing memory management tool (for allocate,
> dealloacate and tracks chunks) worked for dedicated segment only.
JEMalloc is almost certainly overkill for this application.  It sounds
likely you're allocating fixed sized objects (or at least not too many
of them).  For that, a simple slab allocator should do the trick.
Alterntively, rtld contains a simple pooled malloc  that's not hard
to adapt to random backing stores (libexec/rtld-elf/malloc.c).   I've
got version adapted for CHERI with pluggable storage backends at
https://github.com/CTSRD-CHERI/cheribsd/tree/master/lib/libmalloc_simple.
It wouldn't be too hard turn it back into portable C.

-- Brooks

signature.asc (465 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: jmalloc in shared memory

Slawa Olhovchenkov
On Mon, Mar 27, 2017 at 04:26:37PM +0000, Brooks Davis wrote:

> On Mon, Mar 27, 2017 at 04:53:33PM +0300, Slawa Olhovchenkov wrote:
> > On Mon, Mar 27, 2017 at 03:27:31PM +0200, Alexander Tarasikov wrote:
> >
> > > Hi Slawa,
> > > I'm not sure jemalloc is supposed to be used this way, but you should
> > > be able to achieve this functionality in two ways:
> > > 1. Editing jemalloc's allocator ("src/pages.c") to allocate in your
> > > region instead of calling mmap()
> > > 2. Hook mmap() and brk(), sbrk() with LD_PRELOAD and provide
> > > implementations that will allocate in the shared memory
> > >
> > > Hope this helps.
> >
> > I am don't need to redirect ALL allocations in the shared memory.
> > I am need only do it for selected structures.
> > For example, I am need create red-black tree and update it (in shared
> > memory). For this, I am need create own memory management in this
> > region or use existing memory management tool (for allocate,
> > dealloacate and tracks chunks) worked for dedicated segment only.
>
> JEMalloc is almost certainly overkill for this application.  It sounds
> likely you're allocating fixed sized objects (or at least not too many
> of them).  For that, a simple slab allocator should do the trick.

I am planed to allocate (and free) also strings.
JEMalloc already present in libc.
I am mean JEMalloc allow to create arena/some_other_stuff and point
this stuff in call.

> Alterntively, rtld contains a simple pooled malloc  that's not hard
> to adapt to random backing stores (libexec/rtld-elf/malloc.c).   I've
> got version adapted for CHERI with pluggable storage backends at
> https://github.com/CTSRD-CHERI/cheribsd/tree/master/lib/libmalloc_simple.
> It wouldn't be too hard turn it back into portable C.

Thanks, I am discover this.

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"