More fine-grained NUMA knobs

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

More fine-grained NUMA knobs

John Baldwin
Some of the I/O device affinity stuff such as bus_get_domain() (and the
bus_get_cpus() I have in review in D5519) are useful on their own right even
if the VM system is not doing NUMA-aware allocations.  I think it would be
useful to be able to enable these two "prongs" of NUMA awareness
independently.  To that end, I have a little strawman patch that adds two
new kernel options: VM_NUMA_ALLOC and DEVICE_NUMA.  I actually think it is
probably worth enabling DEVICE_NUMA by default on x86 (and bumping the
default MAXMEMDOM to, say, 8 (quad-socket haswell)).

You can see the simple patch at:

https://github.com/bsdjhb/freebsd/compare/master...bsdjhb:numa_opts

Thoughts?

--
John Baldwin
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: More fine-grained NUMA knobs

Adrian Chadd-4
+1 by me!


-a


On 22 March 2016 at 17:13, John Baldwin <[hidden email]> wrote:

> Some of the I/O device affinity stuff such as bus_get_domain() (and the
> bus_get_cpus() I have in review in D5519) are useful on their own right even
> if the VM system is not doing NUMA-aware allocations.  I think it would be
> useful to be able to enable these two "prongs" of NUMA awareness
> independently.  To that end, I have a little strawman patch that adds two
> new kernel options: VM_NUMA_ALLOC and DEVICE_NUMA.  I actually think it is
> probably worth enabling DEVICE_NUMA by default on x86 (and bumping the
> default MAXMEMDOM to, say, 8 (quad-socket haswell)).
>
> You can see the simple patch at:
>
> https://github.com/bsdjhb/freebsd/compare/master...bsdjhb:numa_opts
>
> Thoughts?
>
> --
> John Baldwin
> _______________________________________________
> [hidden email] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-arch
> To unsubscribe, send any mail to "[hidden email]"
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: More fine-grained NUMA knobs

Jack Vogel
Cool! Glad to see this happening :)



On Tue, Mar 22, 2016 at 8:15 PM, Adrian Chadd <[hidden email]>
wrote:

> +1 by me!
>
>
> -a
>
>
> On 22 March 2016 at 17:13, John Baldwin <[hidden email]> wrote:
> > Some of the I/O device affinity stuff such as bus_get_domain() (and the
> > bus_get_cpus() I have in review in D5519) are useful on their own right
> even
> > if the VM system is not doing NUMA-aware allocations.  I think it would
> be
> > useful to be able to enable these two "prongs" of NUMA awareness
> > independently.  To that end, I have a little strawman patch that adds two
> > new kernel options: VM_NUMA_ALLOC and DEVICE_NUMA.  I actually think it
> is
> > probably worth enabling DEVICE_NUMA by default on x86 (and bumping the
> > default MAXMEMDOM to, say, 8 (quad-socket haswell)).
> >
> > You can see the simple patch at:
> >
> > https://github.com/bsdjhb/freebsd/compare/master...bsdjhb:numa_opts
> >
> > Thoughts?
> >
> > --
> > John Baldwin
> > _______________________________________________
> > [hidden email] mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-arch
> > To unsubscribe, send any mail to "[hidden email]"
> _______________________________________________
> [hidden email] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-arch
> To unsubscribe, send any mail to "[hidden email]"
>
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: More fine-grained NUMA knobs

Konstantin Belousov
In reply to this post by John Baldwin
On Tue, Mar 22, 2016 at 05:13:05PM -0700, John Baldwin wrote:

> Some of the I/O device affinity stuff such as bus_get_domain() (and the
> bus_get_cpus() I have in review in D5519) are useful on their own right even
> if the VM system is not doing NUMA-aware allocations.  I think it would be
> useful to be able to enable these two "prongs" of NUMA awareness
> independently.  To that end, I have a little strawman patch that adds two
> new kernel options: VM_NUMA_ALLOC and DEVICE_NUMA.  I actually think it is
> probably worth enabling DEVICE_NUMA by default on x86 (and bumping the
> default MAXMEMDOM to, say, 8 (quad-socket haswell)).
>
> You can see the simple patch at:
>
> https://github.com/bsdjhb/freebsd/compare/master...bsdjhb:numa_opts
>
> Thoughts?

I do not like that you check both defined(xxx_NUMA) and MAXMEMDOM > 1.
IMO the > 1 part should be dropped.  I do not see why would it be useful
even as optimization.

Otherwise, this looks good.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"