Scheduled ideas: Two threads with one "stone"

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Scheduled ideas: Two threads with one "stone"

Zaphod Beeblebrox-2
W.R.T. hyperthreading, it's come to my attention that FreeBSD has had some
recommendations for awhile --- namely hyperthreading is bad for secure
systems.  That-is-to-say this is not about recent vulnerabilities of recent
Intel flavoured CPUs, but rather some thoughts on a different solution to a
different problem.

I was building world on my laptop when I noticed how sucky it had become.
Now... I well know this is due to the fact that even a "nice -19 make -j32
buildworld" will slow the normal priority userland down by about 1/2
because the thread that is "me" on the CPU is competing with one of the
threads that is saturating the CPU.  My normal priority thread will get
scheduled, but it competes equally for resources with threads at much lower
priority.

It then struck me that the fix for this problem would also address the
hyperthreading side channel attack.  It struck me that two simple rules
would apply to scheduling hyperthreads (rather than the global "turn them
off for security" ...

To be scheduled on a paired hyperthread, the candidate thread should:

   1. Belong to the same user
   2. Be the same (or the same within a margin) effective priority

My argument here for (1) is that it makes the hyperthreading side channel
attack moot.  The same user has many ways to access the other process...
the hyperthreading attack would just be a waste of time.  My argument for
(2) is that it would restore the experience of responsiveness the local
machine that we have on non-hyperthreading processors.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Scheduled ideas: Two threads with one "stone"

Jan Bramkamp-2
On 24.01.19 17:32, Zaphod Beeblebrox wrote:

> W.R.T. hyperthreading, it's come to my attention that FreeBSD has had some
> recommendations for awhile --- namely hyperthreading is bad for secure
> systems.  That-is-to-say this is not about recent vulnerabilities of recent
> Intel flavoured CPUs, but rather some thoughts on a different solution to a
> different problem.
>
> I was building world on my laptop when I noticed how sucky it had become.
> Now... I well know this is due to the fact that even a "nice -19 make -j32
> buildworld" will slow the normal priority userland down by about 1/2
> because the thread that is "me" on the CPU is competing with one of the
> threads that is saturating the CPU.  My normal priority thread will get
> scheduled, but it competes equally for resources with threads at much lower
> priority.

You can use `idprio 10 ...` to minimize the impact of CPU bound tasks.
If you want to go even further you could apply hierarchical resource
limits on the make process. The idprio chainloader puts make and all its
descendants into the idle scheduling class which allows normal time
sharing threads to preempt them.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Scheduled ideas: Two threads with one "stone"

Gary Jennejohn-6
On Thu, 24 Jan 2019 18:34:21 +0100
Jan Bramkamp <[hidden email]> wrote:

> On 24.01.19 17:32, Zaphod Beeblebrox wrote:
> > W.R.T. hyperthreading, it's come to my attention that FreeBSD has had some
> > recommendations for awhile --- namely hyperthreading is bad for secure
> > systems.  That-is-to-say this is not about recent vulnerabilities of recent
> > Intel flavoured CPUs, but rather some thoughts on a different solution to a
> > different problem.
> >
> > I was building world on my laptop when I noticed how sucky it had become.
> > Now... I well know this is due to the fact that even a "nice -19 make -j32
> > buildworld" will slow the normal priority userland down by about 1/2
> > because the thread that is "me" on the CPU is competing with one of the
> > threads that is saturating the CPU.  My normal priority thread will get
> > scheduled, but it competes equally for resources with threads at much lower
> > priority.  
>
> You can use `idprio 10 ...` to minimize the impact of CPU bound tasks.
> If you want to go even further you could apply hierarchical resource
> limits on the make process. The idprio chainloader puts make and all its
> descendants into the idle scheduling class which allows normal time
> sharing threads to preempt them.
>

Or switch to SCHED_4BSD.  I ran your command line and was still
able to watch a HD movie in full screen mode without a single
glitch.  My Ryzen has 6 physical cores and 6 SMTs.

--
Gary Jennejohn
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Scheduled ideas: Two threads with one "stone"

Zaphod Beeblebrox-2
In reply to this post by Jan Bramkamp-2
On Thu, Jan 24, 2019 at 12:36 PM Jan Bramkamp <[hidden email]> wrote:

> On 24.01.19 17:32, Zaphod Beeblebrox wrote:
> > W.R.T. hyperthreading, it's come to my attention that FreeBSD has had
> some
> > recommendations for awhile --- namely hyperthreading is bad for secure
> > systems.  That-is-to-say this is not about recent vulnerabilities of
> recent
> > Intel flavoured CPUs, but rather some thoughts on a different solution
> to a
> > different problem.
> >
> > I was building world on my laptop when I noticed how sucky it had become.
> > Now... I well know this is due to the fact that even a "nice -19 make
> -j32
> > buildworld" will slow the normal priority userland down by about 1/2
> > because the thread that is "me" on the CPU is competing with one of the
> > threads that is saturating the CPU.  My normal priority thread will get
> > scheduled, but it competes equally for resources with threads at much
> lower
> > priority.
>
> You can use `idprio 10 ...` to minimize the impact of CPU bound tasks.
> If you want to go even further you could apply hierarchical resource
> limits on the make process. The idprio chainloader puts make and all its
> descendants into the idle scheduling class which allows normal time
> sharing threads to preempt them.
>

No.  This is insufficient to the task.  If your goal  is that the "idle" or
"low priority" task does not effect the high priority tasks, this is
insufficient w.r.t. hyperthreading.  In hyperthreading (to discuss just one
variant of many), two CPU cores share all of their functional bits.  This
is useful because, in general, CPU functional bits are fairly idle.  Even
if both threads on hyperthreaded pair are "runable" right now, they can be
fetching from memory or, in the exreme, one could be issuing floating point
and one could be issuing integer, IIRC.  The problem I'm proposing to solve
is when the two processes are of sufficiently differing effective
priorities (however they get to that point) that they not be scheduled on
the same hyperthreaded pair.  The reason for this is that beyond scheduling
the processes to the cores, we have no control over the resource contention
of the cores --- and by that I claim that the lower priority process can
force the higher priority process to wait as it consumes resources shared
by the two cores.

In an ideal world, the hyperthread interface on the cpu would allow some
combination of prioritizing cores and their resource use, but that is not
how Intel's chips currently work.

To be clear, and this is easy to experience, find a FreeBSD desktop with a
i7 (or similar) running X and (say) kde.  Not something lightweight.  Make
sure it has enough RAM (16 or 32 big is probably good for this
experiment).  When idle, take a "feel" on the interface.  No lag, things
are spiffy.  Now, go to /usr/src.  Make sure it's there.  Then run "nice
-19 make -j32 buildworld" there.  For extra credit, you can even use idprio
if you like.

Now... duplicate your feel test.  The interface will be laggy.  Things will
be slower.  If you look at the process table, it's not that your effective
priorities are coming anywhere close to those from -19 ... or even if you
use idprio... still laggy.  What is happening here?  Think about your
processor.  FreeBSD will ensure that your normal priority processes are
scheduled first on a few of the 8 cores that show with an i7. But chances
are very high that your processes are landing opposite low priority
processes on the hyperthreaded pairs.  This means that your effective
processsor use is being cut in half (-ish) by the hyperthreaded processes
grabbing the shared resources roughly half the time.  You can game out
having two processes ready and how often they fall on the same pair vs.
falling on non-paired.  You can even do the thought experiment of higher
priority processes being placed on cores with pairs deliberately not used
until all non-pared cores are in use --- urm... this is a big diversion...
then we have the opportunity cost of the then partially unused resources
... etc.

I should stop there.  There are simple things that make a big difference
before that digression produces many small things with even smaller effects.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Scheduled ideas: Two threads with one "stone"

Zaphod Beeblebrox-2
In reply to this post by Gary Jennejohn-6
On Thu, Jan 24, 2019 at 1:01 PM Gary Jennejohn <[hidden email]> wrote:

>
>
> Or switch to SCHED_4BSD.  I ran your command line and was still
> able to watch a HD movie in full screen mode without a single
> glitch.  My Ryzen has 6 physical cores and 6 SMTs.
>

This really isn't about "can you do X while building world" it's an
argument about how unfairly we allocate CPU these days w.r.t.
hyperthreading and related technologies.  It's not "does this minimally
work" it's about "can we do better" ....
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"