[TEST/REVIEW] CPU accounting patches

classic Classic list List threaded Threaded
41 messages Options
123
Reply | Threaded
Open this post in threaded view
|

[TEST/REVIEW] CPU accounting patches

Poul-Henning Kamp

Here is a new version of my cpu accounting change patch.

        http://phk.freebsd.dk/patch/cpu_acct_1.patch

This patch is supposedly harmless (or at least mostly harmless)
and I'd appreciate it getting a solid trashing.


This patchs changes cpu accounting from accumulating charges
in real-time units and instead accumulates in units of some
per-arch, possibly per-cpu counter.

When the accumulated charge is read by times(2) or getrusage(2) or
similar, the frequency of the counter is interrogated and the charge
normalized to microseconds.

With this patch, the counter is always the timecounter and the only
real difference is therefore a minor performance change (because we
save the normalizing multiplications for each context switch).

On my AMD Athlon 700 and my Sun Ultra 60 the performance difference
is barely 1% and of doubtful statistical quality.

On my Opteron machine I get a 2.7+/-.6% boost on unixbench's
context1 test.

Of course, the scheme used in this patch suffers a bit if the
hardware counter changes to other hardware of a different rate
or simply changes rate.  This has been discussed at length in
a previous thread already, and I'll simply refer to it, rather
than rehash here:

http://lists.freebsd.org/pipermail/freebsd-net/2005-October/008637.html



The other half of this work is in this separate patch, and this is
not yet complete.  You are welcome to test it however, as long as
you are aware of the problems it may hold:

        http://phk.freebsd.dk/patch/cpu_acct_2.patch

It makes i386 and amd64 use the TSC and sparc64 use the "tick"
counter for CPU accounting.

On a sparc64 it gives 3.2+/-.3% speedup on unixbench/context1

On a Athlon700 with i8254 timecounter it gives a 95+/-.8% speedup

On a Opteron with ACPI-fast timecounter it gives a 36+/-.6% speedup.

The downside is, that unless your cpu clock is correctly probed
at boot and stays constant, your cpu accounting numbers will have
a bogus scaling factor.

I belive all the sparc64s we support have constant CPU rates,
so they should be safe.

For i386 and amd64 things are more tricky.  Laptops doing power
saving tricks will probably give bogus cpu accounting values,
but as such the patch should do no other harm than screw up
those values.

If you benchmark this patch, please understand that it is vitally
important that you benchmark relative to the real-time scale (ie:
wall-clock time), the "user" and "system" fields from time(1) are
not usable.

--
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[hidden email]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [TEST/REVIEW] CPU accounting patches

John Baldwin
On Tuesday 24 January 2006 16:10, Poul-Henning Kamp wrote:
> Here is a new version of my cpu accounting change patch.
>
> http://phk.freebsd.dk/patch/cpu_acct_1.patch
>
> This patch is supposedly harmless (or at least mostly harmless)
> and I'd appreciate it getting a solid trashing.

The XXX in calcru1() you can remove.  The rux you are adding the current time
to is a local rusage_ext on the stack.  However, your changes probably make
it bogus in that the current code assumes it can subtract the start time of
another CPU (for a thread running on another CPU) from the current time on
this CPU to get the amount of time the other thread has been running on the
other CPU since it last updated p->p_rux.rux_runtime.  However, with the CPUs
having disparate timings this would break as curthread's CPU's notion of now
will be unrelated to the other thread's CPU's notion of now.

Other than that this patch looks fine to me.  FYI, Alpha also has a per-cpu
counter (RPCC) that is used for the timecounter on UP Alphas.

--
John Baldwin <[hidden email]>  <><  http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve"  =  http://www.FreeBSD.org
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [TEST/REVIEW] CPU accounting patches

Poul-Henning Kamp
In message <[hidden email]>, John Baldwin writes:
>On Tuesday 24 January 2006 16:10, Poul-Henning Kamp wrote:
>> Here is a new version of my cpu accounting change patch.
>>

>The XXX in calcru1() you can remove.  The rux you are adding the current time
>to is a local rusage_ext on the stack.

I found that out myself and forgot to remove the XXX :-)

>However, your changes probably make
>it bogus in that the current code assumes it can subtract the start time of
>another CPU (for a thread running on another CPU) from the current time on
>this CPU to get the amount of time the other thread has been running on the
>other CPU since it last updated p->p_rux.rux_runtime.

This is when we call calcru on a running process ?

Yeah, that's a problem.

I'd tend to say we should just forget about accounting for the
current quantum in that case.

This is a valid handling IMO because the result can never be used
in any final or definitive kind of way anyway.  When the process
finishes or deschedules the numbers will get updated correctly.

Doing this may also simplify the locking of calcru ?

>Other than that this patch looks fine to me.  FYI, Alpha also has a per-cpu
>counter (RPCC) that is used for the timecounter on UP Alphas.

My Alpha is hosed right now and doesn't want to boot 6.0-R.  I havn't had
time to boot my 5.0-R on it and do the upgrade to -current the long way.

--
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[hidden email]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [TEST/REVIEW] CPU accounting patches

John Baldwin
On Tuesday 24 January 2006 17:16, Poul-Henning Kamp wrote:

> In message <[hidden email]>, John Baldwin writes:
> >On Tuesday 24 January 2006 16:10, Poul-Henning Kamp wrote:
> >> Here is a new version of my cpu accounting change patch.
> >
> >The XXX in calcru1() you can remove.  The rux you are adding the current
> > time to is a local rusage_ext on the stack.
>
> I found that out myself and forgot to remove the XXX :-)
>
> >However, your changes probably make
> >it bogus in that the current code assumes it can subtract the start time
> > of another CPU (for a thread running on another CPU) from the current
> > time on this CPU to get the amount of time the other thread has been
> > running on the other CPU since it last updated p->p_rux.rux_runtime.
>
> This is when we call calcru on a running process ?
>
> Yeah, that's a problem.

Yeah, SIGINFO is an example, and getrusage() of another process might run into
this, too.

> I'd tend to say we should just forget about accounting for the
> current quantum in that case.

It was added to fix some of the "time going backwards" and "negative uptime"
stuff I think.  Ask Bruce.

> This is a valid handling IMO because the result can never be used
> in any final or definitive kind of way anyway.  When the process
> finishes or deschedules the numbers will get updated correctly.

Unfortunately calcru() updates the millisecond counts (uu, su, iu) for running
processes and I think having a bogus runtime gets that very confused.

> Doing this may also simplify the locking of calcru ?

Nah, it already runs lockless in some cases I think.

> >Other than that this patch looks fine to me.  FYI, Alpha also has a
> > per-cpu counter (RPCC) that is used for the timecounter on UP Alphas.
>
> My Alpha is hosed right now and doesn't want to boot 6.0-R.  I havn't had
> time to boot my 5.0-R on it and do the upgrade to -current the long way.

Heh, I have one running head atm.  However, it's second CPU is having issues
(I think the CPU fan has died) so it's less useful than in the past. :(

--
John Baldwin <[hidden email]>  <><  http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve"  =  http://www.FreeBSD.org
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [TEST/REVIEW] CPU accounting patches

Alexander Leidinger
In reply to this post by Poul-Henning Kamp
Poul-Henning Kamp <[hidden email]> wrote:

[first patch]
> Of course, the scheme used in this patch suffers a bit if the
> hardware counter changes to other hardware of a different rate
> or simply changes rate.


[second patch]
> The downside is, that unless your cpu clock is correctly probed
> at boot and stays constant, your cpu accounting numbers will have
> a bogus scaling factor.

> For i386 and amd64 things are more tricky.  Laptops doing power
> saving tricks will probably give bogus cpu accounting values,
> but as such the patch should do no other harm than screw up
> those values.

Are you going to fix those issues for machines which do power saving tricks
(which may even be useful on servers for some people, not only on laptops),
or do you not intend to further work on this besides the patches you present
here?

Bye,
Alexander.

--
http://www.Leidinger.net  Alexander @ Leidinger.net: PGP ID = B0063FE7
http://www.FreeBSD.org     netchild @ FreeBSD.org  : PGP ID = 72077137
One man tells a falsehood, a hundred repeat it as true.


_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [TEST/REVIEW] CPU accounting patches

Poul-Henning Kamp
In message <[hidden email]>, Alexander Lei
dinger writes:

>Are you going to fix those issues for machines which do power saving tricks
>(which may even be useful on servers for some people, not only on laptops),
>or do you not intend to further work on this besides the patches you present
>here?

My plan is to add some code to measure and record the maximum "cpu_tick"
frequency we see, and use that to normalize the cpu accounting.

That way, the user/system time reported will get units of "cpu seconds
if the cpu ran full speed".

--
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[hidden email]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [TEST/REVIEW] CPU accounting patches

Alexander Leidinger
Poul-Henning Kamp <[hidden email]> wrote:

> In message <[hidden email]>,
> Alexander Lei
> dinger writes:
>
>> Are you going to fix those issues for machines which do power saving tricks
>> (which may even be useful on servers for some people, not only on laptops),
>> or do you not intend to further work on this besides the patches you present
>> here?
>
> My plan is to add some code to measure and record the maximum "cpu_tick"
> frequency we see, and use that to normalize the cpu accounting.
>
> That way, the user/system time reported will get units of "cpu seconds
> if the cpu ran full speed".

How large do you expect the error will be?

Bye,
Alexander.

--
http://www.Leidinger.net  Alexander @ Leidinger.net: PGP ID = B0063FE7
http://www.FreeBSD.org     netchild @ FreeBSD.org  : PGP ID = 72077137
Eternity is a terrible thought.  I mean, where's it going to end?
                -- Tom Stoppard


_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [TEST/REVIEW] CPU accounting patches

Poul-Henning Kamp
In message <[hidden email]>, Alexander Lei
dinger writes:


>> That way, the user/system time reported will get units of "cpu seconds
>> if the cpu ran full speed".
>
>How large do you expect the error will be?

I don't consider it an error, I consider it increasing precision.


If you run

        time mycommand

on your laptop, and along the way the CPU clock ramps up from
75 MHz to 600 MHz before it reports

        user 2.01  sys 0.30 real 4.00

What exactly have you learned from the first two numbers with the
current definition of "cpu second" ?


With my definition you would be more likely to see lower numbers
maybe
        user 0.20  sys 0.03 real 4.00

And they would have meaning, they should be pretty much the same
no matter what speed your CPU runs at any instant in time.

In theory, it should be possible to compare user/sys numbers
you collect while running at 75 MHz with the ones you got
under full steam at 1600 MHz.

In practice however, things that run on the real time, HZ
interrupting to run hardclock() for instance, will still make
comparison of such numbers quite shaky.

But at least they will not be random as they are now.


--
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[hidden email]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [TEST/REVIEW] CPU accounting patches

Ian FREISLICH
"Poul-Henning Kamp" wrote:
> In message <[hidden email]>, Alexander L
ei

> dinger writes:
>
>
> >> That way, the user/system time reported will get units of "cpu seconds
> >> if the cpu ran full speed".
> >
> >How large do you expect the error will be?
>
> I don't consider it an error, I consider it increasing precision.
>
>
> If you run
>
> time mycommand
>
> on your laptop, and along the way the CPU clock ramps up from
> 75 MHz to 600 MHz before it reports
>
> user 2.01  sys 0.30 real 4.00
>
> What exactly have you learned from the first two numbers with the
> current definition of "cpu second" ?

"One second's worth of the computer's processing time, which is
based on actual machine cycles used, not calendar time." ?

Is the getrusage() manual page out of date?  It claims that user
and system time is is "the total amount of time spent executing in
user mode" and "the total amount of time spent in the system executing
on behalf of the process(es)".

> With my definition you would be more likely to see lower numbers
> maybe
> user 0.20  sys 0.03 real 4.00
>
> And they would have meaning, they should be pretty much the same
> no matter what speed your CPU runs at any instant in time.

For how much of those 4 real seconds was the computer doing something
else using your definition?  It's certainly not 3.77.  It's probably
closer to 1.69.

> In theory, it should be possible to compare user/sys numbers
> you collect while running at 75 MHz with the ones you got
> under full steam at 1600 MHz.

If my CPU clock runs slower for a period of time, processes remain
on the CPU for longer.  I don't really see how 0.23 [wallclock]
seconds _if_ the cpu ran [at] full speed is different to 2.31
wallclock seconds in this context.  One is scaled to maximum CPU
clock frequency and the other is scaled to wallclock time.

I find the wallclock scale a bit less confusing because I normally
exist in that scale[1]: on my two hypothetical identical servers, one
clocked down to 50% for some reason, the same job takes twice the
wallclock time but identical CPU time?

Ian

--
Ian Freislich

1. It would be nice to say to my boss that this project would have
   taken a week if I'd worked faster and get a fat bonus because I
   could have done it faster.
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [TEST/REVIEW] CPU accounting patches

Poul-Henning Kamp
In message <[hidden email]>, Ian FREISLICH writes:

>"One second's worth of the computer's processing time, which is
>based on actual machine cycles used, not calendar time." ?
>
>Is the getrusage() manual page out of date?

Yes.

It was written before anybody had gotten the rather weird idea to
have a CPU change frequency.  Back then it was all about running
as fast as possible all the time.

We are therefore forced to try to divine the intent behind the text,
and as somebody who were around back in the eighties I can testify
that the intent was to be able to bill computer users for CPU
instructions.

Since the clock rate was constant, cpu seconds was a usable
approximation.

These days with variable clockrate, the cpu second is a bad approximation.
If my CPU runs at 600MHz, even if used 100%, it can still do three times
as much work, so the fact that my process takes 3 seconds to complete
does not mean that I have used (in the sense of denying other users the
ability to use) all of the CPU for three seconds.  If I had monopolized
the entire CPU to its fullest potential, it would have taken only one
second.

--
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[hidden email]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [TEST/REVIEW] CPU accounting patches

Peter Jeremy
On Wed, 2006-Jan-25 20:09:54 +0100, Poul-Henning Kamp wrote:
>We are therefore forced to try to divine the intent behind the text,
>and as somebody who were around back in the eighties I can testify
>that the intent was to be able to bill computer users for CPU
>instructions.

This implies that RDTSC (and equivalents) would be the best source of
accounting information, with CPU usage billed in CPU cycles used.
It's just users who expect to be billed in seconds.

>These days with variable clockrate, the cpu second is a bad approximation.

Agreed.

>If my CPU runs at 600MHz, even if used 100%, it can still do three times
>as much work, so the fact that my process takes 3 seconds to complete
>does not mean that I have used (in the sense of denying other users the
>ability to use) all of the CPU for three seconds.

This depends on why the CPU was running at 600MHz instead of 1800MHz.
If the user requested that speed (for whatever reason), then that user
_was_ denying other users the ability to use the CPU.

--
Peter Jeremy
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [TEST/REVIEW] CPU accounting patches

Poul-Henning Kamp
In message <[hidden email]>, Peter Jeremy wri
tes:
>On Wed, 2006-Jan-25 20:09:54 +0100, Poul-Henning Kamp wrote:
>>We are therefore forced to try to divine the intent behind the text,
>>and as somebody who were around back in the eighties I can testify
>>that the intent was to be able to bill computer users for CPU
>>instructions.
>
>This implies that RDTSC (and equivalents) would be the best source of
>accounting information, with CPU usage billed in CPU cycles used.
>It's just users who expect to be billed in seconds.

Right, so we bill users in "full speed CPU second equvivalents"

--
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[hidden email]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [TEST/REVIEW] CPU accounting patches

Scott Long-2
Poul-Henning Kamp wrote:

> In message <[hidden email]>, Peter Jeremy wri
> tes:
>
>>On Wed, 2006-Jan-25 20:09:54 +0100, Poul-Henning Kamp wrote:
>>
>>>We are therefore forced to try to divine the intent behind the text,
>>>and as somebody who were around back in the eighties I can testify
>>>that the intent was to be able to bill computer users for CPU
>>>instructions.
>>
>>This implies that RDTSC (and equivalents) would be the best source of
>>accounting information, with CPU usage billed in CPU cycles used.
>>It's just users who expect to be billed in seconds.
>
>
> Right, so we bill users in "full speed CPU second equvivalents"
>

Regardless of the technical merits of one accounting method or another,
changing the results of rusage is going to result in many years of
questions to the mailing lists and grumbling from uneducated sysadmins
that FreeBSD is somehow inferior because of this one detail.  I know
that's an emotional argument and not a technical one, but it's also
important to consider.

Scott

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [TEST/REVIEW] CPU accounting patches

Poul-Henning Kamp
In message <[hidden email]>, Scott Long writes:

>Regardless of the technical merits of one accounting method or another,
>changing the results of rusage is going to result in many years of
>questions to the mailing lists and grumbling from uneducated sysadmins
>that FreeBSD is somehow inferior because of this one detail.  I know
>that's an emotional argument and not a technical one, but it's also
>important to consider.

Well, there is up to 30% improvement in contextswitches to pay for
the grumbling.

I think more people care about context switches than cpu accounting,
but I also think they may not know this.

--
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[hidden email]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [TEST/REVIEW] CPU accounting patches

Kris Kennaway
In reply to this post by Poul-Henning Kamp
On Tue, Jan 24, 2006 at 10:10:45PM +0100, Poul-Henning Kamp wrote:

> On a sparc64 it gives 3.2+/-.3% speedup on unixbench/context1

For me on an e4500 it gives

> ministat old new
x old
+ new
+--------------------------------------------------------------------------+
|      x xx        x  +                 +     +   x +                     +|
||________M________A________|________|________MA__________________|        |
+--------------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x   5        5006.9        5042.7        5009.3       5016.96     14.894563
+   5        5019.4        5062.8        5039.4       5039.98     15.787717
Difference at 95.0% confidence
        23.02 +/- 22.3836
        0.458844% +/- 0.44616%
        (Student's t, pooled s = 15.3476)

Kris

attachment0 (194 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [TEST/REVIEW] CPU accounting patches

Ian FREISLICH
In reply to this post by Poul-Henning Kamp
"Poul-Henning Kamp" wrote:

> In message <[hidden email]>, Ian FREISLICH writes:
>
> >"One second's worth of the computer's processing time, which is
> >based on actual machine cycles used, not calendar time." ?
> >
> >Is the getrusage() manual page out of date?
>
> Yes.
>
> It was written before anybody had gotten the rather weird idea to
> have a CPU change frequency.  Back then it was all about running
> as fast as possible all the time.
>
> We are therefore forced to try to divine the intent behind the text,
> and as somebody who were around back in the eighties I can testify
> that the intent was to be able to bill computer users for CPU
> instructions.

I wonder how many people still bill for CPU time?  I'd go for the
faster context switches.

Ian

--
Ian Freislich
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [TEST/REVIEW] CPU accounting patches

Brian Candler
In reply to this post by Poul-Henning Kamp
On Wed, Jan 25, 2006 at 09:28:16PM +0100, Poul-Henning Kamp wrote:
> Right, so we bill users in "full speed CPU second equvivalents"

How about "BogoMIPS-seconds"?

<ducks/>

Seriously... don't forget that the *other* usage of CPU-second accounting is
for system administrators to assess the amount of CPU resource used by a
particular task, in order to plan when the machine is going to need
upgrading.

In this case, the administrator is not so much interested in the absolute
amount of work done, as the amount of work done as a proportion of total
work capacity on a particular machine. That is, if task X uses 1200
CPU-seconds over a period of one hour, that's a third of the total available
capacity on that machine [1].

If the CPU were then cranked down to 1/3rd of its clock speed, this task
would be using the full CPU capacity - and observing that this process is
now using 3600 CPU-seconds in an hour is a useful view of the real
situation, rather than some mythical 1200 CPU-seconds which it *would have*
used *if* it had been running on a different machine (i.e. a machine similar
to this one, but running at a faster clock speed). The machine is maxed out
on CPU, and that's what matters.

Another way of looking at this is that if the CPU is running at 1/3rd speed
then CPU cycles are three times as rare, and therefore three times as
expensive. That's not good from the point of view of a timeshare user who
pays for CPU seconds, as they end up paying three times as much for the same
amount of work [2][3]. But it's realistic, especially if the end user owns,
runs and pays for the whole asset (which I suggest is more common than the
timeshare user these days)

Regards,

Brian.

[1] Of course a dual-CPU box has a capacity of 7200 CPU-seconds per hour, so
1200 CPU-seconds would be one sixth. I don't see a need to normalise that,
even if that means I'm taking a slightly inconsistent position :-) Admins
are used to thinking of a 4-CPU box as a kind-of cluster of 4 machines.

[2] If today CPU cycles are three times as expensive as normal, because the
sysadmin needed to reduce the clock speed (e.g. air conditioning failure?)
then the user can always choose to run their application on a different day
instead.

[3] On a multi-CPU machine, bottlenecks such as RAM I/O may mean that the
same sequence of instructions takes more cycles (and hence time) to execute
than on a single CPU machine, even at the same clock speed. The timeshare
user may also feel unfairly penalised for this - but I don't see there's
much that can be done about it. That is, it's very difficult to charge the
timeshare user for absolute work done, completely independent of the
platform their application runs on. I think it's reasonable to charge them
based on the proportion of resource they've used on the actual machine
they've chosen to run it on, at the time they've chosen to run it.
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [TEST/REVIEW] CPU accounting patches

Marius Strobl
In reply to this post by Poul-Henning Kamp
On Tue, Jan 24, 2006 at 10:10:45PM +0100, Poul-Henning Kamp wrote:

>
> Here is a new version of my cpu accounting change patch.
>
> http://phk.freebsd.dk/patch/cpu_acct_1.patch
>
> This patch is supposedly harmless (or at least mostly harmless)
> and I'd appreciate it getting a solid trashing.
>
>
> This patchs changes cpu accounting from accumulating charges
> in real-time units and instead accumulates in units of some
> per-arch, possibly per-cpu counter.
>
> When the accumulated charge is read by times(2) or getrusage(2) or
> similar, the frequency of the counter is interrogated and the charge
> normalized to microseconds.
>
> With this patch, the counter is always the timecounter and the only
> real difference is therefore a minor performance change (because we
> save the normalizing multiplications for each context switch).
>
> On my AMD Athlon 700 and my Sun Ultra 60 the performance difference
> is barely 1% and of doubtful statistical quality.
>
> On my Opteron machine I get a 2.7+/-.6% boost on unixbench's
> context1 test.
>
> Of course, the scheme used in this patch suffers a bit if the
> hardware counter changes to other hardware of a different rate
> or simply changes rate.  This has been discussed at length in
> a previous thread already, and I'll simply refer to it, rather
> than rehash here:
>
> http://lists.freebsd.org/pipermail/freebsd-net/2005-October/008637.html
>
>
>
> The other half of this work is in this separate patch, and this is
> not yet complete.  You are welcome to test it however, as long as
> you are aware of the problems it may hold:
>
> http://phk.freebsd.dk/patch/cpu_acct_2.patch
>
> It makes i386 and amd64 use the TSC and sparc64 use the "tick"
> counter for CPU accounting.
>
> On a sparc64 it gives 3.2+/-.3% speedup on unixbench/context1
>
> On a Athlon700 with i8254 timecounter it gives a 95+/-.8% speedup
>
> On a Opteron with ACPI-fast timecounter it gives a 36+/-.6% speedup.
>
> The downside is, that unless your cpu clock is correctly probed
> at boot and stays constant, your cpu accounting numbers will have
> a bogus scaling factor.
>
> I belive all the sparc64s we support have constant CPU rates,
> so they should be safe.
>

USIIe and greater CPUs support changing the CPU frequency which
also affects the tick counter. Regarding CPUs currently supported
by FreeBSD/sparc64 this translates to the 550MHz and 650MHz USIIi
besides the USIIe CPUs. We currently don't support changing their
frequency though but I think that would be easy to do.
One other thing that might cause problems regarding your work is
that the tick counters are not in sync across the CPUs of a MP
system. We try to sync the tick counter of APs with the tick
counter of the BSP when attaching the APs but that's already
not perfect and they are not that constant over time. That's
why we currently use the timecounter of a host-PCI or a host-SBus
bridge respectively instead of the tick counters on USI/II MP
systems.
With USIII CPUs a MP system additionally can consist of different
CPU models running at different clock speeds (and having different
caches).

Marius

--
This mail was scanned by AntiVir Milter.
This product is licensed for non-commercial use.
See www.antivir.de for details.
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [TEST/REVIEW] CPU accounting patches

Poul-Henning Kamp
In message <[hidden email]>, Marius Strobl writes:

>One other thing that might cause problems regarding your work is
>that the tick counters are not in sync across the CPUs of a MP
>system.

Yes, this is outstanding, see my discussion with jhb about this.

Synchronizing the counters will not be necessary.

--
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[hidden email]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: [TEST/REVIEW] CPU accounting patches

Thomas Sparrevohn
In reply to this post by Ian FREISLICH
On Thursday 26 January 2006 06:06, Ian FREISLICH wrote:

>
> I wonder how many people still bill for CPU time?  I'd go for the
> faster context switches.
>

Almost all major ITO's providers - From SUN, HP, IBM, EDS etc. has offerings
that in some shape or other uses a "Utility model" based upon some sort of
financial model based upon actual CPU/IO etc. usage - It is a major area now
and provides one of the corner stones in the movement towards "Public Utility
models"

So it is very relevant as an area for general improvement and the "historical"
models are not really good enough, for further information take a look a
products as MicroMeasure etc.

Regards
        Thomas



_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
123