FreeBSD 7.1 BETA 2 vs Opensolaris vs Ubuntu performance

classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|

FreeBSD 7.1 BETA 2 vs Opensolaris vs Ubuntu performance

Steven Hartland
http://www.phoronix.com/scan.php?page=article&item=os_threeway_2008&num=1

Was interesting until I saw this:-

"However, it's important to reiterate that all three operating systems were left in their stock configurations and that no
additional tweaking had occurred."

I kernel debugging stuff still enabled in FreeBSD 7.1 BETA 2, if so these results are useless
and someone should contact the article writers to correct this.

    Regards
    Steve


================================================
This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it.

In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337
or return the E.mail to [hidden email].

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD 7.1 BETA 2 vs Opensolaris vs Ubuntu performance

Andrew Thompson-2
On Mon, Nov 24, 2008 at 08:28:11PM -0000, Steven Hartland wrote:

> http://www.phoronix.com/scan.php?page=article&item=os_threeway_2008&num=1
>
> Was interesting until I saw this:-
>
> "However, it's important to reiterate that all three operating systems were
> left in their stock configurations and that no additional tweaking had
> occurred."
>
> I kernel debugging stuff still enabled in FreeBSD 7.1 BETA 2, if so these results are useless
> and someone should contact the article writers to correct this.

The stable branches do not have any debugging turned on, only head. This
would have been true if they tested a point-zero release like 7.0 BETA 2.


Andrew
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD 7.1 BETA 2 vs Opensolaris vs Ubuntu performance

mdtancsa
In reply to this post by Steven Hartland
At 03:28 PM 11/24/2008, Steven Hartland wrote:
>http://www.phoronix.com/scan.php?page=article&item=os_threeway_2008&num=1
>
>Was interesting until I saw this:-
>
>"However, it's important to reiterate that all three operating
>systems were left in their stock configurations and that no
>additional tweaking had occurred."

They all seem to be fairly close in the majority of tests.

In the conclusion they write, "In our LAME MP3 encoding test, Ubuntu
8.10 was the fastest followed by FreeBSD 7.1".... One needs to
consider, the difference was 2%... Thats hardly anything to get
excited about, especially if that difference can be accounted for by
how much priority the scheduler gives to a userland app vs what the
system is doing etc....

Also, it would have been interesting to see more multithreaded work
loads.  A lot of the apps they tested seemed to be single threaded,
doing one job.

         ---Mike

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD 7.1 BETA 2 vs Opensolaris vs Ubuntu performance

Phil Brennan
In reply to this post by Steven Hartland
Sorry thats one of the worst pieces of "journalism" I've ever
encountered. Benchmarks useless ( single threaded ), no useful
analysis, not even any speculation ( informed or otherwise ).


On Mon, Nov 24, 2008 at 8:28 PM, Steven Hartland
<[hidden email]> wrote:

> http://www.phoronix.com/scan.php?page=article&item=os_threeway_2008&num=1
>
> Was interesting until I saw this:-
>
> "However, it's important to reiterate that all three operating systems were
> left in their stock configurations and that no additional tweaking had
> occurred."
>
> I kernel debugging stuff still enabled in FreeBSD 7.1 BETA 2, if so these
> results are useless
> and someone should contact the article writers to correct this.
>
>   Regards
>   Steve
>
> ================================================
> This e.mail is private and confidential between Multiplay (UK) Ltd. and the
> person or entity to whom it is addressed. In the event of misdirection, the
> recipient is prohibited from using, copying, printing or otherwise
> disseminating it or any information contained in it.
> In the event of misdirection, illegible or incomplete transmission please
> telephone +44 845 868 1337
> or return the E.mail to [hidden email].
>
> _______________________________________________
> [hidden email] mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-performance
> To unsubscribe, send any mail to
> "[hidden email]"
>
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD 7.1 BETA 2 vs Opensolaris vs Ubuntu performance

K. Macy
On Mon, Nov 24, 2008 at 11:43 PM, Phil Brennan <[hidden email]> wrote:
> Sorry thats one of the worst pieces of "journalism" I've ever
> encountered. Benchmarks useless ( single threaded ), no useful
> analysis, not even any speculation ( informed or otherwise ).

It is representative. This stands out in your mind because you are
familiar with the subject matter.



> On Mon, Nov 24, 2008 at 8:28 PM, Steven Hartland
> <[hidden email]> wrote:
>> http://www.phoronix.com/scan.php?page=article&item=os_threeway_2008&num=1
>>
>> Was interesting until I saw this:-
>>
>> "However, it's important to reiterate that all three operating systems were
>> left in their stock configurations and that no additional tweaking had
>> occurred."
>>
>> I kernel debugging stuff still enabled in FreeBSD 7.1 BETA 2, if so these
>> results are useless
>> and someone should contact the article writers to correct this.
>>
>>   Regards
>>   Steve
>>
>> ================================================
>> This e.mail is private and confidential between Multiplay (UK) Ltd. and the
>> person or entity to whom it is addressed. In the event of misdirection, the
>> recipient is prohibited from using, copying, printing or otherwise
>> disseminating it or any information contained in it.
>> In the event of misdirection, illegible or incomplete transmission please
>> telephone +44 845 868 1337
>> or return the E.mail to [hidden email].
>>
>> _______________________________________________
>> [hidden email] mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-performance
>> To unsubscribe, send any mail to
>> "[hidden email]"
>>
> _______________________________________________
> [hidden email] mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-performance
> To unsubscribe, send any mail to "[hidden email]"
>



--
If we desire respect for the law, we must first make the law respectable.
- Louis D. Brandeis
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD 7.1 BETA 2 vs Opensolaris vs Ubuntu performance

Liudas B.
In reply to this post by Steven Hartland
Hello,

        I did some calculations to take into account not only best
result, but how much it is better. I used following formulas:

        If more is better then result/max(ubuntu, opensolaris,
freebsd)
        If less is better then min(ubuntu, opensolaris, freebsd)/
result

Results in phoronix article

                                Ubuntu OpenSolaris
FreeBSD
LAME Encoding 43,3 53,51 45,87
7-Zip Compression 10612 9655,99
9323,66
Timed Gzip Compression 111,08 116,54 122,32
GnuPG 43,49 62,39 44,72
Tandem XML 30,89 54,52 29,01
Bork File Encrypter 78,4 55,31 79,36
Java SciMark FFT 455,53 480,93 447,24
Java SciMark matrix mul. 549,16 438,25 543,02
Java SciMark SOR 742,77 734 731,22
Java SciMark Montecarlo 162,11 167,89 160,53
Sunflow rendering 6,24 6,57 5,62
Bonnie++ seq 34861 61387 56124
Bonnie++ rand read 50511 60626 56255
Bonnie++ rand del 209,3 362,4 145,4
BYTE dhrystone 8604934,5 7486935,6
12122380,5
BYTE register 553949 555089,5
547974,3
BYTE fp 1284795 933096,5
1271991,7

Results after calculations

Ubuntu OpenSolaris FreeBSD
1 0,81 0,94
1 0,91 0,88
1 0,95 0,91
1 0,7 0,97
0,94 0,53 1
0,71 1 0,7
0,95 1 0,93
1 0,8 0,99
1 0,99 0,98
0,97 1 0,96
0,9 0,86 1
0,57 1 0,91
0,83 1 0,93
0,58 1 0,4
0,71 0,62 1
1 1 0,99
1 0,73 0,99

SUM
15,14 14,89 15,48

AVERAGE
0,89 0,88 0,91


I hope results are readable

--
Liudas


>At 03:28 PM 11/24/2008, Steven Hartland wrote:
>>http://www.phoronix.com/scan.php?
page=article&item=os_threeway_2008&num=1

>>
>>Was interesting until I saw this:-
>>
>>"However, it's important to reiterate that all three operating
>>systems were left in their stock configurations and that no
>>additional tweaking had occurred."
>
>They all seem to be fairly close in the majority of tests.
>
>In the conclusion they write, "In our LAME MP3 encoding test, Ubuntu
>8.10 was the fastest followed by FreeBSD 7.1".... One needs to
>consider, the difference was 2%... Thats hardly anything to get
>excited about, especially if that difference can be accounted for by
>how much priority the scheduler gives to a userland app vs what the
>system is doing etc....
>
>Also, it would have been interesting to see more multithreaded work
>loads.  A lot of the apps they tested seemed to be single threaded,
>doing one job.
>
>         ---Mike
>
>_______________________________________________
>[hidden email] mailing list
>http://lists.freebsd.org/mailman/listinfo/freebsd-performance
>To unsubscribe, send any mail to "freebsd-performance-
[hidden email]"
>
>

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD 7.1 BETA 2 vs Opensolaris vs Ubuntu performance

Ivan Voras
In reply to this post by Steven Hartland
Steven Hartland wrote:
> http://www.phoronix.com/scan.php?page=article&item=os_threeway_2008&num=1
>
> Was interesting until I saw this:-
>

The results seem well within expectations, for the sort of benchmarks
they did: there is little difference between the systems. Depending on
the details of how they did the benchmarks and how they processed the
results (if at all), the results can even be within the margin of error
(i.e. useless for mutual comparison except to show the systems are all
very similar).

The benchmarks they did are mostly focused on number crunching and do
not even touch the area of system scalability to multiple CPUs, which
could have been easily done but they chose not to. Number crunching is a
bad choice for system scalability measure because down to the metal, all
systems use similar compilers and there's nothing the OS can do to
improve (or actually hinder, in the common case) CPU's execution of math
code.

For example, they could have included the "shells concurrent" benchmark
from unixbench, started multiple bonnie++'s in parallel, and included a
few system benchmarks like lighttpd+siege, parallel gzip (pigz) and
blogbench.

> "However, it's important to reiterate that all three operating systems
> were left in their stock configurations and that no additional tweaking
> had occurred."
>
> I kernel debugging stuff still enabled in FreeBSD 7.1 BETA 2, if so
> these results are useless
> and someone should contact the article writers to correct this.

There is no debugging in -STABLE. WYSIWYG :)



signature.asc (260 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD 7.1 BETA 2 vs Opensolaris vs Ubuntu performance

Adrian Chadd-2
In reply to this post by mdtancsa
2% may not sound like a lot but it starts becoming measurable savings
when the number of boxes involved is ${LARGE}.

2c,


Adrian

2008/11/24 Mike Tancsa <[hidden email]>:

> At 03:28 PM 11/24/2008, Steven Hartland wrote:
>>
>> http://www.phoronix.com/scan.php?page=article&item=os_threeway_2008&num=1
>>
>> Was interesting until I saw this:-
>>
>> "However, it's important to reiterate that all three operating systems
>> were left in their stock configurations and that no additional tweaking had
>> occurred."
>
> They all seem to be fairly close in the majority of tests.
>
> In the conclusion they write, "In our LAME MP3 encoding test, Ubuntu 8.10
> was the fastest followed by FreeBSD 7.1".... One needs to consider, the
> difference was 2%... Thats hardly anything to get excited about, especially
> if that difference can be accounted for by how much priority the scheduler
> gives to a userland app vs what the system is doing etc....
>
> Also, it would have been interesting to see more multithreaded work loads.
>  A lot of the apps they tested seemed to be single threaded, doing one job.
>
>        ---Mike
> _______________________________________________
> [hidden email] mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-performance
> To unsubscribe, send any mail to
> "[hidden email]"
>
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD 7.1 BETA 2 vs Opensolaris vs Ubuntu performance

mdtancsa
In reply to this post by mdtancsa
At 12:06 PM 11/25/2008, Adrian Chadd wrote:
>2% may not sound like a lot but it starts becoming measurable savings
>when the number of boxes involved is ${LARGE}.

True, but then again is there such a thing as a synthetic benchmark
that would have a margin of error less than 2% while representing
your real world application mix?

         ---Mike

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD 7.1 BETA 2 vs Opensolaris vs Ubuntu performance

Adrian Chadd-2
2008/11/25 Mike Tancsa <[hidden email]>:
> At 12:06 PM 11/25/2008, Adrian Chadd wrote:
>>
>> 2% may not sound like a lot but it starts becoming measurable savings
>> when the number of boxes involved is ${LARGE}.
>
> True, but then again is there such a thing as a synthetic benchmark that
> would have a margin of error less than 2% while representing your real world
> application mix?

It depends, what do you think people read when making decisions? :)

Bout the only thing that could fix this is for people involved with
FreeBSD to re-do the benchmark, documenting how it was done wrong, and
reporting on the results.

Note that I didn't say "show FreeBSD is better", at least not in the
initial round.



Adrian
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD 7.1 BETA 2 vs Opensolaris vs Ubuntu performance

Roman Divacky
In reply to this post by Ivan Voras
On Tue, Nov 25, 2008 at 12:08:27PM +0100, Ivan Voras wrote:

> Steven Hartland wrote:
> > http://www.phoronix.com/scan.php?page=article&item=os_threeway_2008&num=1
> >
> > Was interesting until I saw this:-
> >
>
> The results seem well within expectations, for the sort of benchmarks
> they did: there is little difference between the systems. Depending on
> the details of how they did the benchmarks and how they processed the
> results (if at all), the results can even be within the margin of error
> (i.e. useless for mutual comparison except to show the systems are all
> very similar).
>
> The benchmarks they did are mostly focused on number crunching and do
> not even touch the area of system scalability to multiple CPUs, which
> could have been easily done but they chose not to. Number crunching is a
> bad choice for system scalability measure because down to the metal, all
> systems use similar compilers and there's nothing the OS can do to

I believe most of the synthetic numbers (mp3 encoding etc.) difference
comes from the different version of gcc the different OS uses...
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD 7.1 BETA 2 vs Opensolaris vs Ubuntu performance

Ivan Voras
Roman Divacky wrote:

> On Tue, Nov 25, 2008 at 12:08:27PM +0100, Ivan Voras wrote:
>> Steven Hartland wrote:
>>> http://www.phoronix.com/scan.php?page=article&item=os_threeway_2008&num=1
>>>
>>> Was interesting until I saw this:-
>>>
>> The results seem well within expectations, for the sort of benchmarks
>> they did: there is little difference between the systems. Depending on
>> the details of how they did the benchmarks and how they processed the
>> results (if at all), the results can even be within the margin of error
>> (i.e. useless for mutual comparison except to show the systems are all
>> very similar).
>>
>> The benchmarks they did are mostly focused on number crunching and do
>> not even touch the area of system scalability to multiple CPUs, which
>> could have been easily done but they chose not to. Number crunching is a
>> bad choice for system scalability measure because down to the metal, all
>> systems use similar compilers and there's nothing the OS can do to
>
> I believe most of the synthetic numbers (mp3 encoding etc.) difference
> comes from the different version of gcc the different OS uses...
You're very likely right. Ubuntu 8.10 has gcc 4.3.x - it could make for
the small difference in gzip and 7z compression performance.



signature.asc (266 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD 7.1 BETA 2 vs Opensolaris vs Ubuntu performance

Adrian Chadd-2
2008/11/25 Ivan Voras <[hidden email]>:

>> I believe most of the synthetic numbers (mp3 encoding etc.) difference
>> comes from the different version of gcc the different OS uses...
>
> You're very likely right. Ubuntu 8.10 has gcc 4.3.x - it could make for
> the small difference in gzip and 7z compression performance.

Well, that should be a reasonably easy thing to test and feed back to
the author.



Adrian
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD 7.1 BETA 2 vs Opensolaris vs Ubuntu performance

Ivan Voras
2008/11/25 Adrian Chadd <[hidden email]>:

> 2008/11/25 Ivan Voras <[hidden email]>:
>
>>> I believe most of the synthetic numbers (mp3 encoding etc.) difference
>>> comes from the different version of gcc the different OS uses...
>>
>> You're very likely right. Ubuntu 8.10 has gcc 4.3.x - it could make for
>> the small difference in gzip and 7z compression performance.
>
> Well, that should be a reasonably easy thing to test and feed back to
> the author.

OTOH if the goal is to measure "operating system" performance, this
must also include the compiler, libraries and all. (for example, what
does Solaris default to nowadays? I think it ships with gcc but not as
default). The hold on gcc 4.3 in FreeBSD is, after all, political
(licencing).

If FreeBSD base ever switches to LLVM+clang, this means libc will be
compiled with a non-gcc compiler which will forever change the
performance for simple "real world" benchmarks.
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD 7.1 BETA 2 vs Opensolaris vs Ubuntu performance

Mark Kirkwood
In reply to this post by Adrian Chadd-2
Adrian Chadd wrote:
> 2% may not sound like a lot but it starts becoming measurable savings
> when the number of boxes involved is ${LARGE}.
>
> 2c,
>
>  
Yeah, but the margin for error in these tests is undoubtedly > 2%.

AFAICS this benchmark shows that Freebsd is "up there" with these guys -
slower on some, faster on some - no problem.

regards

Mark
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD 7.1 BETA 2 vs Opensolaris vs Ubuntu performance

Alexander Leidinger
In reply to this post by Ivan Voras
Quoting Ivan Voras <[hidden email]> (from Tue, 25 Nov 2008  
21:46:35 +0100):

> 2008/11/25 Adrian Chadd <[hidden email]>:
>> 2008/11/25 Ivan Voras <[hidden email]>:
>>
>>>> I believe most of the synthetic numbers (mp3 encoding etc.) difference
>>>> comes from the different version of gcc the different OS uses...
>>>
>>> You're very likely right. Ubuntu 8.10 has gcc 4.3.x - it could make for
>>> the small difference in gzip and 7z compression performance.
>>
>> Well, that should be a reasonably easy thing to test and feed back to
>> the author.
>
> OTOH if the goal is to measure "operating system" performance, this

If you want to test OS performance and use Java programs in there to  
do so, you would use the same Java version, wouldn't you? They didn't.

If you want to run some high performance java software and you want to  
know on which OS it performs best, you would test the same Java  
version on the OS' in question (or at least you should do that, to not  
compare apples and oranges).

If you want to run number crunching software, you are interested in  
high computing throughput of your app, so you use a compiler which  
performs best for your code in question (which would mean probably the  
Intel compiler or the Portland compiler on Linux, maybe the Sun  
compiler on Solaris, and probably gcc on FreeBSD). You also want to  
optimize the code for your CPU (it makes a difference if you do  
floating point calculations and are allowed to use the SSEx or  
whatever instructions), and not some generic settings the OS comes with.

The "benchmark" presented there is flawed in a lot of ways. No  
descrition what they really want to benchmark, no description what  
each subtest benchmarks (e.g. lame is performing on one CPU and  
occasionally performs IO, what does this benchmark mean? That your  
multi-CPU system is mostly idle and can be used to browse the net  
without that you notice any impact). Only absolute numbers and no  
relative performance comparision (percentage of difference).  
Inconsistent starting point (not the same compiler, not the same java  
version, ...) in case you want to promote an OS for specialized tasks  
(there are comments which tell FreeBSD would be good for raytracing,  
as the corresponding subtest was the fastest on FreeBSD), and so on.

Did I overlook some part where they tell how they test? Do they  
calculate the average of several runs?

> must also include the compiler, libraries and all. (for example, what
> does Solaris default to nowadays? I think it ships with gcc but not as
> default). The hold on gcc 4.3 in FreeBSD is, after all, political
> (licencing).

Users most of the time don't care what the reasons are, they use what  
is there and complain or switch if it works better somewhere else.  
People which care about compute intense stuff, will install their  
preferred compiler anyway.

Bye,
Alexander.

--
So so is good, very good, very excellent good:
and yet it is not; it is but so so.
                -- William Shakespeare, "As You Like It"

http://www.Leidinger.net    Alexander @ Leidinger.net: PGP ID = B0063FE7
http://www.FreeBSD.org       netchild @ FreeBSD.org  : PGP ID = 72077137
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD 7.1 BETA 2 vs Opensolaris vs Ubuntu performance

Ivan Voras
2008/11/26 Alexander Leidinger <[hidden email]>:

> If you want to test OS performance and use Java programs in there to do so,
> you would use the same Java version, wouldn't you? They didn't.

Linux: 1.6.0_0-b12
Solaris: 1.6.0_10-b33
FreeBSD: 1.6.0_07-b02

Since system have their local patches (I know FreeBSD does), I don't
think it's even possible to test "exactly the same" version ;)

But this also goes into the "What OS ships with" category.

> If you want to run number crunching software, you are interested in high
> computing throughput of your app, so you use a compiler which performs best
> for your code in question (which would mean probably the Intel compiler or
> the Portland compiler on Linux, maybe the Sun compiler on Solaris, and
> probably gcc on FreeBSD). You also want to optimize the code for your CPU
> (it makes a difference if you do floating point calculations and are allowed
> to use the SSEx or whatever instructions), and not some generic settings the
> OS comes with.

I think they went with the "stock" configurations as that's what
almost all users will use.

> The "benchmark" presented there is flawed in a lot of ways.

Yes.
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD 7.1 BETA 2 vs Opensolaris vs Ubuntu performance

Alexander Leidinger
Quoting Ivan Voras <[hidden email]> (from Wed, 26 Nov 2008  
10:55:39 +0100):

> 2008/11/26 Alexander Leidinger <[hidden email]>:
>
>> If you want to test OS performance and use Java programs in there to do so,
>> you would use the same Java version, wouldn't you? They didn't.
>
> Linux: 1.6.0_0-b12
> Solaris: 1.6.0_10-b33
> FreeBSD: 1.6.0_07-b02

The important part is the _XX, not the -bYY. The bYY may be something  
we don't care about, but the _XX part is something which may cause  
performance differences.

> Since system have their local patches (I know FreeBSD does), I don't
> think it's even possible to test "exactly the same" version ;)
>
> But this also goes into the "What OS ships with" category.

We don't ship with java at all... strictly speaking. ;)

>> If you want to run number crunching software, you are interested in high
>> computing throughput of your app, so you use a compiler which performs best
>> for your code in question (which would mean probably the Intel compiler or
>> the Portland compiler on Linux, maybe the Sun compiler on Solaris, and
>> probably gcc on FreeBSD). You also want to optimize the code for your CPU
>> (it makes a difference if you do floating point calculations and are allowed
>> to use the SSEx or whatever instructions), and not some generic settings the
>> OS comes with.
>
> I think they went with the "stock" configurations as that's what
> almost all users will use.

I fully agree. But number crunching (as benchmarked, and I'm not  
talking about LAME which has a 2% difference) is not something almost  
all users will do. Something the masses may do with the OS is not  
covered at all, no browser tests, no interactivity (maybe with high  
load in the background) tests. As I said, they don't even tell what  
they want to test (and as such, everything we can do is speculate...  
that's not something which will lead to interesting results in the  
thread).

Bye,
Alexander.

--
Man must shape his tools lest they shape him.
                -- Arthur R. Miller

http://www.Leidinger.net    Alexander @ Leidinger.net: PGP ID = B0063FE7
http://www.FreeBSD.org       netchild @ FreeBSD.org  : PGP ID = 72077137
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD 7.1 BETA 2 vs Opensolaris vs Ubuntu performance

O. Hartmann-4
In reply to this post by Ivan Voras
Ivan Voras wrote:
...

>
> OTOH if the goal is to measure "operating system" performance, this
> must also include the compiler, libraries and all. (for example, what
> does Solaris default to nowadays? I think it ships with gcc but not as
> default). The hold on gcc 4.3 in FreeBSD is, after all, political
> (licencing).

This is very bad to read :-(
Many of my colleaugues are involved in HPC, very little of them
(including myself) utilizing FreeBSD even due to the lack of fast
compilers. Yes, we all can use the port, that is right, but for those
not so familiar and deep inside the underlying OS, with newer, better
hardware (CPUs with some interesting hardware features like SSE3/4) a
on-track-following compiler like GCC 4.3 could make use of special
features introduced in newer hardware and even due to better
optimizations compile a faster OS. And the result, even in 3% or 5%
performance gain is appreciated if model-runs taking days or weeks!


Regards,
Oliver
>
> If FreeBSD base ever switches to LLVM+clang, this means libc will be
> compiled with a non-gcc compiler which will forever change the
> performance for simple "real world" benchmarks.
> _______________________________________________
> [hidden email] mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-performance
> To unsubscribe, send any mail to "[hidden email]"
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD 7.1 BETA 2 vs Opensolaris vs Ubuntu performance

Ivan Voras
O. Hartmann wrote:

> Ivan Voras wrote:
> ...
>
>>
>> OTOH if the goal is to measure "operating system" performance, this
>> must also include the compiler, libraries and all. (for example, what
>> does Solaris default to nowadays? I think it ships with gcc but not as
>> default). The hold on gcc 4.3 in FreeBSD is, after all, political
>> (licencing).
>
> This is very bad to read :-(
I agree. GPL 3 is a bit hard on the non-GPL systems (i.e. harder than
GPL 2).

> Many of my colleaugues are involved in HPC, very little of them
> (including myself) utilizing FreeBSD even due to the lack of fast
> compilers. Yes, we all can use the port, that is right, but for those
> not so familiar and deep inside the underlying OS, with newer, better
> hardware (CPUs with some interesting hardware features like SSE3/4) a
> on-track-following compiler like GCC 4.3 could make use of special
> features introduced in newer hardware and even due to better
> optimizations compile a faster OS. And the result, even in 3% or 5%
> performance gain is appreciated if model-runs taking days or weeks!

AFAIK, gcc 4.3+ will always be available in the ports so users that need
it will always have it available (it's available there now!). It's just
that the base compiler will either stay 4.2, switch to something else
(Roman Divacky is working on LLVM+clang), or bite the bullet (with
possible workarounds for undesireable parts of GPL3) and switch to a
newer gcc.


signature.asc (260 bytes) Download Attachment
12