Phoronix comparision of HAMMER, UFS, ZFS, EXT3, EXT4, Btrfs

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

Phoronix comparision of HAMMER, UFS, ZFS, EXT3, EXT4, Btrfs

Miroslav Lachman
Another filesystem benchmark from Phoronix. This time comparing HAMMER,
UFS, ZFS, EXT3, EXT4 and Btrfs on DragonFly BSD, PC-BSD and Ubuntu.

http://www.phoronix.com/scan.php?page=article&item=dragonfly_hammer

I think it is almost useless test if systems were crippled to UP,
because of bad SMP performance of DragonFly BSD.

citation: "...the SMP performance under our setup was actually much
slower than with its UP kernel. As a result, we used the stock
DragonFlyBSD UP kernel and when benchmarking PC-BSD and Ubuntu we
disabled the SMP support there."

Miroslav Lachman
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Phoronix comparision of HAMMER, UFS, ZFS, EXT3, EXT4, Btrfs

pathiaki2
This is almost laughable.  I'd like to know what parameters they were tuning.  I
used FreeBSD with ZFS to make a point to people using Debian on EXT3, EXT4, XFS
just two years ago.  They were interested in total throughput and TPS.  Well, I
used the SAME MACHINE and rebuilt it from scratch with the same parameters
except the filesystems and the last time I changed the OS to FreeBSD with ZFS.

It was on a SAS drive using JBOD.  It was an HP 1xx series box.  The transaction
size was 4K.  I believed this was going to skew things heavily in Debian's
favor.  The machine had a dual core intel processor, an LSI controller card and
about 4 GB of RAM.  The OS and the test data were on separate drives.  Total
data written and read was 20 GB in sequential.

The results came out like this:

EXT3 - ~3000 tps
EXT4 - ~3800 tps
XFS - ~ 1800 tps
ZFS - 75000 tps

This benchmark ran flatout and I made 4 runs of each and took the average of the
slowest 2 of the bunch. (I do this so as not to get caught with my pants down on
real world performance and for best practices as we all know that due to heat
and resistance, things get slower until full operating temperature is reached.  
The engineers who were developing the software, gave me their benchmark to run.  
Their target was 15,000 tps and they were struggling.  I asked to see their
performance testing.  They had done none!  Also, they had done their entire
product development with out a systems architect or Sr. Systems Administrator in
the mix.

They later on created a new benchmark to exonerate themselves and it performed
random access reads, writes and deletes.  The ratio where cut by about 1/2 on
Debian and about 1/3 on ZFS.  I performed little tuning on anything.  I wanted
to see it all straight of the box.  All SMP and all 64 bit OS.

Paul Pathiakis
Systems Architect/Sr Admin/Geek
All around nice guy.





________________________________
From: Miroslav Lachman <[hidden email]>
To: [hidden email]
Sent: Fri, January 7, 2011 6:41:07 AM
Subject: Phoronix comparision of HAMMER, UFS, ZFS, EXT3, EXT4, Btrfs

Another filesystem benchmark from Phoronix. This time comparing HAMMER, UFS,
ZFS, EXT3, EXT4 and Btrfs on DragonFly BSD, PC-BSD and Ubuntu.

http://www.phoronix.com/scan.php?page=article&item=dragonfly_hammer

I think it is almost useless test if systems were crippled to UP, because of bad
SMP performance of DragonFly BSD.

citation: "...the SMP performance under our setup was actually much slower than
with its UP kernel. As a result, we used the stock DragonFlyBSD UP kernel and
when benchmarking PC-BSD and Ubuntu we disabled the SMP support there."

Miroslav Lachman
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"



     
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Phoronix comparision of HAMMER, UFS, ZFS, EXT3, EXT4, Btrfs

Christopher J. Ruwe-2
On Fri, 7 Jan 2011 06:12:38 -0800 (PST)
Paul Pathiakis <[hidden email]> wrote:

> This is almost laughable.  I'd like to know what parameters they were
> tuning.  

Some time ago Phoronix compared a FreeBSD with kernel debugging turned
on to an Ubuntu to show that FreeBSD is slow and Linuxes way faster.
Since then I have privately dubbed that site Moronix.

--
Christopher J. Ruwe
TZ GMT + 1

signature.asc (851 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Phoronix comparision of HAMMER, UFS, ZFS, EXT3, EXT4, Btrfs

Kevin Wilcox-3
In reply to this post by pathiaki2
On 7 January 2011 09:12, Paul Pathiakis <[hidden email]> wrote:

> This is almost laughable.  I'd like to know what parameters they were tuning.  I
> used FreeBSD with ZFS to make a point to people using Debian on EXT3, EXT4, XFS
> just two years ago.  They were interested in total throughput and TPS.  Well, I
> used the SAME MACHINE and rebuilt it from scratch with the same parameters
> except the filesystems and the last time I changed the OS to FreeBSD with ZFS.

Paul - Phoronix has a history of "performance benchmarks" that skew
*heavily* towards Linux and away from the BSDs, which is to say their
"benchmarks" are hardly indicative of Real Life.  I know I'm in for a
laugh any time I see someone reference one of their "performance
tests" regarding Linux/BSD.

Sadly, a lot of folks will look at that and say, "See? An independent,
non-biased review..."

kmw
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Phoronix comparision of HAMMER, UFS, ZFS, EXT3, EXT4, Btrfs

Bruce Cran
In reply to this post by Christopher J. Ruwe-2
On Fri, 7 Jan 2011 15:39:00 +0100
"Christopher J. Ruwe" <[hidden email]> wrote:

> Some time ago Phoronix compared a FreeBSD with kernel debugging turned
> on to an Ubuntu to show that FreeBSD is slow and Linuxes way faster.
> Since then I have privately dubbed that site Moronix.

People seem to forget that debugging is turned off before the RC builds
are done, which is what Phoronix tested (8.0 RC1).

--
Bruce Cran
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Phoronix comparision of HAMMER, UFS, ZFS, EXT3, EXT4, Btrfs

Stefan Lambrev-2
In reply to this post by pathiaki2
Hi,

Having in mind that a SAS enterprise disk normally can handle 150-180IOPS, this benchmark is testing something else ;)
Well there is a one thing which is clear from almost every Phoronix benchmark - Linux is heavily optimized ... for unpacking the linux kernel :)

On Jan 7, 2011, at 4:12 PM, Paul Pathiakis wrote:

> The results came out like this:
>
> EXT3 - ~3000 tps
> EXT4 - ~3800 tps
> XFS - ~ 1800 tps
> ZFS - 75000 tps

--
Best Wishes,
Stefan Lambrev
ICQ# 24134177





_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Phoronix comparision of HAMMER, UFS, ZFS, EXT3, EXT4, Btrfs

pathiaki2
In reply to this post by Kevin Wilcox-3
Kevin,

Sadly, true.  They are a heavy Ubuntu supporting shop.  I always laugh at their
benchmarks.  However, I may try their benches on FreeBSD against different
tuning parameters on *BSDs, just to see where things are and see if there's any
improvement or degradation depending on various settings.  (I'm kind of happy
that Debian is realizing some of the shortcomings of the kernel and now have a
FreeBSD kernel to put in. )  I'm always shocked at how few people realize that
Cisco, Juniper, Yahoo, Nokia, Apple, NetApp, HP, Dell, etc (yeah, all those
small names) use BSD and not Linux as their development platform or as their
platform of choice.  (I'm still amazed that Google uses Linux but throws lots of
money at SoC and FreeBSD - that's curious to me.  Of course, where does everyone
get their IP stacks from including MS? :-) )  


After 22 years, I've realized just how much is lacking between people with Eng
and Comp Sci theorists and the "certification" people who have a degree in
liberal arts.  Someone tried to compare themselves to me recently and they were
of the latter.  I smiled and put a simple automata machine on the board with a
given input string and asked them what it meant.  Just a blank stare.  (I mean
no disrespect to those persons who have gone beyond their degrees and dug into
the nuts and bolts to learn this stuff - kudos to them as it's a harder row to
hoe without the basis.)  


BTW, that company I mentioned in my previous post, same thing.  Lots of people,
big egos, little ability.  People don't like it when you show up and, within two
months, learn their proprietary product and prove to them it will never work
after they spent 6 years and millions on development.  They trashed the project
a year after I left.  (Their theory was sound but they wanted to build
everything from scratch where most of it was already available as various FOSS
due to the fact not one was a sysadmin and no one knew about technology
available.  (Heck, when I want to know what's available, I go to ports and
perform a search.  Then, I evaluate what's there and what's applicable.  After
that, I google to see what else competes.  This is now the fourth company in as
many years that has these types of developers/issues.  I don't see how any of
them will ever be successful.)


I really have to get my own company up and running.  22 years of this,
especially the last 5, has left me very jaded.  I wonder if the FreeBSD
foundation will pay my salary and I can work on FreeBSD. (*sigh* it's nice to
dream)

PJP


________________________________
From: Kevin Wilcox <[hidden email]>
To: [hidden email]
Sent: Fri, January 7, 2011 9:40:15 AM
Subject: Re: Phoronix comparision of HAMMER, UFS, ZFS, EXT3, EXT4, Btrfs

On 7 January 2011 09:12, Paul Pathiakis <[hidden email]> wrote:

> This is almost laughable.  I'd like to know what parameters they were tuning.
> I
> used FreeBSD with ZFS to make a point to people using Debian on EXT3, EXT4,
XFS
> just two years ago.  They were interested in total throughput and TPS.  Well,
I
> used the SAME MACHINE and rebuilt it from scratch with the same parameters
> except the filesystems and the last time I changed the OS to FreeBSD with ZFS.

Paul - Phoronix has a history of "performance benchmarks" that skew
*heavily* towards Linux and away from the BSDs, which is to say their
"benchmarks" are hardly indicative of Real Life.  I know I'm in for a
laugh any time I see someone reference one of their "performance
tests" regarding Linux/BSD.

Sadly, a lot of folks will look at that and say, "See? An independent,
non-biased review..."

kmw
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"



     
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Phoronix comparision of HAMMER, UFS, ZFS, EXT3, EXT4, Btrfs

Mark Felder-4
In reply to this post by Bruce Cran
On Fri, 07 Jan 2011 09:19:30 -0600, Bruce Cran <[hidden email]> wrote:

> People seem to forget that debugging is turned off before the RC builds
> are done, which is what Phoronix tested (8.0 RC1).

The GENERIC kernel has DEBUG=-g enabled; perhaps this is what he is  
referring to?

They certainly have earned the title Moronix either way.


Regards,


Mark
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Phoronix comparision of HAMMER, UFS, ZFS, EXT3, EXT4, Btrfs

Bruce Cran
On Fri, 07 Jan 2011 10:19:05 -0600
"Mark Felder" <[hidden email]> wrote:

> On Fri, 07 Jan 2011 09:19:30 -0600, Bruce Cran <[hidden email]>
> wrote:
>
> > People seem to forget that debugging is turned off before the RC
> > builds are done, which is what Phoronix tested (8.0 RC1).
>
> The GENERIC kernel has DEBUG=-g enabled; perhaps this is what he is  
> referring to?

I believe he means WITNESS, INVARIANTS, malloc debugging etc. - I
remember the discussion at the time.

--
Bruce Cran
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Phoronix comparision of HAMMER, UFS, ZFS, EXT3, EXT4, Btrfs

Christopher J. Ruwe-2
In reply to this post by Mark Felder-4
On Fri, 07 Jan 2011 10:19:05 -0600
"Mark Felder" <[hidden email]> wrote:

> On Fri, 07 Jan 2011 09:19:30 -0600, Bruce Cran <[hidden email]>
> wrote:
>
> > People seem to forget that debugging is turned off before the RC
> > builds are done, which is what Phoronix tested (8.0 RC1).
>
> The GENERIC kernel has DEBUG=-g enabled; perhaps this is what he is  
> referring to?

To be fair, I did not remember correctly anymore, but now as you are
saying it, yes, it was an RC build, so I think that it was 8.0RC1. Was
some time ago, at least not very recently, so time matches also.

Regards,
--
Christopher J. Ruwe
TZ GMT + 1

signature.asc (851 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Phoronix comparision of HAMMER, UFS, ZFS, EXT3, EXT4, Btrfs

Dieter BSD
In reply to this post by Miroslav Lachman
Paul:
> EXT3 - ~3000 tps
> EXT4 - ~3800 tps
> XFS - ~ 1800 tps
> ZFS - 75000 tps

The result for FFS w/softdeps seems to have been lost in the mail.  :-(

Mark:
> They certainly have earned the title Moronix either way.

For me, getting a debugging flag wrong (assuming they did, there seems
to be some question) pales in comparison with "my adventure to the
Chernobyl Nuclear Disaster Site".

There's a reason I think of linux types as bird brains, and it isn't
just
because of Tux.


_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Phoronix comparision of HAMMER, UFS, ZFS, EXT3, EXT4, Btrfs

pathiaki2
I'm sorry.  I didn't run it.  I should have but a journal was required.  I could
have used FFS w GEOM Journal but I was winging this in my free time while
performing my "job".  (Bored easily, and easily interested when something just
"feels" wrong)  So, I went with ZFS due to the gobs of memory on the machine.

Paul




________________________________
From: "[hidden email]" <[hidden email]>
To: [hidden email]
Sent: Fri, January 7, 2011 7:33:39 PM
Subject: Re: Phoronix comparision of HAMMER, UFS, ZFS, EXT3, EXT4, Btrfs

Paul:
> EXT3 - ~3000 tps
> EXT4 - ~3800 tps
> XFS - ~ 1800 tps
> ZFS - 75000 tps

The result for FFS w/softdeps seems to have been lost in the mail.  :-(

Mark:
> They certainly have earned the title Moronix either way.

For me, getting a debugging flag wrong (assuming they did, there seems
to be some question) pales in comparison with "my adventure to the
Chernobyl Nuclear Disaster Site".

There's a reason I think of linux types as bird brains, and it isn't just
because of Tux.


_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"



     
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Phoronix comparision of HAMMER, UFS, ZFS, EXT3, EXT4, Btrfs

Bruce Cran
In reply to this post by pathiaki2
On Fri, 7 Jan 2011 06:12:38 -0800 (PST)
Paul Pathiakis <[hidden email]> wrote:
> The results came out like this:
>
> EXT3 - ~3000 tps
> EXT4 - ~3800 tps
> XFS - ~ 1800 tps
> ZFS - 75000 tps

ZFS seems very good at keeping the disk busy with lots of buffering - on
my machine gstat shows the disk at 100% for several seconds even after
the application has finished. Despite seeing iops go as high as 65k the
average seems not so impressive at around 15k, though it is only on a
single SATA drive.

--
Bruce Cran
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Phoronix comparision of HAMMER, UFS, ZFS, EXT3, EXT4, Btrfs

Ivan Voras
In reply to this post by Stefan Lambrev-2
On 07/01/2011 16:23, Stefan Lambrev wrote:
> Hi,
>
> Having in mind that a SAS enterprise disk normally can handle 150-180IOPS, this benchmark is testing something else ;)

It depends - since ZFS is logging all the time it doesn't have to seek
as much; if all transactions are WRITE and given sequentially, they will
be written to the drive sequentially, even with full fsync semantics.
But 75k IOPS is a bit too much :)


_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Phoronix comparision of HAMMER, UFS, ZFS, EXT3, EXT4, Btrfs

Bruce Cran
On Mon, 10 Jan 2011 13:49:08 +0100
Ivan Voras <[hidden email]> wrote:

> It depends - since ZFS is logging all the time it doesn't have to
> seek as much; if all transactions are WRITE and given sequentially,
> they will be written to the drive sequentially, even with full fsync
> semantics. But 75k IOPS is a bit too much :)

I've been doing some benchmarking using sysutils/fio recently. It seems
that for my desktop SATA disk (a Samsung F3) around 28-30k iops is about
the maximum, seen both on Windows 7 (NTFS) and FreeBSD (ZFS).
FreeBSD is much more bursty compared to Windows, getting 80k iops and
210MB/s for a few seconds followed by several of 0.

--
Bruce Cran
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Phoronix comparision of HAMMER, UFS, ZFS, EXT3, EXT4, Btrfs

Ivan Voras
On 10/01/2011 14:07, Bruce Cran wrote:

> On Mon, 10 Jan 2011 13:49:08 +0100
> Ivan Voras<[hidden email]>  wrote:
>
>> It depends - since ZFS is logging all the time it doesn't have to
>> seek as much; if all transactions are WRITE and given sequentially,
>> they will be written to the drive sequentially, even with full fsync
>> semantics. But 75k IOPS is a bit too much :)
>
> I've been doing some benchmarking using sysutils/fio recently. It seems
> that for my desktop SATA disk (a Samsung F3) around 28-30k iops is about
> the maximum, seen both on Windows 7 (NTFS) and FreeBSD (ZFS).
> FreeBSD is much more bursty compared to Windows, getting 80k iops and
> 210MB/s for a few seconds followed by several of 0.

I've also noticed it is bursty - this can be moderated by tuning
vfs.zfs.txg.timeout and vfs.zfs.vdev.max_pending. But I think you must
agree that 210 MB/s on a single drive looks impossible :) I get that
much in a SAS RAID-10 configuration.

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Phoronix comparision of HAMMER, UFS, ZFS, EXT3, EXT4, Btrfs

Bruce Cran
On Mon, 10 Jan 2011 14:12:32 +0100
Ivan Voras <[hidden email]> wrote:

> I've also noticed it is bursty - this can be moderated by tuning
> vfs.zfs.txg.timeout and vfs.zfs.vdev.max_pending. But I think you
> must agree that 210 MB/s on a single drive looks impossible :) I get
> that much in a SAS RAID-10 configuration.

Exactly: since the disk obviously can't write at 210MB/s (115 seems to
be about its maximum) ZFS is buffering the data and then has to spend
time flushing it to disk during which time it can't accept any new IO
requests.

--
Bruce Cran
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"