Re: FreeBSD bind performance in FreeBSD 7

classic Classic list List threaded Threaded
43 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD bind performance in FreeBSD 7

Kris Kennaway-3
Oliver Herold wrote:
> Hi,
>
> I saw this bind benchmarks just some minutes ago,
>
> http://new.isc.org/proj/dnsperf/OStest.html
>
> is this true for FreeBSD 7 (current state: RELENG_7/7.0R) too? Or is
> this something verified only for the state of development back in August
> 2007?

I have been trying to replicate this.  ISC have kindly given me access
to their test data but I am seeing Linux performing much slower than
FreeBSD with the same ISC workload.

   http://people.freebsd.org/~kris/scaling/bind-pt.png

Summary:

* FreeBSD 7.0-R with 4BSD scheduler has close to ideal scaling on this test.

* The drop above 6 threads is due to limitations within BIND.

* Linux 2.6.24 has about 35% lower performance than FreeBSD, which is
significantly at variance with the ISC results.  It also doesn't scale
above 3 CPUs.

* I am trying to understand what is different about the ISC
configuration but have not yet found the cause.  They were testing
2.6.20.7 so it is possible that there was a major regression before the
2.6.22 and .24 kernels I tested.  Or maybe something is broken with the
Intel gige driver in Linux (they were using broadcom hardware).  The
graph is showing performance over 10ge, but I get the same peak
performance over gige when I query from 2 clients (the client benchmark
is very sensitive to network latency so a single client is not enough to
saturate BIND over gige).

* 7.0 with ULE has a bug on this workload (actually to do with workloads
involving high interrupt rates).  It is fixed in 8.0.

* Changes we have in progress to improve UDP performance do not help
much with this particular workload (only about 5%), but with more
scalable applications we see 30-40% improvement.  e.g. NSD
(ports/dns/nsd) is a much faster and more scalable DNS server than BIND
(because it is better optimized for the smaller set of features it
supports).

Kris


_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD bind performance in FreeBSD 7

Chris-69
>
> * 7.0 with ULE has a bug on this workload (actually to do with workloads
> involving high interrupt rates).  It is fixed in 8.0.

Kris can you say anything more about interrupt workload bugs on ULE?
On all my 7.0 servers I now am using ULE even on the UP ones as it was
said there is slight improvements for UP also but all the machines can
get intterupts intensive, lots of high speed transfers using nic
interrupts.  In this scenario am I better of using 4BSD?

Chris
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD bind performance in FreeBSD 7

Kris Kennaway-3
Chris wrote:
>> * 7.0 with ULE has a bug on this workload (actually to do with workloads
>> involving high interrupt rates).  It is fixed in 8.0.
>
> Kris can you say anything more about interrupt workload bugs on ULE?
> On all my 7.0 servers I now am using ULE even on the UP ones as it was
> said there is slight improvements for UP also but all the machines can
> get intterupts intensive, lots of high speed transfers using nic
> interrupts.  In this scenario am I better of using 4BSD?

I can't say for sure, you would have to do measurements of your
throughput.  It probably won't matter on UP though.

Kris
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD bind performance in FreeBSD 7

O. Hartmann-4
In reply to this post by Kris Kennaway-3
Kris Kennaway wrote:
[SCHNIPP]
>
> * 7.0 with ULE has a bug on this workload (actually to do with workloads
> involving high interrupt rates).  It is fixed in 8.0.

will this patch also be available for 7.0?


Regards,
Oliver
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD bind performance in FreeBSD 7

Kris Kennaway-3
O. Hartmann wrote:
> Kris Kennaway wrote:
> [SCHNIPP]
>>
>> * 7.0 with ULE has a bug on this workload (actually to do with
>> workloads involving high interrupt rates).  It is fixed in 8.0.
>
> will this patch also be available for 7.0?

If you mean "will it be merged to RELENG_7", absolutely.  If you mean
"will an errata be released and merged to RELENG_7_0", that is up to the
release engineers.

Kris
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

RE: FreeBSD bind performance in FreeBSD 7

Ted Mittelstaedt
In reply to this post by Kris Kennaway-3


> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]]On Behalf Of Kris Kennaway
> Sent: Monday, February 25, 2008 12:18 PM
> To: Oliver Herold; [hidden email];
> [hidden email]
> Subject: Re: FreeBSD bind performance in FreeBSD 7
>
>
> Oliver Herold wrote:
> > Hi,
> >
> > I saw this bind benchmarks just some minutes ago,
> >
> > http://new.isc.org/proj/dnsperf/OStest.html
> >
> > is this true for FreeBSD 7 (current state: RELENG_7/7.0R) too? Or is
> > this something verified only for the state of development back in August
> > 2007?
>
> I have been trying to replicate this.  ISC have kindly given me access
> to their test data but I am seeing Linux performing much slower than
> FreeBSD with the same ISC workload.
>

Kris,

  Every couple years we go through this with ISC.  They come out with
a new version of BIND then claim that nothing other than Linux can
run it well.  I've seen this nonsense before and it's tiresome.

Incidentally, the query tool they used, queryperf, has been changed
to dnsperf.  Someone needs to look at that port - /usr/ports/dns/dnsperf -
as it has a build depend of bind9 - well bind 9.3.4 is part of 6.3-RELEASE
and I was rather irked when I ran the dnsperf port maker and the
maker stupidly began the process of downloading and building the
same version of BIND that I was already running on my server.

>
> * I am trying to understand what is different about the ISC
> configuration but have not yet found the cause.

It's called "Anti-FreeBSD bias".  You won't find anything.

> e.g. NSD
> (ports/dns/nsd) is a much faster and more scalable DNS server than BIND
> (because it is better optimized for the smaller set of features it
> supports).
>

When you make remarks like that it's no wonder ISC is in the business
of slamming FreeBSD.  People used to make the same claims about djbdns
but I noticed over the last few years they don't seem to be doing
that anymore.

If nsd is so much better than yank bind out of the base FreeBSD and
replace it with nsd.  Of course that will make more work for me
when I regen our nameservers here since nsd will be the first thing
on the "rm" list.

Ted

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD bind performance in FreeBSD 7

Predrag Punosevac
Ted Mittelstaedt wrote:

>  
>> -----Original Message-----
>> From: [hidden email]
>> [mailto:[hidden email]]On Behalf Of Kris Kennaway
>> Sent: Monday, February 25, 2008 12:18 PM
>> To: Oliver Herold; [hidden email];
>> [hidden email]
>> Subject: Re: FreeBSD bind performance in FreeBSD 7
>>
>>
>> Oliver Herold wrote:
>>    
>>> Hi,
>>>
>>> I saw this bind benchmarks just some minutes ago,
>>>
>>> http://new.isc.org/proj/dnsperf/OStest.html
>>>
>>> is this true for FreeBSD 7 (current state: RELENG_7/7.0R) too? Or is
>>> this something verified only for the state of development back in August
>>> 2007?
>>>      
>> I have been trying to replicate this.  ISC have kindly given me access
>> to their test data but I am seeing Linux performing much slower than
>> FreeBSD with the same ISC workload.
>>
>>    
>
> Kris,
>
>   Every couple years we go through this with ISC.  They come out with
> a new version of BIND then claim that nothing other than Linux can
> run it well.  I've seen this nonsense before and it's tiresome.
>
> Incidentally, the query tool they used, queryperf, has been changed
> to dnsperf.  Someone needs to look at that port - /usr/ports/dns/dnsperf -
> as it has a build depend of bind9 - well bind 9.3.4 is part of 6.3-RELEASE
> and I was rather irked when I ran the dnsperf port maker and the
> maker stupidly began the process of downloading and building the
> same version of BIND that I was already running on my server.
>
>  
>> * I am trying to understand what is different about the ISC
>> configuration but have not yet found the cause.
>>    
>
> It's called "Anti-FreeBSD bias".  You won't find anything.
>
>  
You just described the tests up to isomorphism in the terminology of
mathematics which is more familiar
subject to me :-)

The results of OpenBSD has been discussed and analyzed on the
misc.at.openbsd.org. Even to a hobbyist  like myself was not clear why
did they chose to test OpenBSD 4.1
when only in two month the stable version of OpenBSD will be 4.3. For
those unfamiliar performance of
OpenBSD 4.2 as a DNS server has been dramatically improved from the 4.1
version.
The  question  of multi-threading  (no-no in OpenBSD world) and its role
in above results was also analyzed.


>> e.g. NSD
>> (ports/dns/nsd) is a much faster and more scalable DNS server than BIND
>> (because it is better optimized for the smaller set of features it
>> supports).
>>
>>    
>
> When you make remarks like that it's no wonder ISC is in the business
> of slamming FreeBSD.  People used to make the same claims about djbdns
> but I noticed over the last few years they don't seem to be doing
> that anymore.
>
> If nsd is so much better than yank bind out of the base FreeBSD and
> replace it with nsd.  Of course that will make more work for me
> when I regen our nameservers here since nsd will be the first thing
> on the "rm" list.
>
>  
I sincerely hope for the above. Hopefully Ted finally can buy that
Mercedes to his wife which she deserves so much ;-) .

Cheers,
Predrag


> Ted
>
> _______________________________________________
> [hidden email] mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to "[hidden email]"
>  

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD bind performance in FreeBSD 7

Sam Leffler
In reply to this post by Ted Mittelstaedt
Ted Mittelstaedt wrote:

>  
>> -----Original Message-----
>> From: [hidden email]
>> [mailto:[hidden email]]On Behalf Of Kris Kennaway
>> Sent: Monday, February 25, 2008 12:18 PM
>> To: Oliver Herold; [hidden email];
>> [hidden email]
>> Subject: Re: FreeBSD bind performance in FreeBSD 7
>>
>>
>> Oliver Herold wrote:
>>    
>>> Hi,
>>>
>>> I saw this bind benchmarks just some minutes ago,
>>>
>>> http://new.isc.org/proj/dnsperf/OStest.html
>>>
>>> is this true for FreeBSD 7 (current state: RELENG_7/7.0R) too? Or is
>>> this something verified only for the state of development back in August
>>> 2007?
>>>      
>> I have been trying to replicate this.  ISC have kindly given me access
>> to their test data but I am seeing Linux performing much slower than
>> FreeBSD with the same ISC workload.
>>
>>    
>
> Kris,
>
>   Every couple years we go through this with ISC.  They come out with
> a new version of BIND then claim that nothing other than Linux can
> run it well.  I've seen this nonsense before and it's tiresome.
>
> Incidentally, the query tool they used, queryperf, has been changed
> to dnsperf.  Someone needs to look at that port - /usr/ports/dns/dnsperf -
> as it has a build depend of bind9 - well bind 9.3.4 is part of 6.3-RELEASE
> and I was rather irked when I ran the dnsperf port maker and the
> maker stupidly began the process of downloading and building the
> same version of BIND that I was already running on my server.
>
>  
>> * I am trying to understand what is different about the ISC
>> configuration but have not yet found the cause.
>>    
>
> It's called "Anti-FreeBSD bias".  You won't find anything.
>
>  
>> e.g. NSD
>> (ports/dns/nsd) is a much faster and more scalable DNS server than BIND
>> (because it is better optimized for the smaller set of features it
>> supports).
>>
>>    
>
> When you make remarks like that it's no wonder ISC is in the business
> of slamming FreeBSD.  People used to make the same claims about djbdns
> but I noticed over the last few years they don't seem to be doing
> that anymore.
>
> If nsd is so much better than yank bind out of the base FreeBSD and
> replace it with nsd.  Of course that will make more work for me
> when I regen our nameservers here since nsd will be the first thing
> on the "rm" list.
>  

Please save your rhetoric for some other forum.  The ISC folks have been
working with us to understand what's going on.  I'm not aware of any
anit-FreeBSD slams going on; mostly uninformed comments.

We believe FreeBSD does very well in any comparisons of the sort being
discussed and there's still lots of room for improvement.

As to nsd vs bind, understand they are very different applications w/
totally different goals.  Comparing performance is not entirely fair and
certainly is difficult.  Kris investigated the performance of nsd mostly
to understand how bind might scale if certain architectural changes were
made to eliminate known bottlenecks in the application.

    Sam
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD bind performance in FreeBSD 7

Adrian Chadd-2
(Sorry for top posting.)

Its not actually -that- bad an idea to compare different applications.
It sets the "bar" for how far the entire system {hardware, OS,
application, network} can be pushed.

If nsd beats bind9 by say 5 or 10% over all, then its nothing to write
home about. If nsd beats bind9 by 50% and shows similar
kernel/interrupt space time use then thats something to stare at. Even
if its just because nsd 'does less' and gives more CPU time to
system/interrupt processing you've identified that the system -can- be
pushed harder, and perhaps working with the bind9 guys a little more
can identify what they're doing wrong.

Thats how I noticed the performance differences between various
platforms running Squid a few years ago - for example, gettimeofday()
being called way, way too frequently - and I compare Squid's
kernel/interrupt time; syscall footprint; hwpmc/oprofile traces; etc
against other proxy-capable applications (varnish, lighttpd, apache)
to see exactly what they're doing differently.

2c,



adrian


On 28/02/2008, Sam Leffler <[hidden email]> wrote:

> Ted Mittelstaedt wrote:
>  >
>  >> -----Original Message-----
>  >> From: [hidden email]
>  >> [mailto:[hidden email]]On Behalf Of Kris Kennaway
>  >> Sent: Monday, February 25, 2008 12:18 PM
>  >> To: Oliver Herold; [hidden email];
>  >> [hidden email]
>  >> Subject: Re: FreeBSD bind performance in FreeBSD 7
>  >>
>  >>
>  >> Oliver Herold wrote:
>  >>
>  >>> Hi,
>  >>>
>  >>> I saw this bind benchmarks just some minutes ago,
>  >>>
>  >>> http://new.isc.org/proj/dnsperf/OStest.html
>  >>>
>  >>> is this true for FreeBSD 7 (current state: RELENG_7/7.0R) too? Or is
>  >>> this something verified only for the state of development back in August
>  >>> 2007?
>  >>>
>  >> I have been trying to replicate this.  ISC have kindly given me access
>  >> to their test data but I am seeing Linux performing much slower than
>  >> FreeBSD with the same ISC workload.
>  >>
>  >>
>  >
>  > Kris,
>  >
>  >   Every couple years we go through this with ISC.  They come out with
>  > a new version of BIND then claim that nothing other than Linux can
>  > run it well.  I've seen this nonsense before and it's tiresome.
>  >
>  > Incidentally, the query tool they used, queryperf, has been changed
>  > to dnsperf.  Someone needs to look at that port - /usr/ports/dns/dnsperf -
>  > as it has a build depend of bind9 - well bind 9.3.4 is part of 6.3-RELEASE
>  > and I was rather irked when I ran the dnsperf port maker and the
>  > maker stupidly began the process of downloading and building the
>  > same version of BIND that I was already running on my server.
>  >
>  >
>  >> * I am trying to understand what is different about the ISC
>  >> configuration but have not yet found the cause.
>  >>
>  >
>  > It's called "Anti-FreeBSD bias".  You won't find anything.
>  >
>  >
>  >> e.g. NSD
>  >> (ports/dns/nsd) is a much faster and more scalable DNS server than BIND
>  >> (because it is better optimized for the smaller set of features it
>  >> supports).
>  >>
>  >>
>  >
>  > When you make remarks like that it's no wonder ISC is in the business
>  > of slamming FreeBSD.  People used to make the same claims about djbdns
>  > but I noticed over the last few years they don't seem to be doing
>  > that anymore.
>  >
>  > If nsd is so much better than yank bind out of the base FreeBSD and
>  > replace it with nsd.  Of course that will make more work for me
>  > when I regen our nameservers here since nsd will be the first thing
>  > on the "rm" list.
>  >
>
>
> Please save your rhetoric for some other forum.  The ISC folks have been
>  working with us to understand what's going on.  I'm not aware of any
>  anit-FreeBSD slams going on; mostly uninformed comments.
>
>  We believe FreeBSD does very well in any comparisons of the sort being
>  discussed and there's still lots of room for improvement.
>
>  As to nsd vs bind, understand they are very different applications w/
>  totally different goals.  Comparing performance is not entirely fair and
>  certainly is difficult.  Kris investigated the performance of nsd mostly
>  to understand how bind might scale if certain architectural changes were
>  made to eliminate known bottlenecks in the application.
>
>
>     Sam
>
> _______________________________________________
>  [hidden email] mailing list
>  http://lists.freebsd.org/mailman/listinfo/freebsd-performance
>  To unsubscribe, send any mail to "[hidden email]"
>


--
Adrian Chadd - [hidden email]
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD bind performance in FreeBSD 7

Kris Kennaway-3
Adrian Chadd wrote:

> (Sorry for top posting.)
>
> Its not actually -that- bad an idea to compare different applications.
> It sets the "bar" for how far the entire system {hardware, OS,
> application, network} can be pushed.
>
> If nsd beats bind9 by say 5 or 10% over all, then its nothing to write
> home about. If nsd beats bind9 by 50% and shows similar
> kernel/interrupt space time use then thats something to stare at. Even
> if its just because nsd 'does less' and gives more CPU time to
> system/interrupt processing you've identified that the system -can- be
> pushed harder, and perhaps working with the bind9 guys a little more
> can identify what they're doing wrong.
>
> Thats how I noticed the performance differences between various
> platforms running Squid a few years ago - for example, gettimeofday()
> being called way, way too frequently - and I compare Squid's
> kernel/interrupt time; syscall footprint; hwpmc/oprofile traces; etc
> against other proxy-capable applications (varnish, lighttpd, apache)
> to see exactly what they're doing differently.

Yep, and in this case NSD is currently 90% faster with prospects to push
it even higher with some further kernel changes (so far we have improved
it by 45%).  BIND is limited by its own architecture, so improvements
cannot be made by modifying the kernel.

Anyway, the motivation here is not a DNS deathmatch, but part of our
ongoing effort to look for aspects of FreeBSD performance that can be
improved.  Currently we are looking at UDP performance, and DNS serving
was thought to be a good model for that.  It turns out that BIND does
not stress the kernel, but NSD does.

Kris

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

RE: FreeBSD bind performance in FreeBSD 7

Ted Mittelstaedt
In reply to this post by Sam Leffler


> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]]On Behalf Of Sam Leffler
> Sent: Wednesday, February 27, 2008 8:54 AM
> To: Ted Mittelstaedt
> Cc: [hidden email]; Kris Kennaway; Oliver Herold;
> [hidden email]
> Subject: Re: FreeBSD bind performance in FreeBSD 7
>
>
> Ted Mittelstaedt wrote:
> >
> >> -----Original Message-----
> >> From: [hidden email]
> >> [mailto:[hidden email]]On Behalf Of Kris Kennaway
> >> Sent: Monday, February 25, 2008 12:18 PM
> >> To: Oliver Herold; [hidden email];
> >> [hidden email]
> >> Subject: Re: FreeBSD bind performance in FreeBSD 7
> >>
> >>
> >> Oliver Herold wrote:
> >>
> >>> Hi,
> >>>
> >>> I saw this bind benchmarks just some minutes ago,
> >>>
> >>> http://new.isc.org/proj/dnsperf/OStest.html
> >>>
> >>> is this true for FreeBSD 7 (current state: RELENG_7/7.0R) too? Or is
> >>> this something verified only for the state of development
> back in August
> >>> 2007?
> >>>
> >> I have been trying to replicate this.  ISC have kindly given me access
> >> to their test data but I am seeing Linux performing much slower than
> >> FreeBSD with the same ISC workload.
> >>
> >>
> >
> > Kris,
> >
> >   Every couple years we go through this with ISC.  They come out with
> > a new version of BIND then claim that nothing other than Linux can
> > run it well.  I've seen this nonsense before and it's tiresome.
> >
> > Incidentally, the query tool they used, queryperf, has been changed
> > to dnsperf.  Someone needs to look at that port -
> /usr/ports/dns/dnsperf -
> > as it has a build depend of bind9 - well bind 9.3.4 is part of
> 6.3-RELEASE
> > and I was rather irked when I ran the dnsperf port maker and the
> > maker stupidly began the process of downloading and building the
> > same version of BIND that I was already running on my server.
> >
> >
> >> * I am trying to understand what is different about the ISC
> >> configuration but have not yet found the cause.
> >>
> >
> > It's called "Anti-FreeBSD bias".  You won't find anything.
> >
> >
> >> e.g. NSD
> >> (ports/dns/nsd) is a much faster and more scalable DNS server than BIND
> >> (because it is better optimized for the smaller set of features it
> >> supports).
> >>
> >>
> >
> > When you make remarks like that it's no wonder ISC is in the business
> > of slamming FreeBSD.  People used to make the same claims about djbdns
> > but I noticed over the last few years they don't seem to be doing
> > that anymore.
> >
> > If nsd is so much better than yank bind out of the base FreeBSD and
> > replace it with nsd.  Of course that will make more work for me
> > when I regen our nameservers here since nsd will be the first thing
> > on the "rm" list.
> >
>
> Please save your rhetoric for some other forum.  The ISC folks have been
> working with us to understand what's going on.

Did anyone try disabling the onboard NIC and put in an Intel
Pro/1000 in the PCI express slot in the server and retest with
both Linux and FreeBSD?  As I run Proliants for a living,
this stuck out to me like a sore thumb.  The onboard NIC
in the systems they used for the testbed is just shit.  Hell,
just about anything Broadcom makes is shit.  They even managed
to screw up the 3c905 ASIC when 3com switched to using them
as the supplier (from Lucent)( - I've watched those card versions
panic Linux systems and drop massive packets in FreeBSD,
when the Lucent-made chipped cards worked fine.

> I'm not aware of any
> anit-FreeBSD slams going on; mostly uninformed comments.
>

It's customary in the industry before publishing rather unflattering
results to call in the team in charge of the unflattering
product and give them a chance to verify that the tester
really knew what they were doing.

FreeBSD has got slammed a number of times in the past by
testers who didn't do this.  In fact as I recall the impetus
for fixing the
extended greater than 16MB memory test was due to a
slam in a trade rag from a tester who didn't bother
recompiling the FreeBSD kernel to recognize the complete
amount of ram in the server, and running it up against Linux.

Maybe I am wrong and the ISC team did in fact call you guys
in before publishing the results - but the wording of
the entire site (not just the test results) indicated
they did their testing and informed FreeBSD after the fact.
after publishing.  Not nice.

Ted

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Upgrade from 6.3 to 7.0: Result ->Shared object not found :)

Noisex
In reply to this post by Kris Kennaway-3
Hi! The first of all i want felicitate FreeBSD development team and all FBSD
community with 7.0 come-out.

        Ok, the question is not about performance, but anyway maybe you have
some ideas how to solve problem in short time - i don't want rebuld all
needed packages :)

This morning i decided to upgrade one of the my boxes from 6.3 to 7.0 using
http://www.daemonology.net/blog/2007-11-11-freebsd-major-version-upgrade.htm
l script not native freebsd-update().

Scenario:
# fetch http://people.freebsd.org/~cperciva/freebsd-update-upgrade.tgz
# fetch http://people.freebsd.org/~cperciva/freebsd-update-upgrade.tgz.asc
# gpg --verify freebsd-update-upgrade.tgz.asc freebsd-update-upgrade.tgz
# tar -xf freebsd-update-upgrade.tgz
# sh freebsd-update.sh -f freebsd-update.conf -r 7.0-RELEASE upgrade
# sh freebsd-update.sh -f freebsd-update.conf install
# shutdown -r now
# sh freebsd-update.sh -f freebsd-update.conf install
# portupgrade -faP
# sh freebsd-update.sh -f freebsd-update.conf install
# shutdown -r now


All goes fine till second reboot.

After reboot, i couldn't logon to remote box, because default shell is bash,
and i got error on SSH login:

/libexec/ld-elf.so.1: Shared object "libncurses.so.6" not found, required by
"-bash"

Thanx GOD, this box is with ILO management and i didnt need go to the data
center, so i connect through this one, reboot server to single-user mode,
mount all necessary partitions, changed default shell to csh...I have a look
to dmesg/messages and a roger that some services also didn't start up
because some of the shared libs are missing or something like that:

Checked some daemons depencies with ldd and libs like: libc, libm, libthr,
libcrypt, libcrypto are missing, or version is changed etc.

Question: what i did wrong in upgrade process? And how can i fix this errors
now i short time, because portupgrade now also not working because ruby
depencies/libs is missing and go on problems.

Manualy rebuild every all packages is pain in the ass on Friday night :D

Cheers, Noisex



_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Upgrade from 6.3 to 7.0: Result ->Shared object not found :)

Stefan Lambrev-2
Greetings,

Noisex wrote:
> Hi! The first of all i want felicitate FreeBSD development team and all FBSD
> community with 7.0 come-out.
>
> Ok, the question is not about performance, but anyway maybe you have
> some ideas how to solve problem in short time - i don't want rebuld all
> needed packages :)
>  
Yes -stable or -questions will be more proper mail list to ask this :)
If you will not update any ports/packages you should install compat6x
and then you will be able to run
bash compiled for 6.X.
The upgrade procedure is discussed every time when new major version is
released, so you can read
this thread -
http://lists.freebsd.org/pipermail/freebsd-stable/2008-February/040809.html
There is very good explanation what are the problems if you do not
reinstall all of your ports/packages.

> This morning i decided to upgrade one of the my boxes from 6.3 to 7.0 using
> http://www.daemonology.net/blog/2007-11-11-freebsd-major-version-upgrade.htm
> l script not native freebsd-update().
>
> Scenario:
> # fetch http://people.freebsd.org/~cperciva/freebsd-update-upgrade.tgz
> # fetch http://people.freebsd.org/~cperciva/freebsd-update-upgrade.tgz.asc
> # gpg --verify freebsd-update-upgrade.tgz.asc freebsd-update-upgrade.tgz
> # tar -xf freebsd-update-upgrade.tgz
> # sh freebsd-update.sh -f freebsd-update.conf -r 7.0-RELEASE upgrade
> # sh freebsd-update.sh -f freebsd-update.conf install
> # shutdown -r now
> # sh freebsd-update.sh -f freebsd-update.conf install
> # portupgrade -faP
> # sh freebsd-update.sh -f freebsd-update.conf install
> # shutdown -r now
>
>
> All goes fine till second reboot.
>
> After reboot, i couldn't logon to remote box, because default shell is bash,
> and i got error on SSH login:
>
> /libexec/ld-elf.so.1: Shared object "libncurses.so.6" not found, required by
> "-bash"
>
> Thanx GOD, this box is with ILO management and i didnt need go to the data
>  
But if you follow documentation you will see that you need access to the
server - via serial
or local, because you have to run mergemaster and installworld in single
user mode.
So you can't blame anybody just because you try to upgrade without
following the instructions?

> center, so i connect through this one, reboot server to single-user mode,
> mount all necessary partitions, changed default shell to csh...I have a look
> to dmesg/messages and a roger that some services also didn't start up
> because some of the shared libs are missing or something like that:
>
> Checked some daemons depencies with ldd and libs like: libc, libm, libthr,
> libcrypt, libcrypto are missing, or version is changed etc.
>
> Question: what i did wrong in upgrade process? And how can i fix this errors
> now i short time, because portupgrade now also not working because ruby
> depencies/libs is missing and go on problems.
>
> Manualy rebuild every all packages is pain in the ass on Friday night :D
>  
compat6x can help you to run the apps and to postpone the upgrade of
everything for Monday morning ;)

--

Best Wishes,
Stefan Lambrev
ICQ# 24134177

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Upgrade from 6.3 to 7.0: Result ->Shared object not found :)

Kris Kennaway-3
In reply to this post by Noisex
Noisex wrote:

> Hi! The first of all i want felicitate FreeBSD development team and all FBSD
> community with 7.0 come-out.
>
> Ok, the question is not about performance, but anyway maybe you have
> some ideas how to solve problem in short time - i don't want rebuld all
> needed packages :)
>
> This morning i decided to upgrade one of the my boxes from 6.3 to 7.0 using
> http://www.daemonology.net/blog/2007-11-11-freebsd-major-version-upgrade.htm
> l script not native freebsd-update().
>
> Scenario:
> # fetch http://people.freebsd.org/~cperciva/freebsd-update-upgrade.tgz
> # fetch http://people.freebsd.org/~cperciva/freebsd-update-upgrade.tgz.asc
> # gpg --verify freebsd-update-upgrade.tgz.asc freebsd-update-upgrade.tgz
> # tar -xf freebsd-update-upgrade.tgz
> # sh freebsd-update.sh -f freebsd-update.conf -r 7.0-RELEASE upgrade
> # sh freebsd-update.sh -f freebsd-update.conf install
> # shutdown -r now
> # sh freebsd-update.sh -f freebsd-update.conf install
> # portupgrade -faP
> # sh freebsd-update.sh -f freebsd-update.conf install
> # shutdown -r now
>
>
> All goes fine till second reboot.
>
> After reboot, i couldn't logon to remote box, because default shell is bash,
> and i got error on SSH login:
>
> /libexec/ld-elf.so.1: Shared object "libncurses.so.6" not found, required by
> "-bash"

If the portupgrade -faP completed successfully and your bash was
installed from ports then it should have been rebuilt.  If you have
non-port 6.x software installed then you need to add the compat6x port
to keep it running.

Kris

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

RE: Upgrade from 6.3 to 7.0: Result ->Shared object not found :)

Noisex
Kris, as i said, i made portupgrade -faP ...and in update process i saw that
many ports upgraded. I don't why old version become as current version. Now
i've installed compat6x (thanx Stefan Lambrev) -> it's works...but anyway
now i will try fix pkgdb and reinstall all necessary ports.

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Kris Kennaway
Sent: Friday, February 29, 2008 4:33 PM
To: Noisex
Cc: [hidden email]
Subject: Re: Upgrade from 6.3 to 7.0: Result ->Shared object not found :)

Noisex wrote:
> Hi! The first of all i want felicitate FreeBSD development team and all
FBSD
> community with 7.0 come-out.
>
> Ok, the question is not about performance, but anyway maybe you have
> some ideas how to solve problem in short time - i don't want rebuld all
> needed packages :)
>
> This morning i decided to upgrade one of the my boxes from 6.3 to 7.0
using
>
http://www.daemonology.net/blog/2007-11-11-freebsd-major-version-upgrade.htm

> l script not native freebsd-update().
>
> Scenario:
> # fetch http://people.freebsd.org/~cperciva/freebsd-update-upgrade.tgz
> # fetch http://people.freebsd.org/~cperciva/freebsd-update-upgrade.tgz.asc
> # gpg --verify freebsd-update-upgrade.tgz.asc freebsd-update-upgrade.tgz
> # tar -xf freebsd-update-upgrade.tgz
> # sh freebsd-update.sh -f freebsd-update.conf -r 7.0-RELEASE upgrade
> # sh freebsd-update.sh -f freebsd-update.conf install
> # shutdown -r now
> # sh freebsd-update.sh -f freebsd-update.conf install
> # portupgrade -faP
> # sh freebsd-update.sh -f freebsd-update.conf install
> # shutdown -r now
>
>
> All goes fine till second reboot.
>
> After reboot, i couldn't logon to remote box, because default shell is
bash,
> and i got error on SSH login:
>
> /libexec/ld-elf.so.1: Shared object "libncurses.so.6" not found, required
by
> "-bash"

If the portupgrade -faP completed successfully and your bash was
installed from ports then it should have been rebuilt.  If you have
non-port 6.x software installed then you need to add the compat6x port
to keep it running.

Kris

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to
"[hidden email]"

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Upgrade from 6.3 to 7.0: Result ->Shared object not found :)

Kris Kennaway-3
Noisex wrote:
> Kris, as i said, i made portupgrade -faP ...and in update process i saw that
> many ports upgraded. I don't why old version become as current version. Now
> i've installed compat6x (thanx Stefan Lambrev) -> it's works...but anyway
> now i will try fix pkgdb and reinstall all necessary ports.

The unspoken part of running portupgrade -faP is checking the output
summary to verify that it completed correctly.

Did you confirm that no packages failed to rebuild?  I'm really not
seeing how portupgrade could have rebuilt it successfully and yet it was
still the old 6.x version.

Kris

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

RE: Upgrade from 6.3 to 7.0: Result ->Shared object not found :)

Noisex
I don't remember during portupgrade process was some kind of errors - how
much i saw of course. It's bad that my SSH client (SecureCRT) is configured
with only 1024 row buffer - i can't scroll it back, or i didn't log upgrade
process to file...so i can't be sure to 100% that there wasn't errors :(

However that may be now i getting all functionality back step by step :)

Noisex

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Kris Kennaway
Sent: Friday, February 29, 2008 5:11 PM
To: Noisex
Cc: [hidden email]
Subject: Re: Upgrade from 6.3 to 7.0: Result ->Shared object not found :)

Noisex wrote:
> Kris, as i said, i made portupgrade -faP ...and in update process i saw
that
> many ports upgraded. I don't why old version become as current version.
Now
> i've installed compat6x (thanx Stefan Lambrev) -> it's works...but anyway
> now i will try fix pkgdb and reinstall all necessary ports.

The unspoken part of running portupgrade -faP is checking the output
summary to verify that it completed correctly.

Did you confirm that no packages failed to rebuild?  I'm really not
seeing how portupgrade could have rebuilt it successfully and yet it was
still the old 6.x version.

Kris

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to
"[hidden email]"

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Upgrade from 6.3 to 7.0: Result ->Shared object not found :)

Stefan Lambrev-2
Greetings,

Noisex wrote:
> I don't remember during portupgrade process was some kind of errors - how
> much i saw of course. It's bad that my SSH client (SecureCRT) is configured
> with only 1024 row buffer - i can't scroll it back, or i didn't log upgrade
> process to file...so i can't be sure to 100% that there wasn't errors :(
>  
portupgrade shows summary at the end which is less then 1024 lines ;)

Also you can use script(1) to log, so you can analyze the output latter.
I'm not sure how script will run, if it is started in screen
(ports/sysutils/screen)
which I always use when I do long upgrades over the network, but you can
try for yourself :)
> However that may be now i getting all functionality back step by step :)
>  
Yes but next time when you update port/package you can be bitten badly
if things are not in sync.

--

Best Wishes,
Stefan Lambrev
ICQ# 24134177

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD bind performance in FreeBSD 7

Chris-69
In reply to this post by Ted Mittelstaedt
On 29/02/2008, Ted Mittelstaedt <[hidden email]> wrote:

>
>
> > -----Original Message-----
> > From: [hidden email]
> > [mailto:[hidden email]]On Behalf Of Sam Leffler
> > Sent: Wednesday, February 27, 2008 8:54 AM
> > To: Ted Mittelstaedt
> > Cc: [hidden email]; Kris Kennaway; Oliver Herold;
> > [hidden email]
> > Subject: Re: FreeBSD bind performance in FreeBSD 7
> >
> >
> > Ted Mittelstaedt wrote:
> > >
> > >> -----Original Message-----
> > >> From: [hidden email]
> > >> [mailto:[hidden email]]On Behalf Of Kris Kennaway
> > >> Sent: Monday, February 25, 2008 12:18 PM
> > >> To: Oliver Herold; [hidden email];
> > >> [hidden email]
> > >> Subject: Re: FreeBSD bind performance in FreeBSD 7
> > >>
> > >>
> > >> Oliver Herold wrote:
> > >>
> > >>> Hi,
> > >>>
> > >>> I saw this bind benchmarks just some minutes ago,
> > >>>
> > >>> http://new.isc.org/proj/dnsperf/OStest.html
> > >>>
> > >>> is this true for FreeBSD 7 (current state: RELENG_7/7.0R) too? Or is
> > >>> this something verified only for the state of development
> > back in August
> > >>> 2007?
> > >>>
> > >> I have been trying to replicate this.  ISC have kindly given me access
> > >> to their test data but I am seeing Linux performing much slower than
> > >> FreeBSD with the same ISC workload.
> > >>
> > >>
> > >
> > > Kris,
> > >
> > >   Every couple years we go through this with ISC.  They come out with
> > > a new version of BIND then claim that nothing other than Linux can
> > > run it well.  I've seen this nonsense before and it's tiresome.
> > >
> > > Incidentally, the query tool they used, queryperf, has been changed
> > > to dnsperf.  Someone needs to look at that port -
> > /usr/ports/dns/dnsperf -
> > > as it has a build depend of bind9 - well bind 9.3.4 is part of
> > 6.3-RELEASE
> > > and I was rather irked when I ran the dnsperf port maker and the
> > > maker stupidly began the process of downloading and building the
> > > same version of BIND that I was already running on my server.
> > >
> > >
> > >> * I am trying to understand what is different about the ISC
> > >> configuration but have not yet found the cause.
> > >>
> > >
> > > It's called "Anti-FreeBSD bias".  You won't find anything.
> > >
> > >
> > >> e.g. NSD
> > >> (ports/dns/nsd) is a much faster and more scalable DNS server than BIND
> > >> (because it is better optimized for the smaller set of features it
> > >> supports).
> > >>
> > >>
> > >
> > > When you make remarks like that it's no wonder ISC is in the business
> > > of slamming FreeBSD.  People used to make the same claims about djbdns
> > > but I noticed over the last few years they don't seem to be doing
> > > that anymore.
> > >
> > > If nsd is so much better than yank bind out of the base FreeBSD and
> > > replace it with nsd.  Of course that will make more work for me
> > > when I regen our nameservers here since nsd will be the first thing
> > > on the "rm" list.
> > >
> >
> > Please save your rhetoric for some other forum.  The ISC folks have been
> > working with us to understand what's going on.
>
> Did anyone try disabling the onboard NIC and put in an Intel
> Pro/1000 in the PCI express slot in the server and retest with
> both Linux and FreeBSD?  As I run Proliants for a living,
> this stuck out to me like a sore thumb.  The onboard NIC
> in the systems they used for the testbed is just shit.  Hell,
> just about anything Broadcom makes is shit.  They even managed
> to screw up the 3c905 ASIC when 3com switched to using them
> as the supplier (from Lucent)( - I've watched those card versions
> panic Linux systems and drop massive packets in FreeBSD,
> when the Lucent-made chipped cards worked fine.
>
> > I'm not aware of any
> > anit-FreeBSD slams going on; mostly uninformed comments.
> >
>
> It's customary in the industry before publishing rather unflattering
> results to call in the team in charge of the unflattering
> product and give them a chance to verify that the tester
> really knew what they were doing.
>
> FreeBSD has got slammed a number of times in the past by
> testers who didn't do this.  In fact as I recall the impetus
> for fixing the
> extended greater than 16MB memory test was due to a
> slam in a trade rag from a tester who didn't bother
> recompiling the FreeBSD kernel to recognize the complete
> amount of ram in the server, and running it up against Linux.
>
> Maybe I am wrong and the ISC team did in fact call you guys
> in before publishing the results - but the wording of
> the entire site (not just the test results) indicated
> they did their testing and informed FreeBSD after the fact.
> after publishing.  Not nice.
>
> Ted
>

A weakness of freebsd is its fussyness over hardware in particular
network cards, time and time again I see posts here telling people to
go out buying expensive intel pro 1000 cards just so they can use the
operating system properly when I think its reasonable to expect
mainstream hardware to work, eg. realtek is mainstream and common as a
onboard nic but the support in freebsd is poor and only serving
datacentres to shy away from freebsd.  If the same hardware performs
better in linux then the hardware isnt to blame for worser performance
in fbsd.

Chris
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FreeBSD bind performance in FreeBSD 7

Tom Evans-3
On Fri, 2008-02-29 at 15:44 +0000, Chris wrote:

> On 29/02/2008, Ted Mittelstaedt <[hidden email]> wrote:
>
> A weakness of freebsd is its fussyness over hardware in particular
> network cards, time and time again I see posts here telling people to
> go out buying expensive intel pro 1000 cards just so they can use the
> operating system properly when I think its reasonable to expect
> mainstream hardware to work, eg. realtek is mainstream and common as a
> onboard nic but the support in freebsd is poor and only serving
> datacentres to shy away from freebsd.  If the same hardware performs
> better in linux then the hardware isnt to blame for worser performance
> in fbsd.
>
> Chris
Not to come down too hard on you, but the reason why Pro/1000 chipsets
are reasonably pricey, and uncommon to find as an integrated NIC, except
on server boards or intel own brand mobos, is that it is decent
hardware, and hence costs real money to use. Consumer NICs like Realtek,
Via Rhine and (imo) Marvell are cheap tat that 'just about' works, until
you put it under heavy stress. I have encountered a series of Marvell
based chips on my personal home computers that work about as well as a
slap around the face. Also, even from the 'good' manufacturers, like
broadcom and intel, you have 'consumer' parts, which are reasonably
cheap, like bge(4) supported parts, and 'professional' parts, like
bce(4) parts. One should work fine under moderate load, one should work
fine under heavy load. One will cost $4, one will cost $100.

I'm not saying the drivers are bug-free, but if you want performance and
reliability, you get an em(4) or another professional chipset. Only a
few months ago at work, we had to  order around 75 Pro/1000s as we had
had enough of crashes from our bce(4) based integrated NICs on our Dell
2950s. Fortunately for our wallet, we managed to fix the issues in the
driver/hardware before our supplier could source that many - thanks
David Christensen!

Personally, I wouldn't put something in a data-centre with only a vr(4)
or re(4), regardless of OS.

Tom


signature.asc (194 bytes) Download Attachment
123