FBSD 1GBit router?

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

FBSD 1GBit router?

Bc. Radek Krejca
Hi,

  I have FreeBSD box as router
  FreeBSD pvt-gw.starnet.cz 6.1-RELEASE-p12 FreeBSD 6.1-RELEASE-p12 #2: Wed Jan 31 21:28:44 CET 2007     [hidden email]:/usr/obj/usr/src/sys/DL360-G4  i386
  But speed is only about 382 Mbit. I have following values in
  sysctl.conf:

net.inet.ip.fastforwarding=1
net.inet.tcp.recvspace=262144
net.inet.tcp.sendspace=262144
kern.ipc.maxsockbuf=33554432

  I need about 600-700 Mbit. Is any chance on freebsd? Hardware is HP
  DL360-G4, interrupt takes about 55 % of CPU. I tested it over
  netperf and result is about 382 Mbit.
 

--
Regards,
 Bc. Radek Krejca
 ICQ: 65895541


_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FBSD 1GBit router?

Zaphod Beeblebrox-2
On Feb 5, 2008 8:00 AM, Bc. Radek Krejca <[hidden email]> wrote:


>   I have FreeBSD box as router
>  FreeBSD pvt-gw.starnet.cz 6.1-RELEASE-p12 FreeBSD 6.1-RELEASE-p12 #2: Wed
> Jan 31 21:28:44 CET 2007     [hidden email]:/usr/obj/usr/src/sys/DL360-G4
>  i386
>  But speed is only about 382 Mbit. I have following values in
>  sysctl.conf:
>
> net.inet.ip.fastforwarding=1
> net.inet.tcp.recvspace=262144
> net.inet.tcp.sendspace=262144
> kern.ipc.maxsockbuf=33554432
>

The ip.fastforwarding makes a tiny insignificant difference with the caveat
that your box won't show up on traceroutes.  Fast forwarding is "fast" by
virtue of the fact that it doesn't decrement TTL.

The other sysctl values effect traffic originating or termating at your
router --- they probably have no effect on performance.


>   I need about 600-700 Mbit. Is any chance on freebsd? Hardware is HP
>  DL360-G4, interrupt takes about 55 % of CPU. I tested it over
>  netperf and result is about 382 Mbit.


If you need large packet performance of 600-700 mbit  (large  in this case
being packets (on average) of 1000 bytes or more), then this hardware is
likely doable.  I'd do the following:

- upgrade at least to 6.3.  upgrading to 7.0 might also be better, depending
on hardware choices
- ensure your ethernet cards are on fast enough busses.  'em' (Intel Ether
Express 1000) flavor ports are my personal favorite
- enable polling (this will make a _huge_ difference by itself)
- your hardware is (likely) dual core.  Make sure every piece of hardware in
use doesn't involve any giant locks.  Under 6.x consider the mpsafenet
sysctl.  This is also a point on which 7.0 will shine.
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FBSD 1GBit router?

Stefan Lambrev-2
Zaphod Beeblebrox wrote:

> On Feb 5, 2008 8:00 AM, Bc. Radek Krejca <[hidden email]> wrote:
>
>
>  
>>   I have FreeBSD box as router
>>  FreeBSD pvt-gw.starnet.cz 6.1-RELEASE-p12 FreeBSD 6.1-RELEASE-p12 #2: Wed
>> Jan 31 21:28:44 CET 2007     [hidden email]:/usr/obj/usr/src/sys/DL360-G4
>>  i386
>>  But speed is only about 382 Mbit. I have following values in
>>  sysctl.conf:
>>
>> net.inet.ip.fastforwarding=1
>> net.inet.tcp.recvspace=262144
>> net.inet.tcp.sendspace=262144
>> kern.ipc.maxsockbuf=33554432
>>
>>    
>
> The ip.fastforwarding makes a tiny insignificant difference with the caveat
> that your box won't show up on traceroutes.  Fast forwarding is "fast" by
> virtue of the fact that it doesn't decrement TTL.
>
> The other sysctl values effect traffic originating or termating at your
> router --- they probably have no effect on performance.
>
>
>  
>>   I need about 600-700 Mbit. Is any chance on freebsd? Hardware is HP
>>  DL360-G4, interrupt takes about 55 % of CPU. I tested it over
>>  netperf and result is about 382 Mbit.
>>    
>
>
> If you need large packet performance of 600-700 mbit  (large  in this case
> being packets (on average) of 1000 bytes or more), then this hardware is
> likely doable.  I'd do the following:
>
> - upgrade at least to 6.3.  upgrading to 7.0 might also be better, depending
> on hardware choices
> - ensure your ethernet cards are on fast enough busses.  'em' (Intel Ether
> Express 1000) flavor ports are my personal favorite
> - enable polling (this will make a _huge_ difference by itself)
>  
Let me disagree with this - while polling reduce CPU utilization it
doesn't perform better.
When the network is under pressure polling can lead to lost packets.
After all one should test how polling works for him, but with polling
enabled my intel network cards
does not work better.
> - your hardware is (likely) dual core.  Make sure every piece of hardware in
> use doesn't involve any giant locks.  Under 6.x consider the mpsafenet
> sysctl.  This is also a point on which 7.0 will shine.
> _______________________________________________
> [hidden email] mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-performance
> To unsubscribe, send any mail to "[hidden email]"
>  

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FBSD 1GBit router?

Mykola Dzham
In reply to this post by Zaphod Beeblebrox-2
 Zaphod Beeblebrox wrote:

> On Feb 5, 2008 8:00 AM, Bc. Radek Krejca <[hidden email]> wrote:
>
>
> >   I have FreeBSD box as router
> >  FreeBSD pvt-gw.starnet.cz 6.1-RELEASE-p12 FreeBSD 6.1-RELEASE-p12 #2: Wed
> > Jan 31 21:28:44 CET 2007     [hidden email]:/usr/obj/usr/src/sys/DL360-G4
> >  i386
> >  But speed is only about 382 Mbit. I have following values in
> >  sysctl.conf:
> >
> > net.inet.ip.fastforwarding=1
> > net.inet.tcp.recvspace=262144
> > net.inet.tcp.sendspace=262144
> > kern.ipc.maxsockbuf=33554432
> >
>
> The ip.fastforwarding makes a tiny insignificant difference with the caveat
> that your box won't show up on traceroutes.  Fast forwarding is "fast" by
> virtue of the fact that it doesn't decrement TTL.

This effect make other sysctl variable: net.inet.ip.stealth
net.inet.ip.fastforwarding described in inet(4).
I have appreciable increase in productivity at the Core2Duo processor
with several em network cards after enabling
net.inet.ip.fastforwarding. Before enabling fastforwarding "swi1: net"
use 100% of one cpu, the second core stood idle. After enabling
fastforwarding "emX taskq" processes use both cores.

--
Mykola Dzham, LEFT-(UANIC|RIPE)
JID: [hidden email]
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re[2]: FBSD 1GBit router?

Bc. Radek Krejca
In reply to this post by Zaphod Beeblebrox-2
Hi,

ZB> - upgrade at least to 6.3.  upgrading to 7.0 might also be better, depending
ZB> on hardware choices
ZB> - ensure your ethernet cards are on fast enough busses.  'em' (Intel Ether
ZB> Express 1000) flavor ports are my personal favorite
ZB> - enable polling (this will make a _huge_ difference by itself)
ZB> - your hardware is (likely) dual core.  Make sure every piece of hardware in
ZB> use doesn't involve any giant locks.  Under 6.x consider the mpsafenet
ZB> sysctl.  This is also a point on which 7.0 will shine.

I upgrade to 7.0RC3 but still the same. 418Mbit is the roof.

--
S pozdravem,
 Bc. Radek Krejca
 ICQ: 65895541


_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"

Reply | Threaded
Open this post in threaded view
|

Re: Re[2]: FBSD 1GBit router?

Adrian Chadd-2
On 25/02/2008, Bc. Radek Krejca <[hidden email]> wrote:

> Hi,
>
>  ZB> - upgrade at least to 6.3.  upgrading to 7.0 might also be better, depending
>  ZB> on hardware choices
>  ZB> - ensure your ethernet cards are on fast enough busses.  'em' (Intel Ether
>  ZB> Express 1000) flavor ports are my personal favorite
>  ZB> - enable polling (this will make a _huge_ difference by itself)
>  ZB> - your hardware is (likely) dual core.  Make sure every piece of hardware in
>  ZB> use doesn't involve any giant locks.  Under 6.x consider the mpsafenet
>  ZB> sysctl.  This is also a point on which 7.0 will shine.
>
>  I upgrade to 7.0RC3 but still the same. 418Mbit is the roof.

What PPS is that then?


Adrian


--
Adrian Chadd - [hidden email]
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"

Reply | Threaded
Open this post in threaded view
|

Re: FBSD 1GBit router?

Daniel Dias Gonçalves
In reply to this post by Bc. Radek Krejca
Bc. Radek Krejca escreveu:

> Hi,
>
> ZB> - upgrade at least to 6.3.  upgrading to 7.0 might also be better, depending
> ZB> on hardware choices
> ZB> - ensure your ethernet cards are on fast enough busses.  'em' (Intel Ether
> ZB> Express 1000) flavor ports are my personal favorite
> ZB> - enable polling (this will make a _huge_ difference by itself)
> ZB> - your hardware is (likely) dual core.  Make sure every piece of hardware in
> ZB> use doesn't involve any giant locks.  Under 6.x consider the mpsafenet
> ZB> sysctl.  This is also a point on which 7.0 will shine.
>
> I upgrade to 7.0RC3 but still the same. 418Mbit is the roof.
>
>  
Is recommendable set MPSAFENET to 1 in versions 6.x to improve the
performance?
and POLLING in the following system with 8 interfaces IF_EM ??

CPU: Intel(R) Xeon(TM) CPU 3.20GHz (3200.13-MHz 686-class CPU)
 Logical CPUs per core: 2
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
SMP: AP CPU #1 Launched!

Interfaces if_em:
   vendor     = 'Intel Corporation'
   device     = '82546EB Dual Port Gigabit Ethernet Controller'

--
Daniel

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FBSD 1GBit router?

Ingo Flaschberger
Hi,

>> ZB> - upgrade at least to 6.3.  upgrading to 7.0 might also be better,
>> depending
>> ZB> on hardware choices
>> ZB> - ensure your ethernet cards are on fast enough busses.  'em' (Intel
>> Ether
>> ZB> Express 1000) flavor ports are my personal favorite
>> ZB> - enable polling (this will make a _huge_ difference by itself)
>> ZB> - your hardware is (likely) dual core.  Make sure every piece of
>> hardware in
>> ZB> use doesn't involve any giant locks.  Under 6.x consider the mpsafenet
>> ZB> sysctl.  This is also a point on which 7.0 will shine.
>>
>> I upgrade to 7.0RC3 but still the same. 418Mbit is the roof.

older fbsd's are faster than newer.

How are the nic's connected to the cpu?

lspci -v

Kind regards,
  Ingo Flaschberger

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FBSD 1GBit router?

Kevin Oberman
> Date: Mon, 25 Feb 2008 15:05:18 +0100 (CET)

> From: Ingo Flaschberger <[hidden email]>
> Sender: [hidden email]
>
> Hi,
>
> >> ZB> - upgrade at least to 6.3.  upgrading to 7.0 might also be better,
> >> depending
> >> ZB> on hardware choices
> >> ZB> - ensure your ethernet cards are on fast enough busses.  'em' (Intel
> >> Ether
> >> ZB> Express 1000) flavor ports are my personal favorite
> >> ZB> - enable polling (this will make a _huge_ difference by itself)
> >> ZB> - your hardware is (likely) dual core.  Make sure every piece of
> >> hardware in
> >> ZB> use doesn't involve any giant locks.  Under 6.x consider the mpsafenet
> >> ZB> sysctl.  This is also a point on which 7.0 will shine.
> >>
> >> I upgrade to 7.0RC3 but still the same. 418Mbit is the roof.
>
> older fbsd's are faster than newer.
>
> How are the nic's connected to the cpu?
>
> lspci -v
V7 is not (in my experience) slower than V4, v5, or v6. I have run a lot
of tests at speeds MUCH higher than 1Gb. With 10Gb cards, I can sustain
transfer rates of over 9Gbps (assuming low RTT and suitable
hardware). 1Gbps is not even a challenge...even over a 100 ms. RTT.

Note that high throughput may require some tuning. Transmit and receive
windows need to be rather large if the RTT is very long at all. (See
"bandwidth-delay product" in Stevens or some other TCP reference.)
--
R. Kevin Oberman, Network Engineer
Energy Sciences Network (ESnet)
Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab)
E-mail: [hidden email] Phone: +1 510 486-8634
Key fingerprint:059B 2DDF 031C 9BA3 14A4  EADA 927D EBB3 987B 3751

attachment0 (232 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: FBSD 1GBit router?

Ingo Flaschberger
Dear Kevin,

>>>> I upgrade to 7.0RC3 but still the same. 418Mbit is the roof.
>>
>> older fbsd's are faster than newer.
>>
>> How are the nic's connected to the cpu?
>>
>> lspci -v
>
> V7 is not (in my experience) slower than V4, v5, or v6.

v6 is at least slower than v4.
http://www.tancsa.com/blast.html
(look at the table at the end)

> I have run a lot
> of tests at speeds MUCH higher than 1Gb. With 10Gb cards, I can sustain
> transfer rates of over 9Gbps (assuming low RTT and suitable
> hardware). 1Gbps is not even a challenge...even over a 100 ms. RTT.

You can route 9Gbps - or only source or sink 9gbps?
What packet size?
Whats the maximum pps (with 64byte packets)? (Thats the real interesting
value, not mbps)

I have a 1.2Ghz Pentium-M appliance, with 4x 32bit, 33MHz pci intel e1000
cards.
With maximum tuning I can "route" ~400mbps with big packets and ~80mbps
with 64byte packets.
around 100kpps, whats not bad for a pci architecture.

To reach higher bandwiths, better busses are needed.
pci-express cards are currently the best choice.
one dedicated pci-express lane (1.25gbps) has more bandwith than a whole
32bit, 33mhz pci-bus.

> Note that high throughput may require some tuning. Transmit and receive
> windows need to be rather large if the RTT is very long at all. (See
> "bandwidth-delay product" in Stevens or some other TCP reference.)

I'm not shure if he's using the nic for a server or for a router?

Kind regards,
  Ingo Flaschberger

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FBSD 1GBit router?

Chris Knight-2
In reply to this post by Bc. Radek Krejca
What kind of network cards are you using, and what bus type are they using?

>From my experience, I was never able to get over ~300Mbps routing from
one PCI-X NIC to another PCI-X NIC.  I switched out the bus to a
PCI-Express and put a couple of Intel PCI-E 4x gigabit cards in and I
was able to get over 950Mbps throughput.   Sorry I don't have my
sysctl.conf settings, that box was reallocated after we replaced it
with some Cisco junk.

-Chris

On Tue, Feb 5, 2008 at 5:00 AM, Bc. Radek Krejca <[hidden email]> wrote:

> Hi,
>
>   I have FreeBSD box as router
>   FreeBSD pvt-gw.starnet.cz 6.1-RELEASE-p12 FreeBSD 6.1-RELEASE-p12 #2: Wed Jan 31 21:28:44 CET 2007     [hidden email]:/usr/obj/usr/src/sys/DL360-G4  i386
>   But speed is only about 382 Mbit. I have following values in
>   sysctl.conf:
>
>  net.inet.ip.fastforwarding=1
>  net.inet.tcp.recvspace=262144
>  net.inet.tcp.sendspace=262144
>  kern.ipc.maxsockbuf=33554432
>
>   I need about 600-700 Mbit. Is any chance on freebsd? Hardware is HP
>   DL360-G4, interrupt takes about 55 % of CPU. I tested it over
>   netperf and result is about 382 Mbit.
>
>
>  --
>  Regards,
>   Bc. Radek Krejca
>   ICQ: 65895541
>
>
>  _______________________________________________
>  [hidden email] mailing list
>  http://lists.freebsd.org/mailman/listinfo/freebsd-performance
>  To unsubscribe, send any mail to "[hidden email]"
>
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FBSD 1GBit router? - solved

Bc. Radek Krejca
In reply to this post by Bc. Radek Krejca
Hi,

  thank you very much. Problem wasn't netither in version of bsd nor
  in configuration. Problem was in hardware. I change bge ethernet
  card to intel and it working.

--
S pozdravem,
 Bc. Radek Krejca
 ICQ: 65895541


_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FBSD 1GBit router?

Willem Jan Withagen-2
In reply to this post by Ingo Flaschberger
> I have a 1.2Ghz Pentium-M appliance, with 4x 32bit, 33MHz pci intel
> e1000 cards.
> With maximum tuning I can "route" ~400mbps with big packets and ~80mbps
> with 64byte packets.
> around 100kpps, whats not bad for a pci architecture.
>
> To reach higher bandwiths, better busses are needed.
> pci-express cards are currently the best choice.
> one dedicated pci-express lane (1.25gbps) has more bandwith than a whole
> 32bit, 33mhz pci-bus.

Like you say routing 400 Mb/s is close to the max of the PCI bus, which
has a theoretical max of 33*4*8 ~ 1Gbps. Now routing is 500Mb/s in,
500Mb/s out. So you are within 80% of the bus-max, not counting
memory-access and others.

PCI express will give you a bus per PCI-E device into a central hub,
thus upping the limit to the speed of the FrontSideBus in Intel
architectures. Which at the moment is a lot higher than what a single
PCI bus does.

What it does not explain is why you can only get 80Mb/s with 64byte
packets, which would suggest other bottlenecks than just the bus.

--WjW
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FBSD 1GBit router?

Ingo Flaschberger

>> I have a 1.2Ghz Pentium-M appliance, with 4x 32bit, 33MHz pci intel e1000
>> cards.
>> With maximum tuning I can "route" ~400mbps with big packets and ~80mbps
>> with 64byte packets.
>> around 100kpps, whats not bad for a pci architecture.
>>
>> To reach higher bandwiths, better busses are needed.
>> pci-express cards are currently the best choice.
>> one dedicated pci-express lane (1.25gbps) has more bandwith than a whole
>> 32bit, 33mhz pci-bus.
>
> Like you say routing 400 Mb/s is close to the max of the PCI bus, which
> has a theoretical max of 33*4*8 ~ 1Gbps. Now routing is 500Mb/s in, 500Mb/s
> out. So you are within 80% of the bus-max, not counting memory-access and
> others.

yes.

> PCI express will give you a bus per PCI-E device into a central hub, thus
> upping the limit to the speed of the FrontSideBus in Intel architectures.
> Which at the moment is a lot higher than what a single PCI bus does.

Thats why my next router will be based at this box:
http://www.axiomtek.com/products/ViewProduct.asp?view=429

Hopefully there will be direct memory bus connected nic's in future.
(HyperTransport connected nic's)

> What it does not explain is why you can only get 80Mb/s with 64byte packets,
> which would suggest other bottlenecks than just the bus.

Perhaps something with interrupts:
http://books.google.at/books?id=pr4fspaQqZkC&pg=PA144&lpg=PA144&dq=pci+interrupt+delay&source=web&ots=zbvVU2CgVx&sig=APe9YjdtK35ccnow7BDI2hzie7s&hl=de#PPA144,M1

MSI (Message-signalled Interrupts) are not very common on PCI
architekture; PCI-E use only MSI.

The kpps keept always around 100, equally if I used fast-forwarding,
fast-interrupts, or higher HZ values than 1000HZ.

But 100kpps is great for a router hardware of about 600eur.

Kind regards,
  Ingo Flaschberger

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FBSD 1GBit router?

Willem Jan Withagen-2
Ingo Flaschberger wrote:

>
>>> I have a 1.2Ghz Pentium-M appliance, with 4x 32bit, 33MHz pci
>>> intel e1000 cards. With maximum tuning I can "route" ~400mbps
>>> with big packets and ~80mbps with 64byte packets. around 100kpps,
>>> whats not bad for a pci architecture.
>>>
>>> To reach higher bandwiths, better busses are needed. pci-express
>>> cards are currently the best choice. one dedicated pci-express
>>> lane (1.25gbps) has more bandwith than a whole 32bit, 33mhz
>>> pci-bus.
>>
>> Like you say routing 400 Mb/s is close to the max of the PCI bus,
>> which has a theoretical max of 33*4*8 ~ 1Gbps. Now routing is
>> 500Mb/s in, 500Mb/s out. So you are within 80% of the bus-max, not
>> counting memory-access and others.
>
> yes.
>
>> PCI express will give you a bus per PCI-E device into a central
>> hub, thus upping the limit to the speed of the FrontSideBus in
>> Intel architectures. Which at the moment is a lot higher than what
>> a single PCI bus does.
>
> Thats why my next router will be based at this box:
> http://www.axiomtek.com/products/ViewProduct.asp?view=429

Nice piece of hardware.
Don't like the 2.5" one disk option though.

And not shure what to think of:
"Seven 10/100/1000Mbps (through PCI-E by one
interface) ports (RJ-45)"
Which seems to suggest everything comes in thru on PCI-E interface.
That than better have 8 or 16 lanes.

> Hopefully there will be direct memory bus connected nic's in future.
> (HyperTransport connected nic's)

Well that is going to be an AMD only solution, and I'm not even shure
that AMD would like to have other things than CPU's on that bus.

>
>> What it does not explain is why you can only get 80Mb/s with 64byte
>>  packets, which would suggest other bottlenecks than just the bus.
>
> Perhaps something with interrupts:
> http://books.google.at/books?id=pr4fspaQqZkC&pg=PA144&lpg=PA144&dq=pci+interrupt+delay&source=web&ots=zbvVU2CgVx&sig=APe9YjdtK35ccnow7BDI2hzie7s&hl=de#PPA144,M1
>
>
>
> MSI (Message-signalled Interrupts) are not very common on PCI
> architekture; PCI-E use only MSI.
>
> The kpps keept always around 100, equally if I used fast-forwarding,
>  fast-interrupts, or higher HZ values than 1000HZ.

MSI is not used for regular PCI busses.Could be that PCI-E does use it.
I believe youon that. But even than I'd like to know where the
bottleneck is in the 100kp/s limit with 64byte pakkets.

> But 100kpps is great for a router hardware of about 600eur.

I've seen routers 10 times that expensive, not able to that.

--WjW
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FBSD 1GBit router?

Ingo Flaschberger
>> Thats why my next router will be based at this box:
>> http://www.axiomtek.com/products/ViewProduct.asp?view=429
>
> Nice piece of hardware.
> Don't like the 2.5" one disk option though.
>
> And not shure what to think of:
> "Seven 10/100/1000Mbps (through PCI-E by one
> interface) ports (RJ-45)"
> Which seems to suggest everything comes in thru on PCI-E interface.
> That than better have 8 or 16 lanes.

Each 1000Mbps port is connected via 1 lane PCI-E, which is fast enough.
1 lane: 250Mbyte/sec  -> 2Gpbs

>> Hopefully there will be direct memory bus connected nic's in future.
>> (HyperTransport connected nic's)
>
> Well that is going to be an AMD only solution, and I'm not even shure
> that AMD would like to have other things than CPU's on that bus.
>
>>
>>> What it does not explain is why you can only get 80Mb/s with 64byte
>>>  packets, which would suggest other bottlenecks than just the bus.
>>
>> Perhaps something with interrupts:
>> http://books.google.at/books?id=pr4fspaQqZkC&pg=PA144&lpg=PA144&dq=pci+interrupt+delay&source=web&ots=zbvVU2CgVx&sig=APe9YjdtK35ccnow7BDI2hzie7s&hl=de#PPA144,M1
>>
>>
>>
>> MSI (Message-signalled Interrupts) are not very common on PCI architekture;
>> PCI-E use only MSI.
>>
>> The kpps keept always around 100, equally if I used fast-forwarding,
>>  fast-interrupts, or higher HZ values than 1000HZ.
>
> MSI is not used for regular PCI busses.Could be that PCI-E does use it.
> I believe youon that. But even than I'd like to know where the bottleneck is
> in the 100kp/s limit with 64byte pakkets.

As I also tested with polling (currently I use interface polling for the router)
and also reached only 100kpps, the bottleneck must be someting
different.

>> But 100kpps is great for a router hardware of about 600eur.
>
> I've seen routers 10 times that expensive, not able to that.

me too.

Kind regards,
  Ingo Flaschberger

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: FBSD 1GBit router?

dieter-7
In reply to this post by Willem Jan Withagen-2
> > Hopefully there will be direct memory bus connected nic's in future.
> > (HyperTransport connected nic's)
>
> Well that is going to be an AMD only solution, and I'm not even shure
> that AMD would like to have other things than CPU's on that bus.

There are FPGAs that plug into a CPU socket (for mainboards with
multiple CPU sockets).  There will be GPUs on the HyperTransport bus.
Putting a network controller there seems a tad extreme.
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"