TCP stack in FreeBSD poor performance in 100MB to Gigabit environment.

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

TCP stack in FreeBSD poor performance in 100MB to Gigabit environment.

David Kwan
 

I have a few questions regarding the TCP:

 

I have a situation with clients on a 100MB network connecting to servers
on a Gigabit network where the client read speeds are very slow from the
FreeBSD server and fast from the Linux server.    Write speeds from the
clients to both servers are fast.  (Clients on the gigabit network work
fine with blazing read and write speeds).  The network traces shows
congestion packets for both servers when doing reads from the clients
(dup acks and retransmissions), but the Linux server seem to handle the
congestion better.   ECN is not enabled on the network and I don't see
any congestion windowing or clients window changing.  The 100/1G switch
is dropping packets.   I double checked the network configuration and
also swapped swithports for the servers to use the others to make sure
the switch configuration are the same and the Linux always does better
than FreeBSD.  Assuming that the network configuration is a constant for
all clients and servers (speed, duplex, and etc...), the only variable
is the servers  themselves (Linux and FreeBSD).  I have tried a couple
of FreeBSD machines with 6.1 and 7.0 and they show the same problem,
with no luck matching the speed and network utilization of Linux (2
years old).  The read speed test I'm referring is doing transferring of
a 100MB file (cifs, nfs, and ftp), and the Linux server does it
constantly in around 10 sec (line speed) with a constant network
utilization chart, while the FreeBSD servers are magnitudes slower with
erratic network utilization chart.  I've attempted to tweak some network
sysctl  options on the FreeBSD, and the only ones that helped were
disabling TSO and inflight; which leads me to think that the
inter-packet gap was slightly increased to partially relieve congestion
on the switch; not a long term solution.

 

My questions are:

  1.  Have you heard of this problem before with 100MB clients to
Gigabit servers?

  2. Are you aware of any Linux fix/patch the TCP stack to better
handling congestion than FreeBSD?  I'm looking to address this issue in
the FreeBSD, but wondering if the Linux stack did something special that
can help with the FreeBSD performance.

 

David K.

 

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: TCP stack in FreeBSD poor performance in 100MB to Gigabit environment.

Hwa Hing
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


I also had such problem with FreeBSD-7 Stable-SMP with 4 port Intel
1GBit NIC em Driver and with Packet Filter, ALTQ enabled.

I tested with iperf from network a to network b. I found freebsd droping
packets and the transfer speed in routed mode is only 2 mbit/s compare
to linux router which able to forward the traffic at almost full speed.



David Kwan wrote:

>  
>
> I have a few questions regarding the TCP:
>
>  
>
> I have a situation with clients on a 100MB network connecting to servers
> on a Gigabit network where the client read speeds are very slow from the
> FreeBSD server and fast from the Linux server.    Write speeds from the
> clients to both servers are fast.  (Clients on the gigabit network work
> fine with blazing read and write speeds).  The network traces shows
> congestion packets for both servers when doing reads from the clients
> (dup acks and retransmissions), but the Linux server seem to handle the
> congestion better.   ECN is not enabled on the network and I don't see
> any congestion windowing or clients window changing.  The 100/1G switch
> is dropping packets.   I double checked the network configuration and
> also swapped swithports for the servers to use the others to make sure
> the switch configuration are the same and the Linux always does better
> than FreeBSD.  Assuming that the network configuration is a constant for
> all clients and servers (speed, duplex, and etc...), the only variable
> is the servers  themselves (Linux and FreeBSD).  I have tried a couple
> of FreeBSD machines with 6.1 and 7.0 and they show the same problem,
> with no luck matching the speed and network utilization of Linux (2
> years old).  The read speed test I'm referring is doing transferring of
> a 100MB file (cifs, nfs, and ftp), and the Linux server does it
> constantly in around 10 sec (line speed) with a constant network
> utilization chart, while the FreeBSD servers are magnitudes slower with
> erratic network utilization chart.  I've attempted to tweak some network
> sysctl  options on the FreeBSD, and the only ones that helped were
> disabling TSO and inflight; which leads me to think that the
> inter-packet gap was slightly increased to partially relieve congestion
> on the switch; not a long term solution.
>
>  
>
> My questions are:
>
>   1.  Have you heard of this problem before with 100MB clients to
> Gigabit servers?
>
>   2. Are you aware of any Linux fix/patch the TCP stack to better
> handling congestion than FreeBSD?  I'm looking to address this issue in
> the FreeBSD, but wondering if the Linux stack did something special that
> can help with the FreeBSD performance.
>
>  
>
> David K.
>
>  
>
> _______________________________________________
> [hidden email] mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-performance
> To unsubscribe, send any mail to "[hidden email]"
>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkhq5ngACgkQn94C8abtJZqjGQCgguGT6NG4LknjD3El+dnamlBv
Z1cAniBSYCJ7n2wi/MpWgRcdVhBWcMVT
=6jzX
-----END PGP SIGNATURE-----
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: TCP stack in FreeBSD poor performance in 100MB to Gigabit environment.

Kris Kennaway-3
Hwa Hing wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> I also had such problem with FreeBSD-7 Stable-SMP with 4 port Intel
> 1GBit NIC em Driver and with Packet Filter, ALTQ enabled.

But it doesn't sound like you are testing the same thing at all.

> I tested with iperf from network a to network b. I found freebsd droping
> packets and the transfer speed in routed mode is only 2 mbit/s compare
> to linux router which able to forward the traffic at almost full speed.

Check that the software isn't making invalid Linux-specific assumptions
about e.g. socket buffer sizes.

Kris

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: TCP stack in FreeBSD poor performance in 100MB to Gigabit environment.

Kris Kennaway-3
In reply to this post by David Kwan
David Kwan wrote:

>  
>
> I have a few questions regarding the TCP:
>
>  
>
> I have a situation with clients on a 100MB network connecting to servers
> on a Gigabit network where the client read speeds are very slow from the
> FreeBSD server and fast from the Linux server.    Write speeds from the
> clients to both servers are fast.  (Clients on the gigabit network work
> fine with blazing read and write speeds).  The network traces shows
> congestion packets for both servers when doing reads from the clients
> (dup acks and retransmissions), but the Linux server seem to handle the
> congestion better.   ECN is not enabled on the network and I don't see
> any congestion windowing or clients window changing.  The 100/1G switch
> is dropping packets.   I double checked the network configuration and
> also swapped swithports for the servers to use the others to make sure
> the switch configuration are the same and the Linux always does better
> than FreeBSD.  Assuming that the network configuration is a constant for
> all clients and servers (speed, duplex, and etc...), the only variable
> is the servers  themselves (Linux and FreeBSD).  I have tried a couple
> of FreeBSD machines with 6.1 and 7.0 and they show the same problem,
> with no luck matching the speed and network utilization of Linux (2
> years old).  The read speed test I'm referring is doing transferring of
> a 100MB file (cifs, nfs, and ftp), and the Linux server does it
> constantly in around 10 sec (line speed) with a constant network
> utilization chart, while the FreeBSD servers are magnitudes slower with
> erratic network utilization chart.  I've attempted to tweak some network
> sysctl  options on the FreeBSD, and the only ones that helped were
> disabling TSO and inflight; which leads me to think that the
> inter-packet gap was slightly increased to partially relieve congestion
> on the switch; not a long term solution.
>
>  
>
> My questions are:
>
>   1.  Have you heard of this problem before with 100MB clients to
> Gigabit servers?

The key point seems to be that the switch is dropping packets.  If you
have packet loss then TCP is not going to perform well.

Kris
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"