I'm experiencing some troubles in tuning Solaris 10 tcp parameters for
a bench I'm running.
I have a FTP server (RHEL 5.5 machine) in front of several Solaris and
Linux clients.
Server and clients are on the same LAN, the average RTT is 0.2 ms.
To be fair, I've raised the tcp_recv_hiwat and tcp_xmit_hiwat to a
value close to what the Linux clients advertise (approx. 750kB) and
set the tcp_max_buf to approx 4MB.
The test consists in transfering a 3.5 GB file on a local drive from
the ftp server. Server has a Gigabit Card.
The test were run several times by client, and all have Gigabit cards.
When running the Linux bench, I can download the file in about 45s
(average between the 5 runs, with no big variations).
On the Solaris side, the download takes average 78s.
I've network traced on the server the different tests, and
systematically, with Solaris I get a lot of tcp_zero_window.
Further analysis shows that, the client side (Solaris) sends acks by
segment (1440 bytes) instead of ack'ing bulks of segments. Which
automatically makes the tcp_window decrease.
The number of frames captured
On Linux client side, this have never happened during my 5 tests. The
network trace shows the the clients acks bulks of segments maintaining
its receive window at the same value.
Solaris shows this behaviour whatever hardware is used:
Solaris 10 10/08 on HP BL460 G6 (Blade) with bnx driver, 2 proc quad
core 2.83 Ghz
Solaris 10 10/07 on SUN X4100 with nvidia Giga fiber driver, 2 proc
quad dualcore 3,00 Ghz
pfiles on the ftp client show that it is using the right tcp window
values
* RCV_BUFF =* XMIT_BUFF = 747520
Detail of the tunables on Solaris side
tcp_sack_permitted = 2
tcp_deferred_acks_max = 2
tcp_wscale_always = 1
tcp_tmstamp_if_scale = 1
tcp_recv_hiwat = 747520
tcp_xmit_hiwat = 747520
Any idea is welcome.
Rgrds
Brem
PS: Linux client is based on the same BL460 G6 in the same enclosure
sharing the same network switch with the Solaris 10 client.
