Boosting Windows XP Gig-E network performance
Today, most PCs, Â laptops and servers are equiped with 10/100/1000 Mbit/sec network adapters. Utilizing the extra bandwidth that becomes available when you upgrade a network from 100 Mbit/sec to 1Gbit/sec is a little bit harder than you might think. Those who expect an instant 10-fold increase in speed will be quickly disappointed, especially the users of Microsoft Windows. In my experience there isn’t that much performance increase at all when switching to Gigabit ethernet. Don’t despair though, this article has a few tricks to boost network performance to acceptable levels.
First, you will have to understand that the network will only go as fast as the data becomes available. If, for instance, you are using the network for filesharing, you are basically reading from a harddisk. Forget about achieving hunderds of Mb/sec’s here, that is simply not as fast as standard harddrives will go in the real world, let alone that the  filesharing protocol ([W:Server Message Block]) can keep up. Still, there is profit to be made by ensuring your network is running at full speed. This is why we’ll look at client-server network performance first. If you’re familiar with the [W:OSI model], my advice is to work through the layers bottom-up, that is starting with the physical layer.
Note that serious performance requires serious hardware. My laptop, which is equiped with a nice Broadcom NetXtreme 10/100/1000Mbit NIC isn’t capable of anything beyond 400-450 Mbit/sec. The computers  i’m using for these tests are equiped with server-grade mainboards and PCI-X NICs. These machines have seperate PCI busses to ensure the entire bus bandwidth is available for the NIC. Standard PCI cards only offer 133MB/sec, which is shared with other devices like graphic cards etc.
Network set-up
I recommend to do these tests by connecting the client and server with a simple cross-cable. This will help us with our optimizations:
- this eliminates active network components (switch!) from interfering with our tests
- this eliminates other network clients from interfering
- this eliminates (most) mis-configuration of the network
Once we’ve achieved decent performance levels, you can re-connect your network switch. This most likely will degrade performance again, which we’ll fix later on. For now, be smart and use a cross-cable.
Measuring network performance
Throughput is the amount of data that can be pushed through the network, usually measured in Megabits per second. The theoretical maximum is of course 1000 Mbit/sec, which equals to 125 Megabytes per second. Most PCs aren’t capable of sending or receiving such amounts of data, so achieving 1000 Mbit/sec is impossible with common hardware.
When two computers are communicating of the network they are most likely using TCP, the [W:Transmission Control Protocol]. A great tool for measuring network performance is iperf, which is very well capable of measuring TCP throughput. Also, iperf comes in Windows and Linux flavor. If you use a linux server (like i do), your not left in the dark. Optimizing a Windows XP client is a bit easier when using a linux server, usually linux has pretty decent Gig-E network performance out of the box. When using windows at the client and server you’ll have to tweak settings at both sides, at the same time.
Using iperf is easy, start iperf in ’server’ mode on one end:
ns:~/mega/mgr# iperf -s ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 1.00 MByte (default) ------------------------------------------------------------
This end is now waiting for an incoming connection for the client. Lets start a basic TCP throughput test right now on the client:
C:Documents and Settingsjay>iperf -c 192.168.3.3 ------------------------------------------------------------ Client connecting to 192.168.3.3, TCP port 5001 TCP window size: 8.00 KByte (default) ------------------------------------------------------------ [1912] local 192.168.3.17 port 1732 connected with 192.168.3.3 port 5001 [ ID] Interval    Transfer   Bandwidth [1912] 0.0-10.0 sec  190 MBytes  159 Mbits/sec
Optimizing ethernet packet size
The first step is to switch to increase the network packet size. The default for any ethernet network is 1500 bytes, which means all data that is sent across the network is broken up into packets that do not exceed over 1500 bytes. With high throughput, many packets per second are sent across the network, which puts quite a burden on your network equipment. If we increase the packet size, less packets have to be sent to transfer the same amount of data. This somewhat frees up resources on your network and helps to achieve more throughput. This maximum packet size is called the [W:Maximum transmission unit] size, or MTU size for short. Before switching MTU size, make sure all your network equipment (Switch + network interface cards) support this. Look for a feature called ‘jumbo packets’ or ‘large frames’, which indicates the hardware can handle packets of up to 9000 bytes.
The MTU size of Windows XP is controlled from the registry. Open up the registry editor (Start -> Run -> Open ‘regedit’ -> press ‘OK’). There is a nice article on Microsoft Technet that will guide us to the registry settings:Â http://support.microsoft.com/kb/314053
You don’t have to read it all the way, whats important for now is the section on how to find the correct registry location:
all the TCP/IP parameters are registry values that are located under one of two different subkeys of HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServices
TcpipParameters
TcpipParametersInterfacesID for AdapterNoteID for Adapter is the network adapter that TCP/IP is bound to. To determine the relationship between an Adapter ID and a network connection, view HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicseTcpipParametersInterfaces{4D36E972-E325-11CE-BFC1-08002BE10318}<ID for Adapter>Connection. The Name value in these keys provides the friendly name for a network connection that is used in the Network Connections folder. Values under these keys are specific to each adapter.
Basically, look up the network adapter name in the ‘Network Connections’ view in Windows Explorer. Next, locate the corresponding registery key by looking at the Connection ‘Name’ in the registry. My Gig-E adapter is named  ’Local Area Connection 3′ by Windows, which translates to registry key HKEY_LOCAL_MACHINESYSTEMCurrentControlSetControlNetwork{4D36E972-E325-11CE-BFC1-08002BE10318}{AE6DDF42-B712-4523-9118-B7D265C0A063}
Now, add a new DWORD value here, name it ‘MTU’ and set it to 9000 (decimal, not hex!). Instead of rebooting, disable and re-enable the network interface. Remember to do the same on the server if its running Windows XP.
When using linux, MTU size is controlled by ifconfig (or the more modern ip tools). Simply execute a ‘ifconfig eth0 mtu 9000′ to set the new MTU. Replace ‘eth0′ with your Gig-E network interface name.
More TCP testing
Update 7-4-2009: found a way around the 64K window size limit today. Automatic window scaling offers even greater performance. You can skip this section and read on in the ‘Enable TCP Window Scaling’ section.
TCP embodies a lot of features which have an effect on performance. We’ll try to control set-up these TCP features to our advantage. This has a positive effect because the default settings for Windows XP are rather poor for Gigabit Ethernet. Let’s retry our little TCP throughput test with the new MTU value:
C:Documents and Settingsjay>iperf -c 192.168.3.3 ------------------------------------------------------------ Client connecting to 192.168.3.3, TCP port 5001 TCP window size: 8.00 KByte (default) ------------------------------------------------------------ [1912] local 192.168.3.18 port 1559 connected with 192.168.3.3 port 5001 [ ID] Interval Transfer Bandwidth [1912] 0.0-10.0 sec 696 KBytes 568 Kbits/sec
Wow! We have completely killed off any performance! The problem here  is the default TCP window size of 8 Kb, which acts as a maximum ‘backlog’ for TCP packets. Because each packet now exceeds the window size (9000 bytes > 8Kb) the TCP mechanisms suffer and throughput plummets to the ground. I left this test in here to show how powerful these settings really are. Now lets set the maximum windows size, which is 64K:
C:Documents and Settingsjay>iperf -c 192.168.3.3 -w 64K ------------------------------------------------------------ Client connecting to 192.168.3.3, TCP port 5001 TCP window size: 64.0 KByte ------------------------------------------------------------ [1912] local 192.168.3.18 port 1573 connected with 192.168.3.3 port 5001 [ ID] Interval Transfer Bandwidth [1912] 0.0-10.0 sec 575 MBytes 481 Mbits/sec
Wow! we have now easily doubled throughput! You can play around to find the optimum window size for your set-up. Values which are even multiples of the MTU size work best. Once you’ve found your best window size, set a new TcpWindowSize:
Add by another registry key in HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicseTcpipParameter
Create a DWORD value, name it ‘TcpWindowSize’.
Set the value to the optimum WindowSize you found earlier (in decimal!)
Disable unused TCP/IP services
To push performance even further, open up the network adapter properties and look for the ‘QoS packet scheduler’ service.
Unless your planning to run [W:Quality of Service] on your network, your better off without this service. Â ninstall it for another 10% boost in performance. This works because with the QoS service, windows reserves about 10% of the available bandwidth for high-prio traffic.
If you really want to squeeze every last ‘bit’ out of your network, you can try to disable the Windows firewall and your Anti-Virus network scanner, if you have one. Ofcourse i don’t recommend to do this full-time, but it is nice for testing and to see what influence this has on performance. Lets re-do the TCP throughput test:
C:Documents and Settingsjay>iperf -c 192.168.3.3 -w 64k ------------------------------------------------------------ Client connecting to 192.168.3.3, TCP port 5001 TCP window size: 64.0 KByte ------------------------------------------------------------ [1912] local 192.168.3.16 port 2026 connected with 192.168.3.3 port 5001 [ ID] Interval Transfer Bandwidth [1912] 0.0-10.0 sec 670 MBytes 561 Mbits/sec
Amazingly we’ve boosted performance to almost four times as much throughput from where we started. Now reconnect anything you previously  disconnected from your network  (switch?) and retry the test. If the throughput plummets once more, check your switch and make sure it is set-up correctly to handle jumbo packets. Switches don’t care about window sizes, so don’t worry about that. If that doesn’t help, try to force your network cards to 1000Mbit/sec, turn flow-control on etc.., instead of relying on auto-negotiation.
Once your network can handle more through-put, the next step is to optimize the services you are planning on using. I’ll try to write an article on that the next time.
Enable TCP window scaling *update*
Today I found a nice wiki page which has instructions on how to enable TCP Window Scaling. This theoratically allows us to increase the window size up to a 1 Gbytes. First, we’ll have to enable RFC1323 window scaling, by adding another registry key:
Go to  HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParameter
Add a DWORD value, name it ‘Tcp1323Opts’, set value to 1
Add another DWORD value, name it ‘TcpWindowSize’, set value to 1048576 decimal (or 0×100000 hex)
Reboot
This enables TCP window size ‘negotiation’, which ensures the TCP window size is automatically scaled up when requested. With this new option, the TcpWindowSize determines the maximum window size, which can now go up to 1Gbytes. If your hardware is up to the job this results in even higher throughput:
C:Documents and Settingsjay>iperf -c 192.168.3.3 -N -t 60 -i 10 -w 1M ------------------------------------------------------------ Client connecting to 192.168.3.3, TCP port 5001 TCP window size: 1.00 MByte ------------------------------------------------------------ [1912] local 192.168.3.15 port 1203 connected with 192.168.3.3 port 5001 [ ID] Interval Transfer Bandwidth [1912] 0.0-10.0 sec 885 MBytes 742 Mbits/sec [1912] 10.0-20.0 sec 892 MBytes 748 Mbits/sec [1912] 20.0-30.0 sec 874 MBytes 734 Mbits/sec [1912] 30.0-40.0 sec 892 MBytes 748 Mbits/sec [1912] 40.0-50.0 sec 892 MBytes 748 Mbits/sec [1912] 50.0-60.0 sec 878 MBytes 736 Mbits/sec [1912] 0.0-60.0 sec 5.19 GBytes 743 Mbits/sec
The last step is to set the default TCP Window Size, by adding another registry key:
Go to HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesAFDParameters
Add a new DWORD value ‘DefaultSendWindow’ and set the value to the new Window Size (1048576 decimal)
This ensures all TCP sessions automatically allow window scaling up 1M. Try it out by running iperf without the ‘-w 1M’ option:
C:Documents and Settingsjay>iperf -c 192.168.3.3 ------------------------------------------------------------ Client connecting to 192.168.3.3, TCP port 5001 TCP window size: 1.00 MByte (default) ------------------------------------------------------------ [1912] local 192.168.3.15 port 1250 connected with 192.168.3.3 port 5001 [ ID] Interval Transfer Bandwidth [1912] 0.0-10.0 sec 880 MBytes 737 Mbits/sec
Summary
We have boosted network performance up to about 550 743 Mbit/sec instead of the  ~160 Mbit/sec we got using the Windows XP defaults. This was achieved by:
- Setting up your network hardware properly
- Increasing MTU size to 9000
- Increasing TCP window size
- Disabling unused TCP/IP services
- Enabling TCP window scaling
- Set the default TCP Send Window
Not all of these steps may have the same result on your network. Experiment and try to find what works and what doesn’t help at all.Â
Somehow, my hardware set-up refuses to go beyond 750 Mbit/sec. To me this is kinda strange, it seems that Windows XP is unable to go beyond that. In comparison, the mac mini running Debian Linux is able to crank out a whopping 990 MBit/sec, without any serious tweaking at all:
jay@minimac:~$ iperf -c 192.168.3.3 ------------------------------------------------------------ Client connecting to 192.168.3.3, TCP port 5001 TCP window size: 27.6 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.3.5 port 41909 connected with 192.168.3.3 port 5001 [ 3] 0.0-10.0 sec 1.15 GBytes 990 Mbits/sec
May 26th, 2009 at 18:35
Hi there
Just a quick question
Is it ok to uninstall the QOS ? I found out that you could change the bandwidth size from 20% and set it to zero, so you have maximum bandwidth.
You can check it out by going to:
Go to Run, type in gpedit.msc
Click on the Administrative Templates
Double-Click on Network
Double-Click on QoS Packet schedular
Then Double-Click on “Limit Reservable Bandwidth
Then from there you can set it from 20 to 0.
Then just click apply.
Sorry if I’m being a pain, I just wanted to know if that could maybe be done instead of Uninstalling anything
Thanks for a great article