InfiniTec - Henning Krauses Blog

Don't adjust your mind - it's reality that is malfunctioning

HttpWebRequest fails when several connections are made concurrently

Affected products

  • .NET Framework 1.0
  • .NET Framework 1.1

Summary

When making several connections concurrently with the HttpWebRequest class, the request objects fail with an exception.

Symptoms

When making several connections concurrently (more than 12 on a single processor machine) with the HttpWebRequest class, the request objects fail with an exception.
If these calls are made from methods running on ThreadPool thhreads, the maximum number of connections is further reduced.

Cause

The HttpWebRequest uses two threads for each connection from the ThreadPool. This pool has a maximal capacity of 25 on one processor machines. Regularly, if the pool is full, it would queue further requests. But the HttpWebRequest class checks whether two threads are available before putting its requests into the queue. It fails with an exception, when there are less than two threads available.

Status

This behaviour is by design.

Solution

Keep concurrent connections as low as possible, and don't initiate requests from methods running on ThreadPool threads.

Other information

To keep the concurrent connections low, you can use the Advanced Threadpool from the Toolbox area of this page. This class implements a thread pool, which is independent of the ThreadPool class. With this class, you can set the maximium number of concurrent threads, so that the limit of the ThreadPool is never reached.

Posted by Henning Krause on Friday, December 31, 2004 1:09 PM, last modified on Friday, December 31, 2004 1:10 PM
Permalink | Post RSSRSS comment feed

HttpWebRequest Class has poor performance

Affected Products

  • .NET Framework 1.0
  • .NET Framework 1.1

Summary

The HttpWebRequest class performs poorly when sending requests to a server which contain a body.

Symptoms

The HttpWebRequest class performs poorly when sending requests which contain body data to a server. When monitoring the network traffic the following behaviour is seen:
  • The client sends the header of the request to the server.
  • After 350ms the rest of the request is send to the server.

Cause

The HttpWebRequest class waits for a HTTP1.1/100-Continue message before sending the body of the request. Additionally, the class supports the Nagle algorithm, which prevents the sending of many small packets to the same target. Instead, the server waits approximately 200 milliseconds, if there are other packets to be send to the same target. It then combines these requests in one packet.

Status

Microsoft has confirmed that this is a problem in the Microsoft products that are listed at the beginning of this article.
A hotfix exists (QFE810814) for version 1.0 of the Framework in the languages english and german. The patch is localized for other languages on request. Since it is a quick fix engineering (QFE), it can only be obtained directly from Microsft Support (PSS).
For the .NET Framework 1.1, no patch is necessary, since the behaviour can be controlled programmatically:
HttpWebrequest webRequest = (HttpWebRequest) Webrequest.Create("http://myserver/myfile");
webRequest.ServicePoint.Expect100Continue = false;
webRequest.ServicePoint.UseNagleAlgorithm = false;

Other Information

Note that you must have installed service pack 2 for the .NET Framework 1.0 before applying the patch.
Additionally, some configuration entries must be made in the machine.config file:
  1. Create a new <settings> section within the <system.net> section with the following content:
    <settings>
    <servicePointManager useNagleAlgorithm="false" />
    <servicePointManager expect100Continue="false" />
    </settings>
  2. In order for the .NET Framework to recognize this new section a settings sectionGroup handler needs to be added to the <configSections> portion of machine.config file:
    <sectionGroup name="system.net" >
    <section name="settings" type="System.Net.Configuration.NetConfigurationHandler, System, Version=1.0.3300.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
    </sectionGroup>

Posted by Henning Krause on Thursday, December 30, 2004 6:36 PM, last modified on Thursday, July 14, 2011 10:32 PM
Permalink | Post RSSRSS comment feed