A Bug in Custom ISAPI Filter Related to Keep-Alive

I once implemented my own Basic authentication logic in a custom ISAPI filter on IIS 6 and IIS 7.5 (classic mode). I met a bug related to keep-alive.

The custom filter checked the credentials ( Authorization: Basic <Base64 encoded name:password>) in the request header within the  CHttpFilter::OnPreprocHeaders function which I had overwritten. My code would send a  401 response to the client and return  SF_STATUS_REQ_FINISHED_KEEP_CONN when the credentials were invalid. 

The bug was: IIS would close TCP connection when the custom filter sent a 401 response to the client even if keep-alive was enabled. I had a client-side application which was written with .NET HttpWebRequest and HttpWebResponse. The application always threw an exception which told the server had closed the connection while keep-alive was expected. So why did the server close the connection?

After some time of investigation, I found this bug was caused by the incorrect behavior of the ISAPI filter itself. As MSDN introduced, by returning SF_STATUS_REQ_FINISHED_KEEP_CONN in CHttpFilter::OnPreprocHeaders, IIS would finish request handling and keep the TCP connection open if keep-alive was enabled. However, Microsoft did not implement this logic! Therefore, returning SF_STATUS_REQ_FINISHED_KEEP_CONN was exactly the same as returning SF_STATUS_REQ_FINISHED. The TCP connection was always closed! This behavior broke HTTP keep-alive protocol so that the client application failed.

Then I had to use a workaround to resolve this problem: adding “Connection: close” in the response header to explicitly tell the client the TCP connection would be closed by the server when I sent the 401 response to the client. I did not worry about the performance because HttpWebRequest.PreAuthenticate was set to true in most cases. That meant the client would receive the 401 response only at the first request. Then keep-alive would still work well for all the following thousands of requests. Here I want to complain why Microsoft did not add credentials to the first request even whenPreAuthenticate is true.

Recently, I abandoned the custom ISAPI filter and started using a custom HTTP module in IIS 7.5 Integrated mode to do Basic authentication. Of course, the problem went away when the client directly connected to the web server. However, the problem came back again when I used some reverse proxy device between the web server and the client, for example, the BigIP device. I did not find the root cause yet. But anyway, the “ Connection: close” workaround still worked well.