What is Stack Overflow?
http://en.wikipedia.org/wiki/Stack_overflow
One of my government customer reported they experienced the unexpected process termination which adversely impacted the end user experience. There seems to be no rule for the occurrence of this issue. The specific W3WP just ended and a new one was started upon the new incoming requests. I planned to use the adplus to capture the crash dump as process termination was usually caused by a fatal error which should be raised to the debugger as a second chance exception. The weird thing is: we can only get the 1st chance exception mini dump. But why? The symptom just told me the process was somehow terminated but didn’t give us any clues on why. So I set the debug break point on Kernel32!ExitProcess and Kernel32!TerminateProcess in order to find any clues on who did it. In the last reoccurrence, I was able to get a 2nd chance exception dump on Kernel32!TerminateProcess. After analyzing the dump, I found this was a Stack Overflow exception and that’s why adplus was unable to catch the 2nd chance exception dump.
Basically, each type of exception (such as an access violation or a stack overflow) can be raised to a debugger as either a first chance exception or a second chance exception. By definition, a first chance exception is non-fatal unless it is not handled correctly by using an error handler. If this problem occurs, the exception is raised again as a second chance exception (only a debugger can handle these). If no debugger handles a second chance exception, the application quits.
In this case, it is ASP.NET will take the StackOverflow exception as a fatal exception and handle it by simple terminating the process. That’s what we saw from the dump.
0:010> knL 10
# ChildEBP RetAddr
00 01cbaa18 7a0960b1 kernel32!TerminateProcess
01 01cbaa7c 7a0961e1 mscorwks!EEPolicy::HandleFatalStackOverflow+0xf2
02 01cbaaa0 7a00b617 mscorwks!EEPolicy::HandleStackOverflow+0x173
03 01cbaabc 79edd77a mscorwks!COMPlusFrameHandler+0x10b
04 01cbaad8 7c828752 mscorwks!COMPlusNestedExceptionHandler+0x65
05 01cbaafc 7c828723 ntdll!ExecuteHandler2+0x26
06 01cbaba4 7c82855e ntdll!ExecuteHandler+0x24
07 01cbaba4 77e4bee7 ntdll!KiUserExceptionDispatcher+0xe
08 01cbaef0 7a0363d7 kernel32!RaiseException+0x53
09 01cbaf0c 7a09af7e mscorwks!ReportStackOverflow+0x61
0a 01cbb090 79f53ab7 mscorwks!GetResourceStringFromManaged+0x44
0b 01cbb13c 799e7fe9 mscorwks!GetResourceFromDefault+0xb6
0c 01cbb15c 0384c6ee mscorlib_ni!System.ArgumentException.get_Message()+0x73f589
Checking the managed call stack, we can see the below calls were kept repeating:
OS Thread Id: 0x137c (10)
ESP EIP
01cbaa3c 77e42004 [FaultingExceptionFrame: 01cbaa3c] d:/nt/base/win32/client/process.c:4905
01cbb0e4 77e42004 [HelperMethodFrame_2OBJ: 01cbb0e4] System.Environment.GetResourceFromDefault(System.String) d:/nt/base/win32/client/process.c:4905
01cbb144 799e7fe9 System.ArgumentException.get_Message() f:/dd/ndp/clr/src/BCL/System/ArgumentException.cs:76
01cbb164 0384c6ee Common.Business.Components.CommonLogger.LogException(System.String, System.String, System.Exception)
01cbb180 03841265 Common.Business.Components.CommonManagement.FolderCreator(System.String)
01cbb224 038410d6 Common.Business.Components.CommonLogger.LogToFile(System.String, System.String, System.String)
01cbb26c 0384c77d Common.Business.Components.CommonLogger.LogException(System.String, System.String, System.Exception)
...
01cde280 03841265 Common.Business.Components.CommonManagement.FolderCreator(System.String)
01cde324 038410d6 Common.Business.Components.CommonLogger.LogToFile(System.String, System.String, System.String)
01cde36c 0384c77d Common.Business.Components.CommonLogger.LogException(System.String, System.String, System.Exception)
01cde388 03841265 Common.Business.Components.CommonManagement.FolderCreator(System.String)
01cde42c 038410d6 Common.Business.Components.CommonLogger.LogToFile(System.String, System.String, System.String)
01cde474 0384c77d Common.Business.Components.CommonLogger.LogException(System.String, System.String, System.Exception)
01cde490 0384c183 Common.Business.Components.CommonManagement.getCountryPrefix(System.String)
01cdf188 0384bacf MBox.Business.Data.MBoxDAC.InsertLog(Business.Entities.mREQ, Business.Entities.mCampaign, Business.Entities.mCampaignDetail[])
01cdf1ec 0384b96e MBox.Business.Components.MBoxMESManager.InsertLog(Business.Entities.mREQ, Business.Entities.mCampaign, Business.Entities.mCampaignDetail[])
01cdf224 03840f98 MBox.MBoxMES.Page_Load(System.Object, System.EventArgs)
…
With analyzing the code, we can tell the whole story. The thread generated an exception in function getCountryPrefix which in turn calls LogException. In LogException, we can see there was a function called LogToFile. It logged the exception information in files stored in the local disk. In the first steps of LogToFile, it naturally needed to examine if the folder did exist. If the folder didn’t exist, FolderCreator will create it. It all went smoothly until the catch (Exception exception) part. If there was any exception occurred in FolderCreator, the LogException would be called again. That’s why we see LogException, LogToFile and FolderCreator were called repeatedly.
public static void LogException(string filename, string filepath, Exception ex)
{
LogToFile(filename, filepath, "ExceptionMessage: " + ex.Message + Environment.NewLine + "StackTrace: " + ex.StackTrace + Environment.NewLine + "Source: " + ex.Source);
}
public static int LogToFile(string filename, string filepath, string s)
{
new CommonManagement().FolderCreator(filepath);
lock (intLock)
{
StreamWriter writer = null;
try
{
writer = File.AppendText(filepath + filename);
writer.WriteLine(DateTime.Now.ToString() + " " + s);
}
catch (Exception exception)
{
error = exception.Message;
}
finally
{
if (writer != null)
{
writer.Close();
writer = null;
}
}
return 1;
}
}
public void FolderCreator(string filepath)
{
try
{
if (!Directory.Exists(filepath))
{
Directory.CreateDirectory(filepath);
}
}
catch (Exception exception)
{
CommonLogger.LogException("CommantManagementErr.txt", ConfigurationManager.AppSettings.Get("errLogPath"), exception);
}
}
Solution:
==========
Use event log instead:
http://msdn.microsoft.com/en-us/library/system.diagnostics.eventlog.aspx