How to RESOLVE a breakpoint on a managed Main method.

Recently I've been playing with Rotor 2.0 (SSCLI 2.0). Previously I've already got a rolling build, but soon I have VS2010 installled and tried to build Rotor with it, which turned out to be a bad idea: when I struggled to get a PAL build successfully, a bad win32 image generated by VS2010 for binpalce tool.

 

I cannot afford wasting the time so I rolled back to VS2008.

 

Finally, I wrote a small program to test it.

using System; namespace Test { class Test { static int Main() { Console.WriteLine("this is a test"); while (true) { Aaa(1); int i = 0; int.TryParse(Console.ReadLine(), out i); if (i != 0) { break; } Bbb(5, 6); } return 0; } static int Aaa(int a) { Console.WriteLine("a = " + a); return a; } static int Bbb(int a, int b) { int c = a / b; Console.WriteLine("a / b = " + c); return c; } } }

 

It saved as test.cs. Then I used the newly generated C# compiler to compile it:

csc test.cs /debug+ /o- /pdb:test.pdb

 

Binary file test.exe generated.

 

Then I run:

 windbg clix test

 

to debug the program built with rotor.

 

Now windbg starts up, the following is its output:

Microsoft (R) Windows Debugger Version 6.12.0002.633 X86
Copyright (c) Microsoft Corporation. All rights reserved.

CommandLine: clix test
Symbol search path is: srv*
Executable search path is:
ModLoad: 00400000 00405000 clix.exe
(a24.1f4): Break instruction exception - code 80000003 (first chance)
eax=002d1eb4 ebx=7ffda000 ecx=00000006 edx=00000040 esi=002d1f48 edi=002d1eb4
eip=7c90120e esp=001afb20 ebp=001afc94 iopl=0 nv up ei pl nz na po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202
ntdll!DbgBreakPoint:
7c90120e cc int 3

 

Then I set the debugger to handle CLR notifications and stop on the load of mscorwks.dll:

sxe clrn sxe ld mscorwks

 

And after that I typed g to let the debugger go. A few seconds later, the debugger stops with the following out put:

ModLoad: 79200000 798b6000 Z:/sscli20/binaries.x86dbg.rotor/mscorwks.dll
eax=00000000 ebx=00000000 ecx=009a0000 edx=7c90e514 esi=00000000 edi=00000000
eip=7c90e514 esp=001af2d4 ebp=001af3c8 iopl=0 nv up ei ng nz ac pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000296
ntdll!KiFastSystemCallRet:
7c90e514 c3 ret

 

Now it's time to load the sos and set up the breakpoint in the program:

.loadby sos mscorwks

The sos just won't load. So I searched the web and found that the sos built with rotor depends on the VS2008 debug CRT. To resolve this problem,  I need to either modify the windbg.exe's manifest so that it loads the debug CRT when it starts up or just copy the very debug CRT into the sos's folder.

 

I copied the corresponding CRT files to the folder where sos residents, and tried to reload sos.  This very first time, I met a similar problem, however, after a second try, it loaded successfully. I dunno why.

 

 

I clicked the retry button on the previous dialog, and retyped the command.

 

0:000> .loadby sos mscorwks
The call to LoadLibrary(Z:/sscli20/binaries.x86dbg.rotor/sos) failed, HRESULT 0x80000003
"One or more arguments are invalid"
Please check your debugger configuration and/or network access.
0:000> .loadby sos mscorwks
0:000>

 

Now the sos successfully loaded. Let's go set up the break point.

!bpmd test Test.Test.Aaa

 

0:000>
0:000> !bpmd test Test.Test.Aaa
Adding pending breakpoints...
0:000> g
Breakpoint 0 hit
eax=001afcc0 ebx=7ffda000 ecx=00000000 edx=00000000 esi=00401282 edi=001afcd2
eip=792e0eb0 esp=001afc30 ebp=001afc4c iopl=0 nv up ei pl nz na po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202
mscorwks!_CorExeMain2:
792e0eb0 55 push ebp
0:000> g

 

At first a break point on mscorwks!_CorExeMain2 hit, not what we expected, just ignore it. Wow, there must be something wrong, it just ignores my break point...

0:000>
0:000> !bpmd test Test.Test.Aaa
Adding pending breakpoints...
0:000> g
Breakpoint 0 hit
eax=001afcc0 ebx=7ffda000 ecx=00000000 edx=00000000 esi=00401282 edi=001afcd2
eip=792e0eb0 esp=001afc30 ebp=001afc4c iopl=0 nv up ei pl nz na po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202
mscorwks!_CorExeMain2:
792e0eb0 55 push ebp
0:000> g

Caused by the previous gratuitous break point? Let's restart all over again.

Firstly, we clear all breakpoints:

(a24.784): Break instruction exception - code 80000003 (first chance)
eax=7ffda000 ebx=00000001 ecx=00000002 edx=00000003 esi=00000004 edi=00000005
eip=7c90120e esp=0396ffcc ebp=0396fff4 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=0038 gs=0000 efl=00000246
ntdll!DbgBreakPoint:
7c90120e cc int 3
0:003> .restart /f
CommandLine: clix test
Symbol search path is: srv*
Executable search path is:
(8d0.8b0): Break instruction exception - code 80000003 (first chance)
eax=002d1eb4 ebx=7ffdc000 ecx=00000006 edx=00000040 esi=002d1f48 edi=002d1eb4
eip=7c90120e esp=001afb20 ebp=001afc94 iopl=0 nv up ei pl nz na po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202
ntdll!DbgBreakPoint:
7c90120e cc int 3
0:000> bl
0 eu 0001 (0001) (mscorwks!_CorExeMain2)
0:000> bc 0

0:000> bl

 

OK, now we have no break points. Retype the previous commands.

 0:000> sxe clrn
0:000> sxe ld mscorwks
0:000> g
ModLoad: 79200000 798b6000 Z:/sscli20/binaries.x86dbg.rotor/mscorwks.dll
eax=00000000 ebx=00000000 ecx=009a0000 edx=7c90e514 esi=00000000 edi=00000000
eip=7c90e514 esp=001af2d4 ebp=001af3c8 iopl=0 nv up ei ng nz ac pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000296
ntdll!KiFastSystemCallRet:
7c90e514 c3 ret
0:000> .loadby sos mscorwks
0:000> !bpmd test Test.Test.Aaa
Adding pending breakpoints...
0:000> bl

Shoot! It seems there is no break point there? Is it the way pending breakpoint works? I doubt it.  Let's continue, anyway.

 

0:000> g
(8d0.324): Break instruction exception - code 80000003 (first chance)
eax=7ffdc000 ebx=00000001 ecx=00000002 edx=00000003 esi=00000004 edi=00000005
eip=7c90120e esp=0396ffcc ebp=0396fff4 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=0038 gs=0000 efl=00000246
ntdll!DbgBreakPoint:
7c90120e cc int 3
0:003> .restart /f
CommandLine: clix test
Symbol search path is: srv*
Executable search path is:
(c3c.9c8): Break instruction exception - code 80000003 (first chance)
eax=002d1eb4 ebx=7ffd5000 ecx=00000006 edx=00000040 esi=002d1f48 edi=002d1eb4
eip=7c90120e esp=001afb20 ebp=001afc94 iopl=0 nv up ei pl nz na po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202
ntdll!DbgBreakPoint:
7c90120e cc int 3

 

You can see that the pending break point is not eventually resolved. We've to restart again.

I've searched the web and found that someone else was experiencing the exactly same problem. And what's more, I saw that Microsoft guys said that it can be reproduced and they are investigating on it.

 

Someone suggests that I load the sos and set breakpoint after mscorjit loads. I've tried that, and it turns out that, mscorjit in rotor is renamed to mscorejt. I attempted to do so, and the behavior is still the same.  There must be something wrong there.

 

All of a sudden, I realized that I've the source of the sos. And I found that if the module cannot be found at the time point that the managed break point set, it saves the pending breakpoint info, and issues a notification command, which eventually does little with the pending breakpoint. Maybe we notification it waits never comes.

 

// strike.cpp line 3860. if (bNotification) { return HandleCLRNotificationEvent(); } // strike.cpp line 3943. // We didn't find a module, add a pending breakpoint, and // wait for the module load notification. g_bpoints.Add(ModuleName, FunctionName); bNeedNotificationExceptions = TRUE; // strike.cpp line 4009. if (bNeedNotificationExceptions) { ExtOut ("Adding pending breakpoints.../n"); sprintf(buffer,"sxe -c /"!bpmd -notification;g/" clrn"); Status = g_ExtControl->Execute(DEBUG_EXECUTE_NOT_LOGGED, buffer ,0); } // strike.cpp line 3764. HRESULT HandleCLRNotificationEvent() { HRESULT Status; /* * Did we get module load notification? If so, check if any in our pending list * need to be registered for jit notification. * * Did we get a jit notification? If so, check if any can be removed and * real breakpoints be set. */ DEBUG_LAST_EVENT_INFO_EXCEPTION dle; ULONG Type, ProcessId, ThreadId; ULONG ExtraInformationUsed; CNotification Notification; Status = g_ExtControl->GetLastEventInformation( &Type, &ProcessId, &ThreadId, &dle, sizeof(DEBUG_LAST_EVENT_INFO_EXCEPTION), &ExtraInformationUsed, NULL, 0, NULL); if (Status != S_OK) { ExtOut ("Error in GetLastEventInformation/n"); return Status; } if (Type != DEBUG_EVENT_EXCEPTION) { ExtOut ("Expecting an exception event/n"); return Status; } // Notification only needs to live for the lifetime of the call below, so it's a non-static // local. if (g_clrData->TranslateExceptionRecordToNotification(&dle.ExceptionRecord, &Notification) != S_OK) { ExtOut ("Error processing exception notification/n"); return Status; } return Status; }

 

 

Now we have two choices:

1. Modify the sos source to make it work.

2. Find another time point that the breakpoint can be resolved.

 

Definitely 1 is not my option, I cannot be biased that far away. So I added something in the code to block the program on IO, that why the Console.ReadLine() call be there.

 

Restart debugging and set breakpoint on Test.Test.Aaa and let go.

ModLoad: 79200000 798b6000 Z:/sscli20/binaries.x86dbg.rotor/mscorwks.dll
eax=00000000 ebx=00000000 ecx=009a0000 edx=7c90e514 esi=00000000 edi=00000000
eip=7c90e514 esp=001af2d4 ebp=001af3c8 iopl=0 nv up ei ng nz ac pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000296
ntdll!KiFastSystemCallRet:
7c90e514 c3 ret
0:000> .loadby sos mscorwks
0:000> g
Breakpoint 0 hit
eax=001afcc0 ebx=7ffd5000 ecx=00000000 edx=00000000 esi=00401282 edi=001afcd2
eip=792e0eb0 esp=001afc30 ebp=001afc4c iopl=0 nv up ei pl nz na po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202
mscorwks!_CorExeMain2:
792e0eb0 55 push ebp
0:000> g
(c3c.89c): Break instruction exception - code 80000003 (first chance)
eax=7ffd5000 ebx=00000001 ecx=00000002 edx=00000003 esi=00000004 edi=00000005
eip=7c90120e esp=0396ffcc ebp=0396fff4 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=0038 gs=0000 efl=00000246
ntdll!DbgBreakPoint:
7c90120e cc int 3
0:003> !bpmd test Test.Test.Aaa
Adding pending breakpoints...
0:003> g
(c3c.c70): Break instruction exception - code 80000003 (first chance)
eax=7ffd5000 ebx=00000001 ecx=00000002 edx=00000003 esi=00000004 edi=00000005
eip=7c90120e esp=0397ffcc ebp=0397fff4 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=0038 gs=0000 efl=00000246
ntdll!DbgBreakPoint:
7c90120e cc int 3

 

It dose not work either. Maybe we should add the .exe extension. Let try again.

0:003> !bpmd test.exe Test.Test.Aaa
Found 1 methods...
MethodDesc = 009f2d30
Setting breakpoint: bp 03879800 [Test.Test.Aaa(Int32)]
0:003> g
Breakpoint 1 hit
eax=009f2e04 ebx=7ffd5000 ecx=00000001 edx=00000000 esi=00000000 edi=001afcd2
eip=03879800 esp=001ae714 ebp=001ae738 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
03879800 55 push ebp
0:000> bl
0 e 792e0eb0 0001 (0001) 0:**** mscorwks!_CorExeMain2
1 e 03879800 0001 (0001) 0:****

 

Good, it works. The debugger stops. Let's verify whether it dose stop in the given method.

 

0:000> !clrstack
OS Thread Id: 0x9c8 (0)
ESP EIP
001ae714 03879800 Test.Test.Aaa(Int32)
001ae718 038afded Test.Test.Main()
001aef68 793662d5 [GCFrame: 001aef68]

 

Definitely, it stop there.  So let move on to find some where to place a native breakpoint so that we can get the managed break point on Main resolved.

 

0:000> kb
ChildEBP RetAddr Args to Child
WARNING: Frame IP not in any known module. Following frames may be wrong.
001ae710 038afded 00000001 00000000 00000000 0x3879800
001ae738 793662d5 001ae75c 79366354 001aeb74 0x38afded
001ae748 7937d5d2 001aecf0 00000000 001aecc0 mscorwks!CallDescrWorkerInternal+0x33
001aeb74 7937d4c7 001aecf0 00000000 001aecc0 mscorwks!CallDescrWorker+0xd2 [z:/sscli20/clr/src/vm/class.cpp @ 11285]
001aeca0 794a79b1 001aecf0 00000000 001aecc0 mscorwks!CallDescrWorkerWithHandler+0x187 [z:/sscli20/clr/src/vm/class.cpp @ 11198]
001aee5c 794a7296 009f2f00 001aef88 001aef58 mscorwks!MethodDesc::CallDescr+0x711 [z:/sscli20/clr/src/vm/method.cpp @ 1883]
001aee84 792f8114 009f2f00 001aef88 001aef58 mscorwks!MethodDesc::CallTargetWorker+0x46 [z:/sscli20/clr/src/vm/method.cpp @ 1572]
001aeea4 792f80b1 001aef58 00000000 b6464bac mscorwks!MethodDescCallSite::CallTargetWorker+0x34 [z:/sscli20/clr/src/vm/method.hpp @ 1804]
001aeef8 79361227 001aef58 b64655fc 001afcd2 mscorwks!MethodDescCallSite::Call_RetArgSlot+0x111 [z:/sscli20/clr/src/vm/method.hpp @ 1910]
001af0a8 7938853f 009f2d08 00000001 001af2f8 mscorwks!ClassLoader::RunMain+0x377 [z:/sscli20/clr/src/vm/clsload.cpp @ 5750]
001af30c 79301654 00000000 b6465dc8 003c0000 mscorwks!Assembly::ExecuteMainMethod+0x8f [z:/sscli20/clr/src/vm/assembly.cpp @ 1565]
001af89c 792e14e2 00000000 001af940 b6465e34 mscorwks!SystemDomain::ExecuteMainMethod+0x574 [z:/sscli20/clr/src/vm/appdomain.cpp @ 2195]
001afb60 792e0fd0 001afcc0 b6465978 001afcd2 mscorwks!ExecuteEXE+0x102 [z:/sscli20/clr/src/vm/ceemain.cpp @ 1734]
001afc2c 79e020c8 00000000 00000000 001afcc0 mscorwks!_CorExeMain2+0x120 [z:/sscli20/clr/src/vm/ceemain.cpp @ 1659]
001afc4c 00401ab0 00000000 00000000 001afcc0 sscoree!_CorExeMain2+0x78 [z:/sscli20/palrt/sscoree/sscoree_shims.h @ 145]
001afee0 00401ce9 001afcc0 00020cce 00000000 clix!Launch+0x2b0 [z:/sscli20/clr/src/tools/clix/clix.cpp @ 147]
001aff28 00401535 00000002 003c4fc8 001aff50 clix!PAL_startup_main+0x1c9 [z:/sscli20/clr/src/tools/clix/clix.cpp @ 267]
001aff38 79e89f56 001aff60 001affa8 79e89ef0 clix!run_main+0x15 [z:/sscli20/palrt/inc/palstartup.h @ 42]
001aff50 00401504 00401520 001aff60 00000002 rotor_pal!PAL_EntryPoint+0x26 [z:/sscli20/pal/win32/exception.c @ 690]
001aff68 00402048 00000002 003c4fc8 003c3498 clix!main+0x44 [z:/sscli20/palrt/inc/palstartup.h @ 70]
001affb8 00401e8f 001afff0 7c817077 00b9f6ee clix!__tmainCRTStartup+0x1a8 [f:/dd/vctools/crt_bld/self_x86/crt/src/crtexe.c @ 586]
001affc0 7c817077 00b9f6ee 00b9f71c 7ffd5000 clix!mainCRTStartup+0xf [f:/dd/vctools/crt_bld/self_x86/crt/src/crtexe.c @ 403]
001afff0 00000000 00401e80 00000000 78746341 kernel32!BaseProcessStart+0x23

 

 

It seems that when mscorwks!ClassLoader::RunMain is executing, the executable is loaded and the class that contains the Main method is loaded. Let's go and set a breakpoint on it and restart.

 

 0:000> .restart /f
CommandLine: clix test
Symbol search path is: srv*
Executable search path is:
(684.eb8): Break instruction exception - code 80000003 (first chance)
eax=002d1eb4 ebx=7ffd6000 ecx=00000006 edx=00000040 esi=002d1f48 edi=002d1eb4
eip=7c90120e esp=001afb20 ebp=001afc94 iopl=0 nv up ei pl nz na po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202
ntdll!DbgBreakPoint:
7c90120e cc int 3
0:000> bl
0 eu 0001 (0001) (mscorwks!_CorExeMain2)
0:000> bc 0
0:000> bl
0:000> sxe clrn
0:000> sxe ld mscorwks
0:000> g
ModLoad: 79200000 798b6000 Z:/sscli20/binaries.x86dbg.rotor/mscorwks.dll
eax=00000000 ebx=00000000 ecx=009a0000 edx=7c90e514 esi=00000000 edi=00000000
eip=7c90e514 esp=001af2d4 ebp=001af3c8 iopl=0 nv up ei ng nz ac pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000296
ntdll!KiFastSystemCallRet:
7c90e514 c3 ret
0:000> .loadby sos mscorwks
0:000> bp mscorwks!ClassLoader::RunMain
0:000> g
Breakpoint 0 hit
eax=009f2d08 ebx=7ffd6000 ecx=00000000 edx=001af2f8 esi=00401282 edi=001afcd2
eip=79360eb0 esp=001af0ac ebp=001af30c iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
mscorwks!ClassLoader::RunMain:
79360eb0 55 push ebp
0:000> !bpmd test Test.Test.Main
Adding pending breakpoints...
0:000> !bpmd test.exe Test.Test.Main
Found 1 methods...
MethodDesc = 009f2d08
Adding pending breakpoints...
0:000> !bpmd test.exe Test.Test.Aaa
Found 1 methods...
MethodDesc = 009f2d30
Adding pending breakpoints...
0:000> !bpmd test.exe Test.Test.Bbb
Found 1 methods...
MethodDesc = 009f2d58
Adding pending breakpoints...
0:000> g
(684.eb8): CLR notification exception - code e0444143 (first chance)
JITTED test!Test.Test.Main()
Setting breakpoint: bp 038AFD88 [Test.Test.Main()]
Breakpoint 1 hit
eax=009f2d08 ebx=7ffd6000 ecx=001aecf0 edx=001aecf0 esi=00401282 edi=001afcd2
eip=038afd88 esp=001ae73c ebp=001ae748 iopl=0 nv up ei pl nz na po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202
038afd88 55 push ebp
0:000> g
(684.eb8): CLR notification exception - code e0444143 (first chance)
JITTED test!Test.Test.Aaa(Int32)
Setting breakpoint: bp 03889800 [Test.Test.Aaa(Int32)]
Breakpoint 2 hit
eax=009f2d30 ebx=7ffd6000 ecx=00000001 edx=00000000 esi=00000000 edi=001afcd2
eip=03889800 esp=001ae714 ebp=001ae738 iopl=0 nv up ei pl nz ac po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000212
03889800 55 push ebp
0:000> g
(684.eb8): CLR notification exception - code e0444143 (first chance)
JITTED test!Test.Test.Bbb(Int32, Int32)
Setting breakpoint: bp 03971890 [Test.Test.Bbb(Int32, Int32)]
Breakpoint 3 hit
eax=009f2d58 ebx=7ffd6000 ecx=00000005 edx=00000006 esi=00000000 edi=001afcd2
eip=03971890 esp=001ae714 ebp=001ae738 iopl=0 nv up ei pl nz ac po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000212
03971890 55 push ebp
0:000> g
Breakpoint 2 hit
eax=009f2e04 ebx=7ffd6000 ecx=00000001 edx=00000000 esi=00000000 edi=001afcd2
eip=03889800 esp=001ae714 ebp=001ae738 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
03889800 55 push ebp
0:000> g
Breakpoint 3 hit
eax=009f2e08 ebx=7ffd6000 ecx=00000005 edx=00000006 esi=00000000 edi=001afcd2
eip=03971890 esp=001ae714 ebp=001ae738 iopl=0 nv up ei pl nz na po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202
03971890 55 push ebp
0:000> bl
0 e 79360eb0 0001 (0001) 0:**** mscorwks!ClassLoader::RunMain
1 e 038afd88 0001 (0001) 0:****
2 e 03889800 0001 (0001) 0:****
3 e 03971890 0001 (0001) 0:****

Now the debugger stops event on Main method. Done.