Calling Conventions Demystified (Visual C++ calling conventions explained)

Long time no see these tedious 0CCCCCCCCh, "0cch" machine code for int 3, just a way to initialize the stack with int 3 for debug version:)

1. #   lea         edi,[ebp-0C0h] 
2. #   mov         ecx,30h 
3. #   mov         eax,0CCCCCCCCh 
4. #   rep stos    dword ptr [edi] 

Calling Conventions Demystified

By Nemanja Trifunovic

Visual C++ calling conventions explained

Introduction

During the long, hard, but yet beautiful process of learning C++ programming for Windows, you have probably been curious about the strange specifiers that sometime appear in front of function declarations, like __cdecl, __stdcall, __fastcall, WINAPI, etc. After looking through MSDN, or some other reference, you probably found out that these specifiers specify the calling conventions for functions. In this article, I will try to explain different calling conventions used by Visual C++ (and probably other Windows C/C++ compilers). I emphasize that above mentioned specifiers are Microsoft-specific, and that you should not use them if you want to write portable code.

So, what are the calling conventions? When a function is called, the arguments are typically passed to it, and the return value is retrieved. A calling convention describes how the arguments are passed and values returned by functions. It also specifies how the function names are decorated. Is it really necessary to understand the calling conventions to write good C/C++ programs? Not at all. However, it may be helpful with debugging. Also, it is necessary for linking C/C++ with assembly code.

To understand this article, you will need to have some very basic knowledge of assembly programming.

No matter which calling convention is used, the following things will happen:

1. All arguments are widened to 4 bytes (on Win32, of course), and put into appropriate memory locations. These locations are typically on the stack, but may also be in registers; this is specified by calling conventions.
2. Program execution jumps to the address of the called function.
3. Inside the function, registers ESI, EDI, EBX, and EBP are saved on the stack. The part of code that performs these operations is called function prolog and usually is generated by the compiler.
4. The function-specific code is executed, and the return value is placed into the EAX register.
5. Registers ESI, EDI, EBX, and EBP are restored from the stack. The piece of code that does this is called function epilog, and as with the function prolog, in most cases the compiler generates it.
6. Arguments are removed from the stack. This operation is called stack cleanup and may be performed either inside the called function or by the caller, depending on the calling convention used.

As an example for the calling conventions (except for this), we are going to use a simple function:

1. int sumExample (int a, int b)
2. {
3.     return a + b;
4. }

The call to this function will look like this:

1. int c = sum (2, 3);

For __cdecl, __stdcall, and __fastcall calling conventions, I compiled the example code as C (not C++). The function name decorations, mentioned later in the article, apply to the C decoration schema. C++ name decorations are beyond the scope of this article.

C calling convention (__cdecl)

This convention is the default for C/C++ programs (compiler option /Gd). If a project is set to use some other calling convention, we can still declare a function to use __cdecl:

1. int __cdecl sumExample (int a, int b);

The main characteristics of __cdecl calling convention are:

1. Arguments are passed from right to left, and placed on the stack.
2. Stack cleanup is performed by the caller.
3. Function name is decorated by prefixing it with an underscore character '_' .

Now, take a look at an example of a __cdecl call:

1. ; // push arguments to the stack, from right to left
2. push        3    
3. push        2    
4. 
   
5. ; // call the function
6. call        _sumExample 
7. 
   
8. ; // cleanup the stack by adding the size of the arguments to ESP register
9. add         esp,8 
10. 
    
11. ; // copy the return value from EAX to a local variable (int c)
12. mov         dword ptr [c],eax

The called function is shown below:

1. ; // function prolog
2.   push        ebp  
3.   mov         ebp,esp 
4.   sub         esp,0C0h 
5.   push        ebx  
6.   push        esi  
7.   push        edi  
8.   lea         edi,[ebp-0C0h] 
9.   mov         ecx,30h 
10.   mov         eax,0CCCCCCCCh 
11.   rep stos    dword ptr [edi] 
12.   
13. ; //    return a + b;
14.   mov         eax,dword ptr [a] 
15.   add         eax,dword ptr [b] 
16. 
    
17. ; // function epilog
18.   pop         edi  
19.   pop         esi  
20.   pop         ebx  
21.   mov         esp,ebp 
22.   pop         ebp  
23.   ret

Standard calling convention (__stdcall)

This convention is usually used to call Win32 API functions. In fact, WINAPI is nothing but another name for __stdcall:

1. #define WINAPI __stdcall

We can explicitly declare a function to use the __stdcall convention:

1. int __stdcall sumExample (int a, int b);

Also, we can use the compiler option /Gz to specify __stdcall for all functions not explicitly declared with some other calling convention.

The main characteristics of __stdcall calling convention are:

1. Arguments are passed from right to left, and placed on the stack.
2. Stack cleanup is performed by the called function.
3. Function name is decorated by prepending an underscore character and appending a '@' character and the number of bytes of stack space required.

The example follows:

1. ; // push arguments to the stack, from right to left
2.   push        3    
3.   push        2    
4.   
5. ; // call the function
6.   call        _sumExample@8
8. ; // copy the return value from EAX to a local variable (int c)  
9.   mov         dword ptr [c],eax

The function code is shown below:

1. ; // function prolog goes here (the same code as in the __cdecl example)
2. 
   
3. ; //    return a + b;
4.   mov         eax,dword ptr [a] 
5.   add         eax,dword ptr [b] 
6. 
   
7. ; // function epilog goes here (the same code as in the __cdecl example)
8. 
   
9. ; // cleanup the stack and return
10.   ret         8

Because the stack is cleaned by the called function, the __stdcall calling convention creates smaller executables than __cdecl, in which the code for stack cleanup must be generated for each function call. On the other hand, functions with the variable number of arguments (like printf()) must use __cdecl, because only the caller knows the number of arguments in each function call; therefore only the caller can perform the stack cleanup.

Fast calling convention (__fastcall)

Fast calling convention indicates that the arguments should be placed in registers, rather than on the stack, whenever possible. This reduces the cost of a function call, because operations with registers are faster than with the stack.

We can explicitly declare a function to use the __fastcall convention as shown:

1. int __fastcall sumExample (int a, int b);

We can also use the compiler option /Gr to specify __fastcall for all functions not explicitly declared with some other calling convention.

The main characteristics of __fastcall calling convention are:

1. The first two function arguments that require 32 bits or less are placed into registers ECX and EDX. The rest of them are pushed on the stack from right to left.
2. Arguments are popped from the stack by the called function.
3. Function name is decorated by by prepending a '@' character and appending a '@' and the number of bytes (decimal) of space required by the arguments.

Note: Microsoft have reserved the right to change the registers for passing the arguments in future compiler versions.

Here goes an example:

1. ; // put the arguments in the registers EDX and ECX
2.   mov         edx,3 
3.   mov         ecx,2 
4.   
5. ; // call the function
6.   call        @fastcallSum@8
7.   
8. ; // copy the return value from EAX to a local variable (int c)  
9.   mov         dword ptr [c],eax

Function code:

1. ; // function prolog
3.   push        ebp  
4.   mov         ebp,esp 
5.   sub         esp,0D8h 
6.   push        ebx  
7.   push        esi  
8.   push        edi  
9.   push        ecx  
10.   lea         edi,[ebp-0D8h] 
11.   mov         ecx,36h 
12.   mov         eax,0CCCCCCCCh 
13.   rep stos    dword ptr [edi] 
14.   pop         ecx  
15.   mov         dword ptr [ebp-14h],edx 
16.   mov         dword ptr [ebp-8],ecx 
17. ; // return a + b;
18.   mov         eax,dword ptr [a] 
19.   add         eax,dword ptr [b] 
20. ;// function epilog  
21.   pop         edi  
22.   pop         esi  
23.   pop         ebx  
24.   mov         esp,ebp 
25.   pop         ebp  
26.   ret

How fast is this calling convention, comparing to __cdecl and __stdcall? Find out for yourselves. Set the compiler option /Gr, and compare the execution time. I didn't find __fastcall to be any faster than other calling conventons, but you may come to different conclusions.

Thiscall

Thiscall is the default calling convention for calling member functions of C++ classes (except for those with a variable number of arguments).

The main characteristics of thiscall calling convention are:

1. Arguments are passed from right to left, and placed on the stack. this is placed in ECX.
2. Stack cleanup is performed by the called function.

The example for this calling convention had to be a little different. First, the code is compiled as C++, and not C. Second, we have a struct with a member function, instead of a global function.

1. struct CSum
2. {
3.     int sum ( int a, int b) {return a+b;}
4. };

The assembly code for the function call looks like this:

1.   push        3    
2.   push        2    
3.   lea         ecx,[sumObj] 
4.   call        ?sum@CSum@@QAEHHH@Z            ; CSum::sum 
5.   mov         dword ptr [s4],eax

The function itself is given below:

1.   push        ebp  
2.   mov         ebp,esp 
3.   sub         esp,0CCh 
4.   push        ebx  
5.   push        esi  
6.   push        edi  
7.   push        ecx  
8.   lea         edi,[ebp-0CCh] 
9.   mov         ecx,33h 
10.   mov         eax,0CCCCCCCCh 
11.   rep stos    dword ptr [edi] 
12.   pop         ecx  
13.   mov         dword ptr [ebp-8],ecx 
14.   mov         eax,dword ptr [a] 
15.   add         eax,dword ptr [b] 
16.   pop         edi  
17.   pop         esi  
18.   pop         ebx  
19.   mov         esp,ebp 
20.   pop         ebp  
21.   ret         8

Now, what happens if we have a member function with a variable number of arguments? In that case, __cdecl is used, and this is pushed onto the stack last.

Conclusion

To cut a long story short, we'll outline the main differences between the calling conventions:

• __cdecl is the default calling convention for C and C++ programs. The advantage of this calling convetion is that it allows functions with a variable number of arguments to be used. The disadvantage is that it creates larger executables.
• __stdcall is used to call Win32 API functions. It does not allow functions to have a variable number of arguments.
• __fastcall attempts to put arguments in registers, rather than on the stack, thus making function calls faster.
• Thiscall calling convention is the default calling convention used by C++ member functions that do not use variable arguments.

In most cases, this is all you'll ever need to know about the calling conventions.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

Nemanja Trifunovic

Born in Kragujevac, Serbia, and lived there until 2000. Spent four wonderful years in San Diego, California, one year in Boston, one in New York and now lives again in Boston area with his wife and two little daughters. Wrote his first program in 1984, became a professional developer after he graduated in 1994. After all these years still very passionate about programming and software development in general. Occupation: Software Developer Location: United States