The following item is a must for every Visual C++ 6.0 programmer which uses operator new. The default behavior of operator new in Microsoft Visual C++ 6.0 in case of an error is to return NULL, in total contrast to the standard which explicitly states that new raises a bad_alloc exception upon failure. This behavior can lead to many unpleasant surprises, especially when trying to write portable code.

Here are some painful facts to feast your eyes with after you have written so much "error free" code with operator new in VC6:

  • Unlike VC6, Newer versions of Visual Studio conform to the exception throwing standard.
  • The STL is written according to the standard, therefore it assumes an exception will be thrown upon failure.
  • Platform independent code may end up on other machine types that conform to the standard.

Starting to clean the Mess

How is it possible to write safe operator new code in VC6?
After reading knowledge base article 167733 (1) about this issue, it seems that there is a simple solution to the problem. First, define a new_handler function and order it to throw an exception on any failure: 

#include <new>
#include <new.h>
#pragma init_seg(lib)
int my_new_handler(size_t) {
	throw std::bad_alloc();
return 0;
}

struct my_new_handler_obj {
	_PNH _old_new_handler;
	int _old_new_mode;

_tag_g_new_handler_obj() {
	_old_new_mode = _set_new_mode(1);
	_old_new_handler = _set_new_handler(my_new_handler);
}
~_tag_g_new_handler_obj() {
	_set_new_handler(_old_new_handler);
	set_new_mode(_old_new_mode);
}

} _g_new_handler_obj;

Note: _set_new_handler's syntax will not be explained in this item. However, the most important issue here is that my_new_handler will not be called unless you override the default behavior by calling set_new_mode(1) , which orders to alter the default behavior.

new(std::nothrow) Anomalies

The plot thickens. Be aware that using new(std::nothrow) does not shield you from exceptions when using your own new_handler, and still raises an exception upon failure. I shall write it again - new(std::nothrow) throws an exception when using _set_new_handler. This means that every piece of code in your binary that rightfully assumed new(std::nothrow) doesn't throw, will suddenly get exceptions - inner STL code, your own code, and third party code!

The anomaly is even worse than can be thought of - as it turns out in practice, in VC6, when linking the solution above by against the debug runtime libraries, std::nothrow behaves expectedly and does not throw an exception. However, when linking it against the release runtime libraries, operator new(std::nothrow) will throw an exception anyway!!

This irritating behavior in release configuration is a simple result of a compiler optimization that removes the try-catch statements around the new(std::nothrow) because of the reasonable assumption that std::nothrow should not, in theory, throw anything. When the compiler behaves according to the standard, and the runtime doesn't, don't be surprised to find a critical bug.
Needless to say, the worse thing about altering std::nothrow's behavior is that the STL itself uses it itself in many implementations.

In order to save you from the std::nothrow inferno, article 167733 also suggests defining your own operator new with std::nothrow when using set_new_handler, as follows: 

void *__cdecl operator new(size_t cb, const std::nothrow_t&) throw()
   {
    char *p;
    try {
        p = new char[cb];
    }
    catch (std::bad_alloc) {
        p = 0;
    }
    return p;
   }

The code above guarantees that new(std::nothrow) will never throw in your compiled module even if you have written a new_handler that throws, as required. Even when writing for high performance binaries, a few more op codes in your binary are worth the headaches of trying to understand why exceptions come out of nowhere.

The Party Never Ends

Now your code should conform to the standard, even on VC6, isn't it? Sadly enough, NOT EVER!

When using the runtime libraries as dlls, the new handler you have just defined also imposes itself on the global scope - on every other dll loaded in the program. The critical problem is that the operator new(std::nothrow) defined above is only recognized in the local compiled scope. Therefore, when an external dll you load uses new(std::nothrow) itself, its new(std::nothrow) will actually fire exceptions inside that dll (!). This is because on the one hand, the new_handler is global and defines throwing exceptions inside the runtime library's malloc, and on the other, the external dll doesn't link with your implementation of operator new(std::nothrow). Back to the bug's equilibrium - Fixing it in one place just makes it pop up in another place.

Conclusion

Trying to wrestle with this horrifying bug could be very annoying when only after a few days you discover that it is a known issue and in the September 2003 issue of MSDN Magazine, James Haben (2) has published an article exactly about this surprising anomaly.

Haben concluded that the mismatch in Visual C++ 6.0 between operator new, operator new(std::nothrow), and the STL could not be completely resolved. He himself found an implementation of STL that doesn't use std::nothrow and used it instead.

Even experienced programmers would be surprised how profound this bug can be. Unless you were aware of this anomaly, it probably affects almost every piece of C++ code you ever wrote in Visual Studio 6. However, as mentioned before, this bug is only relevant to those of us who really love VC6 and insist on working with it. Moving on to VS2003 or VS2005 completely frees you from all this nonsense, but be careful and make sure that the old code doesn't rely on returning NULL when new fails.