Visual studio using Zeranoe FFmpeg libs and dlls crashes in release mode.

Francesco Pretto 2011-04-02 16:45:18 UTC
I having a crash with import *.libs generated by dlltool and used in
MSVC++ 2008 SP1, 32 bit compilation, Release target. The crash is a
page fault that triggers during DLL_PROCESS_ATTACH. I can reproduce
always and other users[1] have noticed the problem.


I have prepared a simple MSVC 2008 project[2] that manifests the
problem with libav* (ffmpeg) libs generated with dlltool (don't care
wrong references to libtool). It uses MSVC *default* settings for Release target. To reproduce:


1) Decompress;
2) Goto src directory, open the solution;
3) Select Release, compile, run --> crash;
4) Select Debug, compile, run --> OK.


Version here is GNU dlltool (GNU Binutils) 2.20.51.20100410


Follows the command used for *.libs generation:


SLIB_EXTRA_CMD=-'$(DLLTOOL) -m $(LIBTARGET) -d $$(@:$(SLIBSUF)=.def)
-l $(SUBDIR)$(SLIBNAME_WITH_MAJOR:$(SLIBSUF)=.lib) -D
$(SLIBNAME_WITH_MAJOR)'


Please, ask me if you need more info. Sorry if this has been reported earlier.


[1] http://ffmpeg.arrozcru.org/forum/viewtopic.php?f=15&t=1711&st=0&sk=t&sd=a&start=15#p5998
[2] http://robotics.dei.unipd.it/~fpretto/test-libtool-libs.zip
Comment 1 Francesco Pretto 2011-04-06 13:36:46 UTC
Adding the /OPT:NOREF option to the Release target sorts the problem. The option can be found in project Properties -> Configuration Properties -> Linker -> Optimization -> References -> Keep Unreferenced Data (/OPT:NOREF). It appears that /OPT:NOREF is the default in the Debug target. This would explain why the dlltool generated libs work in Debug target with default options. In Release target the default is /OPT:REF .


It still seems a dlltool problem because lib.exe (from the Visual Studio tools) is indeed able to produce *.libs that work correctly in both Debug and Release with (with Eliminate Unreferenced Data /OPT:REF enabled).
Comment 2 Martin Storsjö 2011-05-02 11:26:59 UTC
Created attachment 5700 [details]
Minimal test case


Here's an attached quite minimal testcase for this issue.


The makefile builds a tiny DLL using mingw/gcc, and generates an import library, both using --out-implib with ld and using dlltool. Then the makefile builds two test exes that links to the created import libraries - both exes run just fine on windows.


The batch script msvc.bat builds the same test exes using the MSVC command line tools, both with and without the /OPT:REF flag, against both of the created import libraries. The ones built with /OPT:NOREF work fine, while the ones built with /OPT:REF fail.


For comparison, it also creates an import library from the .def file using lib.exe, and links against it. For the exes built against this import library, both versions work.
Comment 3 Martin Storsjö 2013-05-09 17:12:19 UTC
For what it's worth, it seems that this issue has been fixed in the MSVC 2012 linker - there binaries linked with /OPT:REF against dlltool-produced import libraries seem to work just fine.