问题描述
本人遇到这个问题的环境:
Ubuntu16
CUDA8
Caffe
Matlab2015b
具体环境安装可见个人博客
在编译matlab版本caffe时遇到
安装正常,编译caffe正常,编译matcaffe正常,测试matcaffe也正常,但在用matcaffe训练模型时出现 :
You set your caffe in caffePath.cfg at: ../caffe/matlab
Cleared 0 solvers and 0 stand-alone nets
Setting to GPU mode, using device ID 0
WARNING: Logging before InitGoogleLogging() is written to STDERR
F1206 10:43:20.660126 15482 solver_factory.hpp:76] Check failed: registry.count(type) == 1 (0 vs. 1) Unknown solver type: SGD (known types: )
*** Check failure stack trace: ***
------------------------------------------------------------------------
abort() detected at Thu Dec 6 10:43:20 2018
------------------------------------------------------------------------
Configuration:
Crash Decoding : Disabled
Crash Mode : continue (default)
Current Graphics Driver: Unknown software
Current Visual : None
Default Encoding : UTF-8
GNU C Library : 2.23 stable
Host Name :*****
MATLAB Architecture : glnxa64
MATLAB Root : /usr/local/MATLAB/R2015b
MATLAB Version : 8.6.0.267246 (R2015b)
OpenGL : software
Operating System : Linux 4.13.0-36-generic #40~16.04.1-Ubuntu SMP Fri Feb 16 23:25:58 UTC 2018 x86_64
Processor ID : x86 Family 6 Model 158 Stepping 10, GenuineIntel
Virtual Machine : Java 1.7.0_60-b19 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode
Window System : No active display
Fault Count: 1
Abnormal termination:
abort()
Register State (from fault):
RAX = 0000000000000000 RBX = 00007f02e4cfc420
RCX = 00007f03652c7428 RDX = 0000000000000006
RSP = 00007f034a2a0b78 RBP = 00007f034a2a0e50
RSI = 0000000000003c7a RDI = 0000000000003c4c
R8 = 0000000000000081 R9 = 0000000000000000
R10 = 0000000000000008 R11 = 0000000000000202
R12 = 00007f02e4cfc480 R13 = 000000000000008e
R14 = 00007f02e4cfc420 R15 = 00007f02e4d03de0
RIP = 00007f03652c7428 EFL = 0000000000000202
CS = 0033 FS = 0000 GS = 0000
Stack Trace (from fault):
[ 0] 0x00007f03652c7428 /lib/x86_64-linux-gnu/libc.so.6+00218152 gsignal+00000056
[ 1] 0x00007f03652c902a /lib/x86_64-linux-gnu/libc.so.6+00225322 abort+00000362
[ 2] 0x00007f02e4ae7e49 /usr/lib/x86_64-linux-gnu/libglog.so.0+00040521
[ 3] 0x00007f02e4ae95cd /usr/lib/x86_64-linux-gnu/libglog.so.0+00046541
[ 4] 0x00007f02e4aeb433 /usr/lib/x86_64-linux-gnu/libglog.so.0+00054323 _ZN6google10LogMessage9SendToLogEv+00000643
[ 5] 0x00007f02e4ae915b /usr/lib/x86_64-linux-gnu/libglog.so.0+00045403 _ZN6google10LogMessage5FlushEv+00000187
[ 6] 0x00007f02e4aebe1e /usr/lib/x86_64-linux-gnu/libglog.so.0+00056862 _ZN6google15LogMessageFatalD2Ev+00000014
[ 7] 0x00007f02e56b47ff /home/digisky/MyProjects/LSTM_Pose_Machines/caffe/matlab/+caffe/private/caffe_.mexa64+00034815
[ 8] 0x00007f02e56b1811 /home/digisky/MyProjects/LSTM_Pose_Machines/caffe/matlab/+caffe/private/caffe_.mexa64+00022545 mexFunction+00000169
[ 9] 0x00007f0357b86c4a /usr/local/MATLAB/R2015b/bin/glnxa64/libmex.so+00142410 mexRunMexFile+00000090
[ 10] 0x00007f0357b83244 /usr/local/MATLAB/R2015b/bin/glnxa64/libmex.so+00127556
[ 11] 0x00007f0357b83de4 /usr/local/MATLAB/R2015b/bin/glnxa64/libmex.so+00130532
[ 12] 0x00007f035c124dbd /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_dispatcher.so+00724413 _ZN8Mfh_file16dispatch_fh_implEMS_FviPP11mxArray_tagiS2_EiS2_iS2_+00001549
[ 13] 0x00007f035c125250 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_dispatcher.so+00725584 _ZN8Mfh_file11dispatch_fhEiPP11mxArray_tagiS2_+00000032
[ 14] 0x00007f03567a08af /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+08612015
[ 15] 0x00007f03568c47ff /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+09807871
[ 16] 0x00007f03568ba254 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+09765460
[ 17] 0x00007f0356887901 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+09558273
[ 18] 0x00007f03564e9d6c /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+05766508
[ 19] 0x00007f03564d44c1 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+05678273
[ 20] 0x00007f03564e3075 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+05738613
[ 21] 0x00007f03566b4033 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+07643187
[ 22] 0x00007f03567900f4 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+08544500
[ 23] 0x00007f035c124dbd /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_dispatcher.so+00724413 _ZN8Mfh_file16dispatch_fh_implEMS_FviPP11mxArray_tagiS2_EiS2_iS2_+00001549
[ 24] 0x00007f035c125250 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_dispatcher.so+00725584 _ZN8Mfh_file11dispatch_fhEiPP11mxArray_tagiS2_+00000032
[ 25] 0x00007f0348b57675 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcos_impl.so+01222261
[ 26] 0x00007f0348c19f03 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcos_impl.so+02019075
[ 27] 0x00007f0348c1a1e2 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcos_impl.so+02019810
[ 28] 0x00007f0348ba4f4a /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcos_impl.so+01539914
[ 29] 0x00007f0348ba55c4 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcos_impl.so+01541572
[ 30] 0x00007f0348ba65d6 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcos_impl.so+01545686
[ 31] 0x00007f0348bae4e5 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcos_impl.so+01578213
[ 32] 0x00007f0348ba7892 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcos_impl.so+01550482
[ 33] 0x00007f0348ba7989 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcos_impl.so+01550729
[ 34] 0x00007f0348ba7fa5 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcos_impl.so+01552293
[ 35] 0x00007f0348ba8298 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcos_impl.so+01553048
[ 36] 0x00007f0348c23df7 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcos_impl.so+02059767
[ 37] 0x00007f0348cb2f7b /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcos_impl.so+02645883
[ 38] 0x00007f035c0cac57 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_dispatcher.so+00355415 _ZN13Mfh_MATLAB_fn11dispatch_fhEiPP11mxArray_tagiS2_+00000391
[ 39] 0x00007f0348cb0271 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcos_impl.so+02634353
[ 40] 0x00007f03567ba5fb /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+08717819
[ 41] 0x00007f035682c79d /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+09185181
[ 42] 0x00007f035682c29b /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+09183899
[ 43] 0x00007f0356833a3c /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+09214524
[ 44] 0x00007f0356833e28 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+09215528
[ 45] 0x00007f03568a3f81 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+09674625
[ 46] 0x00007f03564e9d6c /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+05766508
[ 47] 0x00007f03564d44c1 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+05678273
[ 48] 0x00007f03564e3075 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+05738613
[ 49] 0x00007f03566b4033 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+07643187
[ 50] 0x00007f03567923bc /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+08553404
[ 51] 0x00007f035c124b31 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_dispatcher.so+00723761 _ZN8Mfh_file16dispatch_fh_implEMS_FviPP11mxArray_tagiS2_EiS2_iS2_+00000897
[ 52] 0x00007f035c125250 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_dispatcher.so+00725584 _ZN8Mfh_file11dispatch_fhEiPP11mxArray_tagiS2_+00000032
[ 53] 0x00007f03567a08af /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+08612015
[ 54] 0x00007f03568c47ff /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+09807871
[ 55] 0x00007f03568ba47f /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+09766015
[ 56] 0x00007f0356887981 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+09558401
[ 57] 0x00007f03564e9d6c /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+05766508
[ 58] 0x00007f03564d44c1 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+05678273
[ 59] 0x00007f03564e3075 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+05738613
[ 60] 0x00007f03566b4033 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+07643187
[ 61] 0x00007f035667bc40 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+07412800
[ 62] 0x00007f035667e078 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+07422072
[ 63] 0x00007f035667e140 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+07422272
[ 64] 0x00007f03566f56bc /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+07911100
[ 65] 0x00007f03566f5abc /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_lxe.so+07912124
[ 66] 0x00007f035b655d0d /usr/local/MATLAB/R2015b/bin/glnxa64/libmwm_interpreter.so+02600205 _Z51inEvalCmdWithLocalReturnInDesiredWSAndPublishEventsRKSbIDsSt11char_traitsIDsESaIDsEEPibbP15inWorkSpace_tag+00000077
[ 67] 0x00007f035d250a12 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwiqm.so+00915986 _ZNK3iqm18InternalEvalPlugin24inEvalCmdWithLocalReturnERKSbIDsSt11char_traitsIDsESaIDsEEP15inWorkSpace_tag+00000098
[ 68] 0x00007f035d250bd8 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwiqm.so+00916440 _ZN3iqm18InternalEvalPlugin7executeEP15inWorkSpace_tagRN5boost10shared_ptrIN14cmddistributor17IIPCompletedEventEEE+00000120
[ 69] 0x00007f035c409695 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcr.so+00677525
[ 70] 0x00007f035d24a1c6 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwiqm.so+00889286
[ 71] 0x00007f035d237645 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwiqm.so+00812613
[ 72] 0x00007f0357db4bf9 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwbridge.so+00146425
[ 73] 0x00007f0357db51f4 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwbridge.so+00147956
[ 74] 0x00007f0357dba6cd /usr/local/MATLAB/R2015b/bin/glnxa64/libmwbridge.so+00169677
[ 75] 0x00007f0357dba7bc /usr/local/MATLAB/R2015b/bin/glnxa64/libmwbridge.so+00169916
[ 76] 0x00007f0357dbaead /usr/local/MATLAB/R2015b/bin/glnxa64/libmwbridge.so+00171693 _Z8mnParserv+00000749
[ 77] 0x00007f035c40bb4f /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcr.so+00686927 _ZN11mcrInstance30mnParser_on_interpreter_threadEv+00000031
[ 78] 0x00007f035c3f8443 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcr.so+00607299
[ 79] 0x00007f035c3f8a39 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcr.so+00608825 _ZN5boost6detail11task_objectIvNS_3_bi6bind_tIvPFvRKNS_8functionIFvvEEEENS2_5list1INS2_5valueIS6_EEEEEEE6do_runEv+00000025
[ 80] 0x00007f035c3f9f47 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcr.so+00614215 _ZN5boost6detail9task_baseIvE3runEv+00000071
[ 81] 0x00007f035c3f9fa7 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcr.so+00614311
[ 82] 0x00007f035c3f52fa /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcr.so+00594682
[ 83] 0x00007f035cc897ab /usr/local/MATLAB/R2015b/bin/glnxa64/libmwservices.so+01947563
[ 84] 0x00007f03507006ed /usr/local/MATLAB/R2015b/bin/glnxa64/libmwuix.so+00206573
[ 85] 0x00007f035cd8b2ba /usr/local/MATLAB/R2015b/bin/glnxa64/libmwservices.so+03003066
[ 86] 0x00007f035cd8b5f4 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwservices.so+03003892
[ 87] 0x00007f035cd8cd9f /usr/local/MATLAB/R2015b/bin/glnxa64/libmwservices.so+03009951
[ 88] 0x00007f035cd8d84c /usr/local/MATLAB/R2015b/bin/glnxa64/libmwservices.so+03012684 _Z25svWS_ProcessPendingEventsiib+00000092
[ 89] 0x00007f035c3f59b8 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcr.so+00596408
[ 90] 0x00007f035c3f5cd4 /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcr.so+00597204
[ 91] 0x00007f035c3e1fed /usr/local/MATLAB/R2015b/bin/glnxa64/libmwmcr.so+00516077
[ 92] 0x00007f03656636ba /lib/x86_64-linux-gnu/libpthread.so.0+00030394
[ 93] 0x00007f036539941d /lib/x86_64-linux-gnu/libc.so.6+01078301 clone+00000109
[ 94] 0x0000000000000000 <unknown-module>+00000000
This error was detected while a MEX-file was running. If the MEX-file
is not an official MathWorks function, please examine its source code
for errors. Please consult the External Interfaces Guide for information
on debugging MEX-files.
If this problem is reproducible, please submit a Service Request via:
http://www.mathworks.com/support/contact_us/
A technical support engineer might contact you with further information.
Thank you for your help.** This crash report has been saved to disk as /home/digisky/matlab_crash_dump.15436-1 **
MATLAB is exiting because of fatal error
matlab表现:
![在这里插入图片描述](https://i-blog.csdnimg.cn/blog_migrate/98e18db0b27c8b1bc9760ad0c02a4aee.png)
问题分析
如果用matlab调试的话,会发现死在:pSolver = caffe_(‘get_solver’, solver_file);
function solver = get_solver(solver_file)
% solver = get_solver(solver_file)
% Construct a Solver object from solver_file
CHECK(ischar(solver_file), 'solver_file must be a string');
CHECK_FILE_EXIST(solver_file);
pSolver = caffe_('get_solver', solver_file);
solver = caffe.Solver(pSolver);
end
也就是在读取solver文件时不认识SGD,没有注册SGD(当然也可能时Adam或其他,看自己用的什么算法)
问题解决
在matlab的caffe_.cpp注册SGD
打开caffe/matlab/+caffe/private/caffe_.cpp
在开头加上
#include "caffe/sgd_solvers.hpp"
namespace caffe {
INSTANTIATE_CLASS(AdaDeltaSolver);
REGISTER_SOLVER_CLASS(AdaDelta);
INSTANTIATE_CLASS(AdaGradSolver);
REGISTER_SOLVER_CLASS(AdaGrad);
INSTANTIATE_CLASS(AdamSolver);
REGISTER_SOLVER_CLASS(Adam);
INSTANTIATE_CLASS(NesterovSolver);
REGISTER_SOLVER_CLASS(Nesterov);
INSTANTIATE_CLASS(RMSPropSolver);
REGISTER_SOLVER_CLASS(RMSProp);
INSTANTIATE_CLASS(SGDSolver);
REGISTER_SOLVER_CLASS(SGD);
}
重新编译,OK