根据工作需要,开始研究起了mongo。以前用过一些开源,也看过一些开源代码,也看过别人的源码分析,都是大神,甚是羡慕。上一篇中提到的几篇文章写的很好,突然就感觉到其实读代码写记录是最好的。今天从mongo开始,先从简单的来,中间有错误的话,欢迎大家指出,不胜感激。如果有幸被转载,请注明出处,熬夜写博客真心很痛苦。
先从mongo编译后生成的程序开始吧,我的是2.5.4的源码。刚开始编完后在bin目录中有很多可执行文件,比如mongo/mongos/mongod/mongostat/mongotop等,既然是可执行文件,一定有main函数,搜索之后就可以发现在哪些目录中。研究源码流程也就从main函数开始。
mongod是mongo的数据库存储程序,位于src/mongo/db/db.cpp。
主程序代码很简单:
int main(int argc, char* argv[], char** envp) {
int exitCode = mongoDbMain(argc, argv, envp);
::_exit(exitCode);
}
三个参数的main函数,第三个参数是环境变量,可以循环将其打印出来,是a=b型的键值对,需要继续解析才能使用。
mongoDbMain函数:
static int mongoDbMain(int argc, char* argv[], char **envp) {
static StaticObserver staticObserver;
getcurns = ourgetns;
setupSignalHandlers();
dbExecCommand = argv[0];
srand(curTimeMicros());
{
unsigned x = 0x12345678;
unsigned char& b = (unsigned char&) x;
if ( b != 0x78 ) {
out() << "big endian cpus not yet supported" << endl;
return 33;
}
}
if( argc == 1 )
cout << dbExecCommand << " --help for help and startup options" << endl;
mongo::runGlobalInitializersOrDie(argc, argv, envp);
startupConfigActions(std::vector<std::string>(argv, argv + argc));
cmdline_utils::censorArgvArray(argc, argv);
if (!initializeServerGlobalState())
::_exit(EXIT_FAILURE);
// Per SERVER-7434, startSignalProcessingThread() must run after any forks
// (initializeServerGlobalState()) and before creation of any other threads.
startSignalProcessingThread();
dataFileSync.go();
#if defined(_WIN32)
if (ntservice::shouldStartService()) {
ntservice::startService();
// exits directly and so never reaches here either.
}
#endif
StartupTest::runTests();
initAndListen(serverGlobalParams.port);
dbexit(EXIT_CLEAN);
return 0;
}
staticObserver虽然名字是静态观察,但实现很简单,具体作用还不清楚,大概是头文件中说的那样。
ourgentns是一个函数,获取当前的ns名,即db.table,会在log中调用。
setupSignalHandlers是个函数,设置信号的处理,reportEvent和UnhandledException的设置。
argv[0]是程序名字。 中间一段是测试是否是小端机器,目前不支持大端。
mongo::runGlobalInitializersOrDie(argc,argv,envp)主要解析命令行,即启动参数,其中会读取配置文件,解析参数并赋值给变量,比如“cpu”,“noauth”,“quota”,“repair path”,“nohints”等。 全局性质的初始化,上面的参数中有需要执行的命令也会在这里执行。在mongo/src/mongo/base/initializer.cpp。
for (size_t i = 0; i < sortedNodes.size(); ++i) {
InitializerFunction fn = _graph.getInitializerFunction(sortedNodes[i]);
if (!fn) {
return Status(ErrorCodes::InternalError,
"topSort returned a node that has no associated function: \"" +
sortedNodes[i] + '"');
}
status = fn(&context);
if (Status::OK() != status)
return status;
}
其中fn就是要执行的函数。
censor,文档说是一种算法:Look for an equal sign in arg。
initializeServerGlobalState,一些认证设置。
startSignalProcessingThread监听异步信号SIGUSR1,默认会创建boost::thread it( signalProcessingThread );
dataFileSync.go,定时同步文件。
runTests运行一些测试程序。
initAndListen初始化各个模块,并开始监听事件,MessageServer开始运行。
dbexit数据库退出。
其中dataFileSync的go函数是class BackgroundJob的函数。这个类是后台线程分发,并执行子类的run方法。boost::bind比较复杂,简单说就是绑定一些参数到函数上。
BackgroundJob& BackgroundJob::go() {
boost::thread t( boost::bind( &BackgroundJob::jobBody , this, _status ) );
return *this;
}
然后jobBody会执行run。更多BackgroundJob的信息参考头文件说明。
今天先到这里,明天继续分析完善。
现在重点看下initAndListen,这个函数开启了各种监听,包括socket,port,还有检测等。
void initAndListen(int listenPort) {
try {
_initAndListen(listenPort);
}
catch ( DBException &e ) {
log() << "exception in initAndListen: " << e.toString() << ", terminating" << endl;
dbexit( EXIT_UNCAUGHT );
}
catch ( std::exception &e ) {
log() << "exception in initAndListen std::exception: " << e.what() << ", terminating" << endl;
dbexit( EXIT_UNCAUGHT );
}
catch ( int& n ) {
log() << "exception in initAndListen int: " << n << ", terminating" << endl;
dbexit( EXIT_UNCAUGHT );
}
catch(...) {
log() << "exception in initAndListen, terminating" << endl;
dbexit( EXIT_UNCAUGHT );
}
}
参数是要监听的端口,在这传入的是serverGlobalParams.port,这里port的值是枚举类型的DefaultDBPort=27017.
void _initAndListen(int listenPort ) {
Client::initThread("initandlisten");
bool is32bit = sizeof(int*) == 4;
{
ProcessId pid = ProcessId::getCurrent();
LogstreamBuilder l = log();
l << "MongoDB starting : pid=" << pid
<< " port=" << serverGlobalParams.port
<< " dbpath=" << storageGlobalParams.dbpath;
if( replSettings.master ) l << " master=" << replSettings.master;
if( replSettings.slave ) l << " slave=" << (int) replSettings.slave;
l << ( is32bit ? " 32" : " 64" ) << "-bit host=" << getHostNameCached() << endl;
}
DEV log() << "_DEBUG build (which is slower)" << endl;
logStartupWarnings();
#if defined(_WIN32)
printTargetMinOS();
#endif
logProcessDetails();
{
stringstream ss;
ss << endl;
ss << "*********************************************************************" << endl;
ss << " ERROR: dbpath (" << storageGlobalParams.dbpath << ") does not exist." << endl;
ss << " Create this directory or give existing directory in --dbpath." << endl;
ss << " See http://dochub.mongodb.org/core/startingandstoppingmongo" << endl;
ss << "*********************************************************************" << endl;
uassert(10296, ss.str().c_str(), boost::filesystem::exists(storageGlobalParams.dbpath));
}
{
stringstream ss;
ss << "repairpath (" << storageGlobalParams.repairpath << ") does not exist";
uassert(12590, ss.str().c_str(),
boost::filesystem::exists(storageGlobalParams.repairpath));
}
// TODO check non-journal subdirs if using directory-per-db
checkReadAhead(storageGlobalParams.dbpath);
acquirePathLock(mongodGlobalParams.repair);
boost::filesystem::remove_all(storageGlobalParams.dbpath + "/_tmp/");
FileAllocator::get()->start();
// TODO: This should go into a MONGO_INITIALIZER once we have figured out the correct
// dependencies.
if (snmpInit) {
snmpInit();
}
MONGO_ASSERT_ON_EXCEPTION_WITH_MSG( clearTmpFiles(), "clear tmp files" );
dur::startup();
if (storageGlobalParams.durOptions & StorageGlobalParams::DurRecoverOnly)
return;
unsigned long long missingRepl = checkIfReplMissingFromCommandLine();
if (missingRepl) {
log() << startupWarningsLog;
log() << "** WARNING: mongod started without --replSet yet " << missingRepl
<< " documents are present in local.system.replset" << startupWarningsLog;
log() << "** Restart with --replSet unless you are doing maintenance and no"
<< " other clients are connected." << startupWarningsLog;
log() << "** The TTL collection monitor will not start because of this." << startupWarningsLog;
log() << "** For more info see http://dochub.mongodb.org/core/ttlcollections" << startupWarningsLog;
log() << startupWarningsLog;
}
if (mongodGlobalParams.scriptingEnabled) {
ScriptEngine::setup();
globalScriptEngine->setCheckInterruptCallback( jsInterruptCallback );
globalScriptEngine->setGetCurrentOpIdCallback( jsGetCurrentOpIdCallback );
}
// On replica set members we only clear temp collections on DBs other than "local" during
// promotion to primary. On pure slaves, they are only cleared when the oplog tells them to.
// The local DB is special because it is not replicated. See SERVER-10927 for more details.
const bool shouldClearNonLocalTmpCollections = !(missingRepl
|| replSettings.usingReplSets()
|| replSettings.slave == SimpleSlave);
repairDatabasesAndCheckVersion(shouldClearNonLocalTmpCollections);
if (mongodGlobalParams.upgrade)
return;
uassertStatusOK(getGlobalAuthorizationManager()->initialize());
/* this is for security on certain platforms (nonce generation) */
srand((unsigned) (curTimeMicros() ^ startupSrandTimer.micros()));
snapshotThread.go();
d.clientCursorMonitor.go();
PeriodicTask::startRunningPeriodicTasks();
if (missingRepl) {
// a warning was logged earlier
}
else {
startTTLBackgroundJob();
}
#ifndef _WIN32
mongo::signalForkSuccess();
#endif
if(getGlobalAuthorizationManager()->isAuthEnabled()) {
// open admin db in case we need to use it later. TODO this is not the right way to
// resolve this.
Client::WriteContext c("admin", storageGlobalParams.dbpath);
}
getDeleter()->startWorkers();
// Starts a background thread that rebuilds all incomplete indices.
indexRebuilder.go();
listen(listenPort);
// listen() will return when exit code closes its socket.
exitCleanly(EXIT_NET_ERROR);
}
Client::initThread用于创建一个数据库操作的cleint。
ProcessId::getCurrent()获取pid并检测是否32位机器。logStartupWarnings() logProcessDetails()记录一些log信息。
checkReadAhead()检查dbpath。
acquirePathLock,如果使用每一个db一个目录,则检查非journal的子目录。
remove_all删除db目录中的临时目录。
FileAllocator::get()->start()位于mongo/src/mongo/util/file_allocator.cpp.FileAllocator::get()是单例模式,start会执行run方法。
FileAllocator* FileAllocator::get(){
if ( ! _instance )
_instance = new FileAllocator();
return _instance;
}
void FileAllocator::start() {
boost::thread t( boost::bind( &FileAllocator::run , this ) );
}
void FileAllocator::run( FileAllocator * fa ) {
setThreadName( "FileAllocator" );
{
// initialize unique temporary file name counter
// TODO: SERVER-6055 -- Unify temporary file name selection
SimpleMutex::scoped_lock lk(_uniqueNumberMutex);
_uniqueNumber = curTimeMicros64();
}
while( 1 ) {
{
scoped_lock lk( fa->_pendingMutex );
if ( fa->_pending.size() == 0 )
fa->_pendingUpdated.wait( lk.boost() );
}
while( 1 ) {
string name;
long size;
{
scoped_lock lk( fa->_pendingMutex );
if ( fa->_pending.size() == 0 )
break;
name = fa->_pending.front();
size = fa->_pendingSize[ name ];
}
string tmp;
long fd = 0;
try {
log() << "allocating new datafile " << name << ", filling with zeroes..." << endl;
boost::filesystem::path parent = ensureParentDirCreated(name);
tmp = fa->makeTempFileName( parent );
ensureParentDirCreated(tmp);
#if defined(_WIN32)
fd = _open( tmp.c_str(), _O_RDWR | _O_CREAT | O_NOATIME, _S_IREAD | _S_IWRITE );
#else
fd = open(tmp.c_str(), O_CREAT | O_RDWR | O_NOATIME, S_IRUSR | S_IWUSR);
#endif
if ( fd < 0 ) {
log() << "FileAllocator: couldn't create " << name << " (" << tmp << ") " << errnoWithDescription() << endl;
uasserted(10439, "");
}
#if defined(POSIX_FADV_DONTNEED)
if( posix_fadvise(fd, 0, size, POSIX_FADV_DONTNEED) ) {
log() << "warning: posix_fadvise fails " << name << " (" << tmp << ") " << errnoWithDescription() << endl;
}
#endif
Timer t;
/* make sure the file is the full desired length */
ensureLength( fd , size );
close( fd );
fd = 0;
if( rename(tmp.c_str(), name.c_str()) ) {
const string& errStr = errnoWithDescription();
const string& errMessage = str::stream()
<< "error: couldn't rename " << tmp
<< " to " << name << ' ' << errStr;
msgasserted(13653, errMessage);
}
flushMyDirectory(name);
log() << "done allocating datafile " << name << ", "
<< "size: " << size/1024/1024 << "MB, "
<< " took " << ((double)t.millis())/1000.0 << " secs"
<< endl;
// no longer in a failed state. allow new writers.
fa->_failed = false;
}
catch ( const std::exception& e ) {
log() << "error: failed to allocate new file: " << name
<< " size: " << size << ' ' << e.what()
<< ". will try again in 10 seconds" << endl;
if ( fd > 0 )
close( fd );
try {
if ( ! tmp.empty() )
boost::filesystem::remove( tmp );
boost::filesystem::remove( name );
} catch ( const std::exception& e ) {
log() << "error removing files: " << e.what() << endl;
}
scoped_lock lk( fa->_pendingMutex );
fa->_failed = true;
// not erasing from pending
fa->_pendingUpdated.notify_all();
sleepsecs(10);
continue;
}
{
scoped_lock lk( fa->_pendingMutex );
fa->_pendingSize.erase( name );
fa->_pending.pop_front();
fa->_pendingUpdated.notify_all();
}
}
}
}
在run中,_pending存放的可能是文件的名字,并且创建临时目录。
dur::startup();
会开启journal线程,创建持久化目录和文件。
if (mongodGlobalParams.scriptingEnabled) {
ScriptEngine::setup();
globalScriptEngine->setCheckInterruptCallback( jsInterruptCallback );
globalScriptEngine->setGetCurrentOpIdCallback( jsGetCurrentOpIdCallback );
}
ScriptEngine::setup()会创建v8脚本引擎,在mongo/src/mongo/scripting/engine_v8.cpp
snapshotThread.go();
会开启快照线程,定时拍照,默认4秒。在/mongo/src/mongo/db/stats/snapshots.cpp
d.clientCursorMonitor::go();
d是全局的DGlobals,获得clientCursorMonitor()时会创建ClientCursorMonitor对象,在mongo/src/mongo/db/clientcursor.cpp,主要检测内存状态,默认一分钟一次。
PeriodicTask::startRunningPerioodicTasks();
启动定期执行任务的线程,创建PeriodicTaskRunner,检测shutdown信号,没有就等待,默认60秒。
if (missingRepl) {
// a warning was logged earlier
}
else {
startTTLBackgroundJob();
}
如果没有设置"--replSet"则之前会有警告,如果设置了这里会直行startTTLBackgroundJob();创建ttl监视,淘汰过期数据。
getDeleter()->startWorkers();
启动deleter线程,绑定RangeDeleter::doWork(),删除给定namespace的文档,可以immediate也可以lazy删除。
listen(listenPort);
这是监听socket的开始地方。
void listen(int port) {
//testTheDb();
MessageServer::Options options;
options.port = port;
options.ipList = serverGlobalParams.bind_ip;
MessageServer * server = createServer( options , new MyMessageHandler() );
server->setAsTimeTracker();
// we must setupSockets prior to logStartup() to avoid getting too high
// a file descriptor for our calls to select()
server->setupSockets();
logStartup();
startReplication();
if (serverGlobalParams.isHttpInterfaceEnabled)
boost::thread web( boost::bind(&webServerThread, new RestAdminAccess() /* takes ownership */));
#if(TESTEXHAUST)
boost::thread thr(testExhaust);
#endif
server->run();
}
这个监听结构也是消息模式的,类似于android的handler机制。createServer()是关键一句。options包含要监听的ip和port,MyMessageHandler是收到消息后处理的类,继承MessageHandler,返回的是新创建的ProtMessageServer的父类,初始化时就给变量赋值。其中Listener是一个抽象出来的监听类,setupSockets()中会创建,设置,添加socket。
还会启动log和replication。最后server的run开始真正进入了等待循环。用的select监听并处理数据。
run里面的accepted实际执行的是mongo/src/mongo/util/net/message_server_port.cpp中的acceptedMP().
try {
#ifndef __linux__ // TODO: consider making this ifdef _WIN32
{
HandleIncomingMsgParam* himParam = new HandleIncomingMsgParam(p, _handler);
boost::thread thr(boost::bind(&handleIncomingMsg, himParam));
}
.....
其中创建了HandleIncomingMsgParam,并绑定处理函数handleIncomingMsg和参数himParam。当有消息过来时就可以处理了。
先到这里吧。