上一篇介绍了因为子表过多,导致innodb crash的情况,但crash的原因是long semaphore waits。
long semaphore waits又为何物?
背景:Innodb使用了mutex和rw_lock来保护内存数据结构,同步的机制要么是互斥,要么是读写阻塞的模式,
Innode认为mutex和rw_lock hold的时间足够短,所以,如果有线程wait mutex或者rw_lock时间过长,那么
很可能是程序有bug,所以就会异常主动crash。
关于innodb的锁:可以参考前面介绍的blog。
1. Innodb如何界定时间过长?
/*The following is the maximum allowed duration of a lock wait.*/UNIV_INTERN ulint srv_fatal_semaphore_wait_threshold= 600;
Innodb的认为600s足够长。
2. Innodb如何判断?
/*Create the thread which warns of long semaphore waits*/os_thread_create(&srv_error_monitor_thread, NULL,
thread_ids+ 3 + SRV_MAX_N_IO_THREADS);
innodb后台启动了一个线程,专门监控wait队列,
if (sync_array_print_long_waits(&waiter, &sema)&& sema == old_sema &&os_thread_eq(waiter, old_waiter)) {
fatal_cnt++;if (fatal_cnt > 10) {
fprintf(stderr,"InnoDB: Error: semaphore wait has lasted"
"> %lu seconds\n"
"InnoDB: We intentionally crash the server,"
"because it appears to be hung.\n",
(ulong) srv_fatal_semaphore_wait_threshold);
ut_error;
}
判断10次,如果是同一个线程,等待同一个sema,那么就crash掉Innodb。
for (i = 0; i < sync_primary_wait_array->n_cells; i++) {doublediff;void*wait_object;
cell=sync_array_get_nth_cell(sync_primary_wait_array, i);
wait_object= cell->wait_object;if (wait_object == NULL || !cell->waiting) {continue;
}
diff= difftime(time(NULL), cell->reservation_time);if (diff >SYNC_ARRAY_TIMEOUT) {
fputs("InnoDB: Warning: a long semaphore wait:\n",
stderr);
sync_array_cell_print(stderr, cell);
noticed=TRUE;
}if (diff >fatal_timeout) {
fatal=TRUE;
}if (diff >longest_diff) {
longest_diff=diff;*sema =wait_object;*waiter = cell->thread;
sync_primary_wait_array是一个数组,每一个wait sema的进入队列, 找到等待时间最长的,并且大于600s的,就设置fatal=TRUE。